Skip to content

Feature/persist findings to db#15

Merged
ionfwsrijan merged 10 commits into
ionfwsrijan:mainfrom
khushboo-khatoon:feature/persist-findings-to-db
Jun 2, 2026
Merged

Feature/persist findings to db#15
ionfwsrijan merged 10 commits into
ionfwsrijan:mainfrom
khushboo-khatoon:feature/persist-findings-to-db

Conversation

@khushboo-khatoon
Copy link
Copy Markdown
Contributor

Closes #2

What this PR does

After each scan completes, findings and job metadata are now persisted to SQLite. Both /scan and /scan-url endpoints write to the findings and jobs tables. A try/except wrapper ensures DB write failures only log a warning and never crash the scan response.

Type of change

  • Bug fix
  • New feature
  • ML model / training pipeline
  • Refactor (no behaviour change)
  • Documentation
  • Tests only

ML tier (if applicable)

  • Tier 1 — Triage
  • Tier 2 — Predictive
  • Tier 3 — Autonomous
  • Not ML-related

Changes

Backend

  • Updated /scan endpoint to insert a row into jobs and one row per finding into findings after scan completes
  • Updated /scan-url endpoint with the same DB write logic
  • Each finding gets a stable id via uuid4()
  • DB write wrapped in try/except — failure logs a warning, scan response is unaffected

New dependencies

  • uuid — built-in Python module, no install needed
  • logging — built-in Python module, no install needed

Database / schema changes

Testing

Started server locally, uploaded a ZIP — confirmed jobs and findings rows were written to patchpilot.db. Verified with sqlite3 patchpilot.db "SELECT * FROM findings LIMIT 5;".

Checklist

  • Tested locally end-to-end (upload ZIP or GitHub URL → scan → findings returned correctly)
  • No new console.error or unhandled Python exceptions introduced
  • requirements.txt / package.json updated if new dependencies added

Anything reviewers should focus on

The try/except block around DB writes in both /scan and /scan-url — ensures DB failure never affects API response.

@khushboo-khatoon
Copy link
Copy Markdown
Contributor Author

hey @ionfwsrijan

u can review and merge the PR , if all are okay .
thank u !

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds persistence of scan job metadata and aggregated scanner findings to a SQLite database after /scan and /scan-url complete, aligning with the project’s goal of building a durable dataset for later analysis/training.

Changes:

  • Adds backend/app/db.py with init_db() (schema creation) and get_db() connection helper.
  • Initializes the DB on FastAPI startup and attempts to write jobs + findings rows after each scan completes.
  • Documents the SQLite schema in backend/README.md.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 10 comments.

File Description
backend/README.md Adds documentation for the jobs / findings SQLite schema and expected field meanings.
backend/app/main.py Initializes DB on startup and adds post-scan persistence for /scan and /scan-url.
backend/app/db.py Introduces SQLite initialization and connection helpers using aiosqlite.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backend/app/db.py
Comment thread backend/app/db.py
Comment thread backend/app/main.py Outdated
Comment thread backend/app/main.py
Comment thread backend/app/main.py
Comment thread backend/app/main.py
Comment thread backend/app/main.py Outdated
Comment thread backend/app/main.py Outdated
Comment thread backend/app/db.py
Comment thread backend/app/db.py Outdated
@ionfwsrijan
Copy link
Copy Markdown
Owner

@khushboo-khatoon Still checks fail

@khushboo-khatoon
Copy link
Copy Markdown
Contributor Author

Hey @ionfwsrijan

I have fixed all the failing cases.
You may review and merge now.

Thank you for your help and patience throughout the process ((:

@ionfwsrijan
Copy link
Copy Markdown
Owner

ionfwsrijan commented Jun 2, 2026

@khushboo-khatoon Failing checks after merging with main

@ionfwsrijan ionfwsrijan merged commit ec78359 into ionfwsrijan:main Jun 2, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Persist findings to SQLite on every scan completion

3 participants