Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,9 @@ PostgreSQL is on the roadmap behind the storage adapter boundary. For now,
non-SQLite `DATABASE_URL` values fail explicitly so deployments do not silently
write data to the wrong place.

See the [storage adapter boundary](docs/storage-adapter-boundary.md) for the
contract Ghost must satisfy before advertising non-SQLite backends.

> **Note:** Ghost works with just an OpenAI key. Additional keys unlock more modules.

## 📖 Usage
Expand Down
8 changes: 8 additions & 0 deletions V2_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,3 +143,11 @@ Added `ghost doctor --json` so readiness checks are no longer trapped in a Rich
Why this matters: cron, CI, API startup checks, and future hosted deploys can audit Ghost readiness without scraping terminal styling. The human `ghost doctor` table remains unchanged.

Verified with `.venv/bin/python -m pytest -q`, `.venv/bin/ruff check .`, `.venv/bin/ruff format --check .`, and `.venv/bin/ghost doctor --json`.

## Batch 11 — storage adapter boundary

Documented the storage adapter contract in `docs/storage-adapter-boundary.md` so the PostgreSQL roadmap has a clear engineering gate instead of becoming a vague claim.

This keeps Ghost honest: SQLite is supported today, unsupported `DATABASE_URL` schemes fail loudly today, and Postgres remains pending until the shared storage interface, contract tests, migrations, and CI coverage exist.

Verified with `.venv/bin/python -m pytest -q`, `.venv/bin/ruff check .`, `.venv/bin/ruff format --check .`, and `.venv/bin/ghost doctor --json`.
2 changes: 1 addition & 1 deletion docs/issue-1-response.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ What changed:

Current local verification:

- `pytest -q`: 56 passed.
- `pytest -q`: 57 passed.
- `ruff check .`: passed.
- `ruff format --check .`: passed.
- `ghost doctor --json`: ok, 0 errors, 0 warnings.
Expand Down
59 changes: 59 additions & 0 deletions docs/storage-adapter-boundary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Ghost storage adapter boundary

Ghost v2 currently supports SQLite as the durable local case-file store. PostgreSQL is a roadmap item for hosted and team deployments, but it should not be advertised as supported until the storage boundary below exists in code and tests.

## Authorized-use position

Storage work must preserve Ghost's defensive OSINT posture:

- Every investigation record keeps `authorized_use` and `scope`.
- Export/import flows do not strip provenance, errors, module metadata, or authorization context.
- New adapters must not introduce silent fallbacks that write case data to a different backend than the operator configured.
- Tests and demos should use synthetic or explicitly authorized targets.

## Adapter contract

A production storage adapter must support these operations before Ghost claims a non-SQLite backend:

- `init()`: create or migrate schema with an explicit schema version marker.
- `save_investigation(investigation)`: persist the complete investigation, findings, graph entities, relationships, errors, scope, and authorization metadata.
- `get_investigation(id)`: fetch a complete case file by exact investigation ID.
- `list_investigations(limit, offset)`: return newest-first case summaries without loading every finding.
- `get_graph_data(id)`: return graph nodes and links for the dashboard/API.
- `delete_investigation(id)`: delete an investigation and all dependent rows atomically.

## Non-negotiable behavior

- Unsupported `DATABASE_URL` schemes fail loudly during startup or doctor checks.
- Writes are transactional: partial case files are not acceptable.
- Deletes cascade to findings, entities, and relationships.
- Common lookup paths are indexed: investigation target, status, created/start time, finding investigation ID, entity investigation ID, and relationship investigation ID.
- `ghost doctor --json` reports backend readiness without scraping human terminal output.

## Operator verification gate

Before a demo, deployment, or release candidate, run:

```bash
ghost doctor --json
```

The `database` check is the source of truth for storage readiness. A configured
SQLite URL must resolve to the exact local database path Ghost will initialize
and write to. A non-SQLite URL must return a hard database error until a real
adapter implementation and contract test suite exist.

Do not treat a successful default SQLite doctor run as evidence that a custom
`DATABASE_URL` is supported. The configured URL itself must pass the doctor
check, otherwise Ghost could appear healthy while writing case data somewhere
other than the operator intended.

## PostgreSQL readiness checklist

PostgreSQL remains `PENDING` until all of these are true:

- Storage functions are behind an interface with SQLite and Postgres implementations.
- Contract tests run against SQLite and Postgres using the same fixtures.
- CI has a Postgres service job or a documented local integration gate.
- Migration/version handling is explicit for both adapters.
- README and roadmap distinguish "SQLite supported" from "Postgres supported" without ambiguity.
17 changes: 9 additions & 8 deletions ghost/backend/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,11 @@ def resolve_database_path(database_url: str) -> Path:
DB_PATH = resolve_database_path(config.database_url)


def get_connection() -> sqlite3.Connection:
if str(DB_PATH) != ":memory:":
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(str(DB_PATH), timeout=30)
def get_connection(database_path: Path | None = None) -> sqlite3.Connection:
path = database_path or DB_PATH
if str(path) != ":memory:":
path.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(str(path), timeout=30)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON")
Expand All @@ -68,8 +69,8 @@ def get_connection() -> sqlite3.Connection:


@contextmanager
def get_db():
conn = get_connection()
def get_db(database_path: Path | None = None):
conn = get_connection(database_path)
try:
yield conn
conn.commit()
Expand All @@ -80,9 +81,9 @@ def get_db():
conn.close()


def init_db():
def init_db(database_path: Path | None = None):
"""Create all tables if they don't exist."""
with get_db() as conn:
with get_db(database_path) as conn:
conn.executescript("""
CREATE TABLE IF NOT EXISTS investigations (
id TEXT PRIMARY KEY,
Expand Down
9 changes: 5 additions & 4 deletions ghost/core/doctor.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import shutil
from dataclasses import dataclass

from ghost.backend.db import DB_PATH, get_connection, init_db
from ghost.backend.db import DB_PATH, get_connection, init_db, resolve_database_path
from ghost.core.config import Config, config


Expand All @@ -26,10 +26,11 @@ def run_doctor_checks(config_override: Config | None = None) -> list[DoctorCheck
checks: list[DoctorCheck] = []

try:
init_db()
with get_connection() as conn:
database_path = resolve_database_path(cfg.database_url) if config_override else DB_PATH
init_db(database_path)
with get_connection(database_path) as conn:
conn.execute("SELECT 1").fetchone()
checks.append(DoctorCheck("database", True, str(DB_PATH), "error"))
checks.append(DoctorCheck("database", True, str(database_path), "error"))
except Exception as exc:
checks.append(DoctorCheck("database", False, str(exc), "error"))

Expand Down
16 changes: 16 additions & 0 deletions tests/test_investigator.py
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,22 @@ def test_doctor_cli_json_output(self):
assert "checks" in payload
assert "Ghost Doctor" not in result.output

def test_doctor_reports_unsupported_database_override(self):
from ghost.core.config import Config
from ghost.core.doctor import has_error, run_doctor_checks, summarize_doctor_checks

cfg = Config()
cfg.database_url = "postgresql://user:pass@localhost/ghost"

checks = run_doctor_checks(cfg)
database_check = next(check for check in checks if check.name == "database")

assert database_check.ok is False
assert database_check.severity == "error"
assert "Unsupported DATABASE_URL scheme 'postgresql'" in database_check.detail
assert has_error(checks) is True
assert summarize_doctor_checks(checks)["ok"] is False


# ── Report provenance ───────────────────────────────────────────────

Expand Down