Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,9 @@ JARVIS_PORT=8001
JARVIS_ENV=development
JARVIS_STORE_PATH=data/jarvis-store.json
JARVIS_CORS_ORIGINS=*
# Optional: when set, require Authorization: Bearer <key> or X-API-Key

# Auth — required by default. Pick ONE:
# Production / any exposed port:
# JARVIS_API_KEY=
# Local loopback only (explicit opt-out):
JARVIS_ALLOW_UNAUTHENTICATED=1
43 changes: 29 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,19 @@
- **Service identity:** `jarvis-memoryboard` / schema `continuity-ledger-v1`
- **Default URL:** `http://127.0.0.1:8001`

This is **not** a claim of full CCS (Constitutional Continuity Service) readiness. See [docs/RELATIONSHIP_TO_MANDALA.md](docs/RELATIONSHIP_TO_MANDALA.md) and the [maturity scorecard](docs/scorecards/persistence-memory.md).
This is a **ledger of evidence records**, not associative/vector “memory.” See [docs/PLATFORM_LIMITS.md](docs/PLATFORM_LIMITS.md) and [docs/RELATIONSHIP_TO_MANDALA.md](docs/RELATIONSHIP_TO_MANDALA.md). Maturity: [docs/scorecards/persistence-memory.md](docs/scorecards/persistence-memory.md).

## Maturity (evidence-bound)

| Capability | Status | Evidence |
|------------|--------|----------|
| Continuity (A→B→C same restore) | **enforced** | `tests/test_acceptance.py::TestContinuityAcceptance` |
| Replay (why/where/when/session) | **enforced** | `TestReplayAcceptance` + `/api/jarvis/memory/retrieve` |
| Conflict (no silent merge) | **enforced** | `TestConflictAcceptance` + `/api/jarvis/memory/conflicts` |
| Drift (hash fidelity) | **partial** | hash check enforced; multi-day protocol operator-owned |
| Optional API key | **enforced** | `tests/test_auth.py` when `JARVIS_API_KEY` set |
| Conflict (no silent merge) | **enforced** (cross-agent by `subject`) | `TestConflictAcceptance`; groups all agents — not per-agent isolation |
| Drift (hash fidelity) | **partial** | hash check enforced; multi-day semantic protocol operator-owned |
| API key | **enforced** (required by default) | `tests/test_auth.py`; opt-out `JARVIS_ALLOW_UNAUTHENTICATED=1` |
| CCS / multi-product authority | **declared** | not implemented here |
| Multi-writer / HA store | **gap** | single JSON file; atomic write ≠ concurrent-safe |

### Drive-G-2 dimensions (summary)

Expand All @@ -26,49 +27,63 @@ This is **not** a claim of full CCS (Constitutional Continuity Service) readines
| Constitutional model | Moderate |
| Governance methodology | Moderate |
| Reference implementation | Moderate (local vertical slice + acceptance tests) |
| Platform engineering | Moderate (CI + Docker + optional auth; JSON file store, no HA) |
| Platform engineering | Moderate (CI + Docker + auth-by-default; JSON file store, no HA) |
| Commercial operations | Not started |

Full table: `docs/scorecards/persistence-memory.md`. Operator deploy: `docs/OPERATOR_DEPLOY_CHECKLIST.md`. Clause V hygiene: `docs/CLAUSE_V_HYGIENE.md` (**partial** / not API-enforced).
Full table: `docs/scorecards/persistence-memory.md`. Limits: `docs/PLATFORM_LIMITS.md`. Deploy: `docs/OPERATOR_DEPLOY_CHECKLIST.md`. Clause V: `docs/CLAUSE_V_HYGIENE.md` (**partial**).

## Quick start
## Quick start (local)

```powershell
python -m pip install -e ".[dev]"
# Local open auth (loopback only) — or set JARVIS_API_KEY instead:
$env:JARVIS_ALLOW_UNAUTHENTICATED = "1"
python -m app
# or
uvicorn app.main:app --host 127.0.0.1 --port 8001
python -m pytest -q
.\scripts\smoke-test.ps1
```

Docker:
Docker (set a key; do not rely on opt-out when publishing ports):

```bash
export JARVIS_API_KEY="$(python -c 'import secrets; print(secrets.token_hex(32))')"
docker compose up --build -d
```

## API (high level)

| Method | Path | Purpose |
|--------|------|---------|
| GET | `/health` | Liveness + schema |
| GET/POST/PATCH | `/api/jarvis/memory/board` | Board context |
| GET | `/health` | Liveness + schema (public) |
| GET/POST/PATCH | `/api/jarvis/memory/board` | Board context (UI slots ≠ write partitions) |
| GET/POST | `/api/jarvis/memory` | List / create ledger rows |
| GET | `/api/jarvis/memory/retrieve` | Memories + selections + conflicts |
| GET | `/api/jarvis/memory/conflicts` | Conflict sets by subject |
| GET | `/api/jarvis/memory/retrieve` | Explicit filters + selections + conflicts |
| GET | `/api/jarvis/memory/conflicts` | Conflict sets by **subject** (all `source_agent`s) |
| GET/PATCH/DELETE | `/api/jarvis/memory/{id}` | Row CRUD |

Create requires Continuity Ledger fields: `content`, `source_agent`, `session_id`, `type`, plus optional `confidence`, `evidence`, `status`, `subject`, `supersedes`, `tags`.

Retrieval is **filter/query/id** — not embedding similarity.

Legacy on-disk rows (`category` / `truth_status` / …) migrate on load.

## Auth

| Env | Role |
|-----|------|
| `JARVIS_API_KEY` | Required by default for protected routes |
| `JARVIS_ALLOW_UNAUTHENTICATED=1` | Explicit local-dev opt-out when no key |

Details: [SECURITY.md](SECURITY.md).

## Operator notes

- Set `JARVIS_API_KEY` when exposing beyond loopback ([SECURITY.md](SECURITY.md)).
- Prefer `JARVIS_API_KEY` over opt-out whenever the port may leave the machine.
- Tighten `JARVIS_CORS_ORIGINS` in shared networks.
- `JARVIS_ENV=production` disables uvicorn reload.
- Store path: `JARVIS_STORE_PATH` (atomic JSON writes).
- Store path: `JARVIS_STORE_PATH` (atomic JSON writes; single-writer assumption).

## License

Expand Down
23 changes: 22 additions & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,34 @@
Open a private security advisory on GitHub or contact the repository owner.
Do not commit secrets, API keys, or production store dumps.

## Authentication (required by default)

| Mode | Env | Behavior |
|------|-----|----------|
| **Default (secure)** | `JARVIS_API_KEY=<secret>` | Protected routes require `Authorization: Bearer <key>` or `X-API-Key` |
| **Local-dev opt-out** | `JARVIS_ALLOW_UNAUTHENTICATED=1` | Open routes when no key is set — **loopback / trusted host only** |
| **Misconfigured** | neither set | Protected routes return **401** (not open) |

Public paths (no key): `/`, `/health`, `/docs`, `/openapi.json`, `/redoc`.

If both are set, **`JARVIS_API_KEY` wins** — requests must present the key.

Do **not** use the opt-out when the port is forwarded, bound on a shared network, or exposed via Docker/publish without another auth layer.

Generate a key:

```powershell
python -c "import secrets; print(secrets.token_hex(32))"
```

## Operator hardening (baseline)

1. Set `JARVIS_API_KEY` when binding beyond loopback.
1. Set `JARVIS_API_KEY` for any non-throwaway deployment (required by default).
2. Set `JARVIS_CORS_ORIGINS` to explicit origins (never `*` in shared networks).
3. Set `JARVIS_ENV=production` (disables uvicorn reload).
4. Persist `/data` (or `JARVIS_STORE_PATH`) on durable volume; never commit store files.
5. Prefer TLS termination at a reverse proxy; this service speaks plain HTTP.
6. The ledger does **not** enforce multi-tenant isolation — one store per deployment.
7. Prefer `type=decision` (+ evidence) over chat dumps — Clause V hygiene is **partial** / not API-enforced (`docs/CLAUSE_V_HYGIENE.md`).
8. Follow `docs/OPERATOR_DEPLOY_CHECKLIST.md` before shared-network exposure.
9. Treat the JSON store as **single-writer** — atomic writes ≠ multi-writer safety (`docs/PLATFORM_LIMITS.md`).
52 changes: 40 additions & 12 deletions app/auth.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
"""Optional API-key gate for operator deployments.
"""API-key gate for Continuity Ledger deployments.

When JARVIS_API_KEY is unset, all routes remain open (local/dev default).
When set, non-public routes require Bearer or X-API-Key.
Default (secure): JARVIS_API_KEY must be set; protected routes require
Authorization: Bearer <key> or X-API-Key.

Local-dev opt-out: set JARVIS_ALLOW_UNAUTHENTICATED=1 to serve without a key
(open auth). Do not use the opt-out on shared or port-forwarded hosts.
"""

from __future__ import annotations
Expand All @@ -13,12 +16,19 @@
from starlette.requests import Request
from starlette.responses import JSONResponse, Response

_TRUTHY = frozenset({"1", "true", "yes", "on"})


def configured_api_key() -> str | None:
key = (os.getenv("JARVIS_API_KEY") or "").strip()
return key or None


def allow_unauthenticated() -> bool:
raw = (os.getenv("JARVIS_ALLOW_UNAUTHENTICATED") or "").strip().lower()
return raw in _TRUTHY


def extract_presented_key(request: Request) -> str | None:
auth = request.headers.get("Authorization") or ""
if auth.lower().startswith("bearer "):
Expand All @@ -33,16 +43,34 @@ def path_is_public(path: str) -> bool:
return path in {"/", "/health", "/docs", "/openapi.json", "/redoc"}


class OptionalApiKeyMiddleware(BaseHTTPMiddleware):
class ApiKeyMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next) -> Response:
if path_is_public(request.url.path):
return await call_next(request)

expected = configured_api_key()
if expected is None or path_is_public(request.url.path):
if expected is not None:
presented = extract_presented_key(request)
if presented is None or not hmac.compare_digest(presented, expected):
return JSONResponse(
status_code=401,
content={"detail": "Invalid or missing API key"},
)
return await call_next(request)

if allow_unauthenticated():
return await call_next(request)

presented = extract_presented_key(request)
if presented is None or not hmac.compare_digest(presented, expected):
return JSONResponse(
status_code=401,
content={"detail": "Invalid or missing API key"},
)
return await call_next(request)
return JSONResponse(
status_code=401,
content={
"detail": (
"API key required. Set JARVIS_API_KEY, or for local dev only "
"set JARVIS_ALLOW_UNAUTHENTICATED=1."
)
},
)


# Backward-compatible alias (middleware renamed from optional → required-by-default).
OptionalApiKeyMiddleware = ApiKeyMiddleware
7 changes: 6 additions & 1 deletion app/continuity.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,12 @@ def detect_conflicts(
*,
subject: str | None = None,
) -> list[ConflictSet]:
"""Surface disagreeing active memories sharing a subject. Never merges or picks truth."""
"""Surface disagreeing active memories sharing a subject. Never merges or picks truth.

Grouping key is ``subject`` only — ``source_agent`` / ``session_id`` are NOT
partition keys. Cross-agent contradictory claims on the same subject are
compared (Codex vs Devin, etc.). There are no per-agent write slots.
"""
groups: dict[str, list[MemoryRecord]] = {}
for rec in records:
if not rec.subject:
Expand Down
4 changes: 2 additions & 2 deletions app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from fastapi import FastAPI, HTTPException, Query
from fastapi.middleware.cors import CORSMiddleware

from app.auth import OptionalApiKeyMiddleware
from app.auth import ApiKeyMiddleware
from app.continuity import to_selection
from app.models import (
BoardUpdate,
Expand Down Expand Up @@ -36,7 +36,7 @@
allow_methods=["*"],
allow_headers=["*"],
)
app.add_middleware(OptionalApiKeyMiddleware)
app.add_middleware(ApiKeyMiddleware)


@app.get("/")
Expand Down
3 changes: 2 additions & 1 deletion app/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
]
MemoryStatus = Literal["draft", "verified", "archived"]

# Board UI slots (unchanged; board is workspace context, not the ledger record)
# Board UI slots — workspace context / UX only. NOT ledger write partitions.
# Ledger rows live in one shared store; conflicts group by subject across agents.
SlotClass = Literal["foundation", "identity", "preference", "operational"]


Expand Down
7 changes: 6 additions & 1 deletion app/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,12 @@ def _load(self):
self._dirty_migration = False

def _save(self):
"""Atomic replace: write temp sibling then os.replace."""
"""Atomic replace: write temp sibling then os.replace.

Atomicity prevents torn files; it does **not** serialize multi-writer
contention. Concurrent processes can still lose updates (last writer
wins). Single shared ledger — no per-agent write slots.
"""
self._path.parent.mkdir(parents=True, exist_ok=True)
data = {
"board": self._board.model_dump(),
Expand Down
3 changes: 3 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ services:
JARVIS_PORT: "8001"
JARVIS_STORE_PATH: /data/jarvis-store.json
JARVIS_CORS_ORIGINS: ${JARVIS_CORS_ORIGINS:-http://localhost:3000}
# Required by default. For local compose without a key, set
# JARVIS_ALLOW_UNAUTHENTICATED=1 (not recommended when publishing ports).
JARVIS_API_KEY: ${JARVIS_API_KEY:-}
JARVIS_ALLOW_UNAUTHENTICATED: ${JARVIS_ALLOW_UNAUTHENTICATED:-}
volumes:
- ledger-data:/data
restart: unless-stopped
Expand Down
15 changes: 12 additions & 3 deletions docs/CONTINUITY_LEDGER_SOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

## This package owns

- Persist board context + ledger records to JSON store
- Retrieve with selection provenance (why / where / when / session)
- Surface conflicts without silent merge
- Persist board context + ledger records to **one shared** JSON store
- Retrieve with selection provenance (why / where / when / session) via **explicit filters** (id, type, status, session, subject, text query)
- Surface conflicts without silent merge — grouped by **`subject` across all `source_agent`s**
- Content hash (`content_sha256`) for drift checks (**partial** vs multi-day protocol)

## This package does not own
Expand All @@ -15,9 +15,18 @@
- Knowledge / Understanding engines
- Constitutional Continuity Service (CCS) as multi-product authority
- Chat transcript archival (out of policy — store decisions/evidence)
- Embedding / vector / similarity retrieval (deliberate non-goal — this is a ledger)

## Shared continuity vs slots

- **Built:** single shared ledger (pattern **C**). Board `slots` are UI context only.
- **Conflict guarantee:** cross-agent by `subject` (not within-agent-only).
- **Not built:** per-agent write partitions. If added later, must stay pattern **A** (shared conflict read) — never isolated per-agent logs (**B**). See `docs/PLATFORM_LIMITS.md`.

## Non-goals

- Ranking confidence as truth
- Auto-resolving `supersedes` into deletion
- Multi-tenant SaaS isolation
- Recall-by-similarity / associative “memory”
- Claiming multi-writer safety from atomic file replace
17 changes: 15 additions & 2 deletions docs/DRIFT_PROTOCOL.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
# Drift protocol (operator)

**Status:** **partial** — hash fidelity is enforced in tests; multi-day monitoring is operator-owned.
**Status:** **partial** — hash fidelity is enforced in tests; multi-day semantic consistency is operator-owned.

## What the system verifies

1. Capture a baseline row (or use `tests/fixtures/drift_baseline.json` pattern).
2. On later retrieve, compare `content_sha256` to the baseline hash of normalized content.
3. On mismatch: treat as continuity incident — do not silently rewrite; append a new record with `supersedes` if replacing intentionally.
4. Automated multi-day schedulers are **not** shipped in this package.

This catches **tampering / corruption / accidental rewrite** of stored bytes. It does **not** prove that day-30 decisions still agree with day-1 intent.

## What operators must own

- Multi-day / multi-week semantic agreement across agents and sessions
- Scheduled retrieve + human (or consumer) review of open conflicts on critical subjects
- Incident response when hash matches but meaning has drifted (new contradictory posts on same `subject`)

Automated multi-day schedulers are **not** shipped in this package.

**Does this gap matter for unattended long-horizon agents?** Yes — if an agent fleet must stay consistent without an operator, hash checks alone are insufficient. Plan an external protocol or accept that continuity is “byte fidelity + conflict surfacing,” not “semantic lock.”
11 changes: 7 additions & 4 deletions docs/OPERATOR_DEPLOY_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> **Status:** complete — Implementor-filled 2026-07-30 (crew2 trail)
> Evidence required per Drive-G-1 before marking any item "complete".
> Evidence basis: `app/` source, `tests/` suite (51 passed), `Dockerfile`, `docker-compose.yml`, `scripts/smoke-test.ps1`, `SECURITY.md`, `.env.example`.
> Evidence basis: `app/` source, `tests/` suite, `Dockerfile`, `docker-compose.yml`, `scripts/smoke-test.ps1`, `SECURITY.md`, `.env.example`, `docs/PLATFORM_LIMITS.md`.

---

Expand Down Expand Up @@ -35,7 +35,8 @@
| `JARVIS_STORE_PATH` | Recommended | E.g. `data/jarvis-store.json` or absolute path on durable volume |
| `JARVIS_ENV` | **Yes** | Set to `production` — disables uvicorn `--reload` |
| `JARVIS_CORS_ORIGINS` | **Yes** | Set to actual origin(s), comma-separated; **not `*`** in production |
| `JARVIS_API_KEY` | Recommended | Generate with `python -c "import secrets; print(secrets.token_hex(32))"` — store out-of-band, never commit |
| `JARVIS_API_KEY` | **Yes** (default) | Generate with `python -c "import secrets; print(secrets.token_hex(32))"` — store out-of-band, never commit |
| `JARVIS_ALLOW_UNAUTHENTICATED` | Local only | Set `1` only for trusted loopback when no key; never with published ports |

### 1c. Store path and permissions

Expand All @@ -52,7 +53,7 @@

- [ ] Run tests once from the deploy directory to confirm environment:
```powershell
python -m pytest -q # must return 51 passed
python -m pytest -q # all tests must pass
```

---
Expand Down Expand Up @@ -232,10 +233,12 @@ Invoke-RestMethod http://127.0.0.1:8001/api/jarvis/memory/board
| Non-claim | Detail | Where documented |
|-----------|--------|-----------------|
| **TLS** | Service binds HTTP only; TLS is operator-managed via reverse proxy | `SECURITY.md §5`; §2 above |
| **HA / replication** | Single atomic-write JSON file (`os.replace`); not replicated, not HA | Scorecard "Platform engineering" |
| **HA / replication** | Single atomic-write JSON file (`os.replace`); not replicated, not HA; **not multi-writer safe** | `docs/PLATFORM_LIMITS.md` |
| **Per-agent write slots** | Not built; board UI `slots` ≠ partitions. Conflicts are **cross-agent by subject** on one store | `docs/PLATFORM_LIMITS.md` §0 |
| **CCS root authority** | Clause V / Constitutional Continuity Service declared only; not enforced by this service | `docs/CONTINUITY_LEDGER_SOC.md` |
| **Mandala constitutional runtime** | No Cursor hook infra, no CCS engine; see `docs/RELATIONSHIP_TO_MANDALA.md` | `docs/RELATIONSHIP_TO_MANDALA.md` |
| **Multi-tenant / commercial** | Not started; no tenant isolation, billing, or signup flow | Scorecard "Commercial operations" |
| **Vector / similarity memory** | Deliberate non-goal — ledger filters only | `docs/CONTINUITY_LEDGER_SOC.md` |
| **JARVIS_ENV=production** | Disables uvicorn `--reload` only; does not enable HA, TLS, or Postgres | `app/__main__.py`, `Dockerfile` |

**Operator checklist before exposing to a network:**
Expand Down
Loading
Loading