diff --git a/docs/MCP-ARCHITECTURE.md b/docs/MCP-ARCHITECTURE.md new file mode 100644 index 0000000..5d3c826 --- /dev/null +++ b/docs/MCP-ARCHITECTURE.md @@ -0,0 +1,126 @@ +# LambdaForge — Agent-Native Trading Lab + +> Your $0.15/month serverless paper-trading bot, now scriptable by any AI agent. +> One local command. MCP for Claude, REST for everything else. Bring your own AWS, bring your own Alpaca. + +## The pitch (for the launch tweet) + +I open-sourced my serverless paper-trading lab last month. It runs 7 strategies on AWS Lambda for ~$0.15/month. + +This week it learned a new trick: **every function is now callable from Claude, Cursor, Cline, or any MCP-compatible agent — and from any REST client via OpenAPI.** + +"Hey Claude, what's my P&L today?" → answer. +"Engage the kill switch, market's tanking." → confirmation prompt → done. +"Run a backtest of mean-reversion on AAPL for the last 90 days." → chart. + +Local-first. Your AWS, your Alpaca keys, your machine. No SaaS, no hosting fees. + +## Architecture + +``` +┌─────────────────────────────────────────────────────────┐ +│ Existing Lambda functions (unchanged) │ +│ daily_scan, monitor_stops, eod_snapshot, │ +│ weekly_digest, hourly_digest, kill_switch │ +└─────────────────────────────────────────────────────────┘ + │ (shares typed core) + ▼ +┌─────────────────────────────────────────────────────────┐ +│ src/service/ — typed core (Pydantic in/out) │ +│ get_account, list_positions, set_risk_param, │ +│ run_backtest, engage_kill_switch, ... │ +└──────────────┬──────────────────┬───────────────────────┘ + │ │ + ┌──────▼──────┐ ┌──────▼──────┐ ┌─────────────┐ + │ MCP server │ │ FastAPI │ │ Static UI │ + │ (stdio) │ │ REST │ │ (Vite SPA) │ + └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ + │ │ │ + ┌──────▼──────────────────▼──────────────────▼─────┐ + │ Single process. Auth: API key + CSRF guard. │ + │ `lambdaforge serve` → opens browser tab. │ + └──────────────────────────────────────────────────┘ + │ │ │ + ▼ ▼ ▼ + Claude Desktop curl, Postman, localhost UI + Cursor, Cline, Python script, in browser + Continue, Zed any HTTP client (live tiles) +``` + +## Roadmap + +| Codename | Semver | Scope | Target | +|---|---|---|---| +| **v2** | `0.2.0` | Local-only agent layer (this initiative) | 2 weekends | +| **v3** | `0.3.0` | Self-host bundle on user's AWS | 1 weekend after v2 | +| **v4** | `0.4.0` | Native polish (Tauri) | Signal-dependent | + +Codenames track the agent-layer initiative; semver versions track the project release line (current shipped release: `0.1.0`). + +### v2 — Local-only · semver `0.2.0` · target: 2 weekends + +- Stdio MCP server exposing ~25 typed tools +- FastAPI REST with auto-generated OpenAPI spec +- Local SPA bundled in the binary, opens in browser on `lambdaforge serve` +- One auto-generated API key in `~/.config/lambdaforge/` +- SSE stream for live positions/P&L +- Async job model for backtests +- `trades.db` cached locally, refreshed from S3 every 30s via `If-Modified-Since` + +**Constraint:** all mutations to live trading are gated behind explicit confirmation. Manual order placement is paper-mode only in v2. + +**Launch hook:** "MCP-native trading lab — talk to your bot from Claude Desktop." + +### v3 — Self-host bundle · semver `0.3.0` · target: 1 weekend after v2 + +- SAM template extension: 2 new Lambdas (REST + MCP-over-HTTPS) behind API Gateway +- Cognito user pool for proper auth +- One-command `lambdaforge deploy` → user's AWS account gets a public HTTPS endpoint +- Same code, two transports + +**Launch hook:** "Now talks to ChatGPT, Gemini, Perplexity, n8n, Zapier, and anything with function calling. Still your AWS, still your data." + +### v4 — Native polish · semver `0.4.0` · signal-dependent + +- Tauri wrapper over the existing web UI (10x smaller than Electron, Rust-backed) +- AWS account setup wizard +- Alpaca key paste-and-go flow + +Only ship this if v2/v3 produce real adoption signal (≥50 ⭐/week, real issues from real users). + +## Why local-first, not SaaS + +- Hosting trades on behalf of users crosses into regulated financial territory (broker-dealer relationships, RIA registration, compliance counsel) +- Software users install and run with their own AWS + Alpaca keys = open-source software (legally clean precedent) +- The line we do not cross: hosted accounts users sign into + +## Tool surface (v2) + +**Read (~10 tools):** `get_account`, `list_positions`, `get_trade_history` (paginated), `get_daily_pnl` (chart-ready), `list_strategies`, `get_risk_params`, `get_kill_switch_state`, `list_symbols`, `get_rate_limit_hits`, `get_recent_rejections` + +**Write config (~5 tools):** `set_risk_param`, `set_monitor_interval`, `add_symbol`/`remove_symbol`, `enable_strategy`/`disable_strategy`, `set_notify_frequency` + +**Execute (~6 tools, all gated behind confirmation):** `engage_kill_switch`/`release_kill_switch`, `trigger_daily_scan`, `trigger_stop_monitor`, `switch_trading_mode`, `place_manual_order` (paper-only in v2), `cancel_order`/`cancel_all_orders` + +**Analytics (~2 tools):** `run_backtest` (async, job-based), `get_strategy_signal` + +## Security model (local-only) + +- Server binds `127.0.0.1` only — unreachable from network +- API key generated on first launch, stored at `~/.config/lambdaforge/api-key` (mode 600) +- UI sends key via `Authorization: Bearer ` header — protects against malicious websites attempting to hit localhost +- `Origin`/`Referer` check rejects non-localhost origins as belt-and-suspenders +- MCP tools marked with `destructiveHint`/`requiresConfirmation` annotations — clients like Claude Desktop prompt before destructive calls +- REST endpoints for irreversible mutations require a two-step confirmation token + +## Stack + +- Python 3.12, FastAPI, uvicorn +- Official Anthropic `mcp` SDK +- Pydantic v2 for shared schemas across both layers +- Vite + React for the UI +- Existing infra unchanged: AWS Lambda, SSM Parameter Store, S3, SAM template + +## Source-of-truth principle + +One typed service layer. Both MCP and REST are thin adapters over it. The OpenAPI schema generated from FastAPI is the contract advertised to non-MCP clients. The MCP tool list is generated from the same Pydantic models. Validation, auth, and error handling are defined once. diff --git a/docs/PLAN.md b/docs/PLAN.md new file mode 100644 index 0000000..c393070 --- /dev/null +++ b/docs/PLAN.md @@ -0,0 +1,354 @@ +# v2 Implementation Plan — MCP + REST + Local UI + +Companion to [MCP-ARCHITECTURE.md](MCP-ARCHITECTURE.md). One section per branch. Each branch ships as one atomic PR. + +> **Release strategy:** `v2` is the codename for the agent-layer initiative. All seven phases land on `main` as `[Unreleased]` changelog entries; once Phase 7 ships, the whole initiative is cut as a single semver **`0.2.0`** release. No per-phase tags, no patch versions in between — one launch moment, one announcement. + +## Conventions + +- **Branch names:** `v2/`, lowercase, dash-separated. +- **Commits:** conventional commits (`feat:`, `fix:`, `refactor:`, `test:`, `docs:`, `chore:`). +- **Tests:** colocated under `tests//test_*.py`. Every new public function has a test. +- **Type safety:** `mypy --strict` must pass on all new code under `src/api/` and `src/service/`. +- **Dependencies:** added to `requirements.txt` (runtime) or `requirements-dev.txt` (dev/test) and `pyproject.toml`. + +--- + +## Phase 1 — `v2/service-layer` + +**Goal:** extract a transport-agnostic typed core. Nothing in this phase knows what HTTP or MCP is. + +### Files + +| Path | Purpose | +|---|---| +| `src/service/__init__.py` | Re-exports the public surface | +| `src/service/account.py` | `get_account()`, `list_positions()`, `get_rate_limit_hits()` | +| `src/service/trades.py` | `get_trade_history(cursor, limit, filters)`, `get_daily_pnl(days)`, `get_recent_rejections()` | +| `src/service/strategies.py` | `list_strategies()`, `get_strategy_signal(symbol, strategy)`, `enable_strategy()`, `disable_strategy()` | +| `src/service/risk.py` | `get_risk_params()`, `set_risk_param(name, value)`, `set_monitor_interval()`, `set_notify_frequency()` | +| `src/service/symbols.py` | `list_symbols()`, `add_symbol()`, `remove_symbol()` | +| `src/service/control.py` | Kill switch get/engage/release, `trigger_daily_scan()`, `trigger_stop_monitor()`, `switch_trading_mode()`, `place_manual_order()` (paper-only), `cancel_order()`, `cancel_all_orders()` | +| `src/service/backtest.py` | `run_backtest(params, progress_cb=None)` — sync core. Async wrapper added in Phase 5. | +| `src/service/db_cache.py` | S3 `trades.db` read-through cache with `If-Modified-Since`, 30s TTL | +| `src/api/__init__.py` | Empty marker | +| `src/api/schemas.py` | Pydantic v2 models for every input + output | +| `src/api/errors.py` | `ApiError`, `ConfirmationRequired`, error envelope helpers, error code enum | + +### Tests + +- `tests/service/test_db_cache.py` — cache hits, conditional refresh, no-write semantics +- `tests/service/test_account.py` — mocked Alpaca client +- `tests/service/test_risk.py` — SSM roundtrip via `moto` +- `tests/service/test_trades.py` — pagination, filters, empty results +- `tests/service/test_control.py` — confirmation that `place_manual_order` refuses in live mode +- `tests/api/test_schemas.py` — Pydantic edge cases + +### Acceptance criteria + +- [ ] All existing tests still pass +- [ ] `from src.service import get_account` works with zero HTTP/MCP imports +- [ ] `mypy --strict src/service/ src/api/schemas.py src/api/errors.py` passes +- [ ] No `print()` calls; all logging through the existing `stock-trader` logger +- [ ] `place_manual_order` raises `ApiError(code="live_trading_disabled")` when `trading_mode == "live"` in v2 +- [ ] New runtime deps: `pydantic>=2.5` +- [ ] New dev deps: `moto[s3,ssm]`, `pytest-mock` + +### Dependencies blocked by this phase +All other phases. + +--- + +## Phase 2 — `v2/auth-and-config` + +**Goal:** API key, confirmation tokens, config paths. + +### Files + +| Path | Purpose | +|---|---| +| `src/config_paths.py` | `get_config_dir()` honouring `XDG_CONFIG_HOME`, defaults to `~/.config/lambdaforge` | +| `src/api/auth.py` | `generate_api_key()`, `load_or_create_api_key()`, `verify_api_key(header)` | +| `src/api/confirm.py` | `issue_confirmation_token(action_id)`, `verify_and_consume_token(token, action_id)`, in-memory TTL=60s, single-use | +| `src/cli/__init__.py` | CLI package marker | +| `src/cli/keys.py` | `lambdaforge regenerate-key` subcommand | + +### Tests + +- `tests/api/test_auth.py` — valid key, invalid key, missing key, wrong header name +- `tests/api/test_confirm.py` — token issued, single-use enforced, TTL expiry, wrong action_id rejected +- `tests/cli/test_keys.py` — key file mode `0o600`, regenerate invalidates old key +- `tests/test_config_paths.py` — XDG env var override, default fallback, mode enforcement + +### Acceptance criteria + +- [ ] First `load_or_create_api_key()` call creates `/api-key` with mode `0o600` +- [ ] Key never appears in logs (logger filter scrubs `Authorization` headers) +- [ ] Confirmation tokens are 32-byte URL-safe random, in-memory only (no disk persistence) +- [ ] Token bound to a specific action_id (e.g. `engage_kill_switch`) — can't reuse a token issued for one action to confirm a different one +- [ ] `lambdaforge regenerate-key` prints new key once to stdout, atomic file replace + +### Dependencies blocked by this phase +Phases 3, 4. + +--- + +## Phase 3 — `v2/mcp-stdio` + +**Goal:** MCP server over stdio that wraps every service function as a tool. + +### Files + +| Path | Purpose | +|---|---| +| `src/api/mcp.py` | Tool registrations with annotations | +| `src/cli/serve.py` | `--mcp-only` entry point (skeleton; full version in Phase 7) | +| `pyproject.toml` | Add `mcp` dep | + +### Tool inventory (must match service layer exactly) + +- **Read (no confirmation):** `get_account`, `list_positions`, `get_trade_history`, `get_daily_pnl`, `list_strategies`, `get_risk_params`, `get_kill_switch_state`, `list_symbols`, `get_rate_limit_hits`, `get_recent_rejections`, `get_strategy_signal` +- **Write config (no confirmation, reversible):** `set_risk_param`, `set_monitor_interval`, `set_notify_frequency`, `add_symbol`, `remove_symbol`, `enable_strategy`, `disable_strategy` +- **Destructive (`destructiveHint: true`, `requiresConfirmation: true`):** `engage_kill_switch`, `release_kill_switch`, `trigger_daily_scan`, `trigger_stop_monitor`, `switch_trading_mode`, `place_manual_order`, `cancel_order`, `cancel_all_orders` + +### Tests + +- `tests/api/test_mcp.py` — tool list matches expected set exactly (regression guard against accidental tool addition/removal) +- `tests/api/test_mcp_annotations.py` — every destructive tool has both annotations set +- `tests/api/test_mcp_schemas.py` — tool input schemas match `src/api/schemas.py` Pydantic models + +### Acceptance criteria + +- [ ] ~25 tools registered, count asserted in test +- [ ] All destructive tools annotated with `destructiveHint: true` and `requiresConfirmation: true` +- [ ] Tool input/output schemas sourced from `src/api/schemas.py` — no duplication +- [ ] `python -m src.cli.serve --mcp-only` launches stdio server, terminates cleanly on EOF +- [ ] Manual smoke test: Claude Desktop config with this server lists all tools, can call `get_account` end-to-end +- [ ] New runtime dep: `mcp>=1.0` + +### Dependencies blocked by this phase +Phase 7 (CLI orchestration). + +--- + +## Phase 4 — `v2/rest-api` + +**Goal:** FastAPI server with the same surface as MCP, plus SSE streams. + +### Files + +| Path | Purpose | +|---|---| +| `src/api/rest.py` | FastAPI app with all routes | +| `src/api/stream.py` | SSE endpoints: `/api/stream/positions`, `/api/stream/pnl` | +| `src/api/middleware.py` | Loopback bind enforcement, `Origin`/`Referer` allowlist (`localhost`, `127.0.0.1`) | +| `pyproject.toml` | Add `fastapi`, `uvicorn[standard]`, `sse-starlette` deps | + +### Route shape + +- `GET /api/account`, `/api/positions`, `/api/trades?cursor=&limit=&symbol=&strategy=`, `/api/pnl/daily?days=`, `/api/strategies`, `/api/risk`, `/api/kill-switch`, `/api/symbols`, `/api/rejections`, `/api/strategies/{name}/signal/{symbol}` +- `PATCH /api/risk/{param}`, `/api/monitor/interval`, `/api/notify/frequency` +- `POST /api/symbols`, `DELETE /api/symbols/{symbol}` +- `POST /api/strategies/{name}/enable`, `/disable` +- **Confirmation-gated:** `POST /api/kill-switch/engage`, `/release`, `/api/scan/trigger`, `/api/monitor/trigger`, `/api/mode/switch`, `/api/orders`, `DELETE /api/orders/{id}`, `DELETE /api/orders` +- `GET /api/stream/positions`, `/api/stream/pnl` — SSE + +### Tests + +- `tests/api/test_rest_read.py` — every GET returns 200 and a schema-valid body +- `tests/api/test_rest_mutations.py` — every confirmation-gated endpoint returns 409 → 200 flow +- `tests/api/test_rest_auth.py` — missing key → 401, wrong key → 401 +- `tests/api/test_middleware.py` — `Origin: http://evil.com` → 403, `Origin: http://localhost:5173` → 200, no `Origin` header → 403 for mutations, 200 for reads +- `tests/api/test_stream.py` — SSE emits at expected cadence, terminates on client disconnect + +### Acceptance criteria + +- [ ] OpenAPI spec at `/openapi.json` documents all routes +- [ ] Every endpoint returns the error envelope from Phase 1 on failure +- [ ] Confirmation flow: first call → `409 {"error": {"code": "confirmation_required", "token": "...", "expires_in": 60}}`; second call with `X-Confirm-Token` header → 200 +- [ ] SSE streams emit every 5s, send a `:keepalive` comment every 15s +- [ ] Server binds `127.0.0.1` only (test: `0.0.0.0` bind rejected at startup if `LAMBDAFORGE_ALLOW_REMOTE` is not set) +- [ ] CORS allowlist: `http://localhost:*`, `http://127.0.0.1:*` + +### Dependencies blocked by this phase +Phases 5, 6, 7. + +--- + +## Phase 5 — `v2/backtest-jobs` + +**Goal:** async job model. Reusable for any long-running task. + +### Files + +| Path | Purpose | +|---|---| +| `src/api/jobs.py` | `JobStore`, `JobStatus` enum, `submit_job(fn, *args)`, `get_job(id)`, `cancel_job(id)` | +| `src/service/backtest.py` | Modified to accept `progress_cb` callback | +| `src/api/rest.py` | `POST /api/backtest` → `{job_id}`, `GET /api/backtest/{job_id}` → `{status, progress, result, error}` | +| `src/api/mcp.py` | Tools: `start_backtest`, `get_backtest_status`, `cancel_backtest` | + +### Tests + +- `tests/api/test_jobs.py` — submit, poll, complete, fail, cancel, TTL eviction +- `tests/service/test_backtest_progress.py` — `progress_cb` called with values 0–100 monotonically + +### Acceptance criteria + +- [ ] Jobs run as `asyncio.Task`s in the FastAPI event loop +- [ ] Statuses: `pending`, `running`, `completed`, `failed`, `cancelled` +- [ ] Progress is 0–100 integer, monotonically non-decreasing +- [ ] Jobs auto-expire from the in-memory store 1 hour after completion +- [ ] Job persistence is **explicitly out of scope** — server restart loses job history (acceptable for v2; v3 puts this in DynamoDB) + +### Dependencies blocked by this phase +Phase 6 (UI backtest screen). + +--- + +## Phase 6 — `v2/ui` + +**Goal:** React SPA, bundled into the Python package at build time. + +### Files + +``` +ui/ +├── package.json +├── vite.config.ts +├── tsconfig.json +├── index.html +└── src/ + ├── main.tsx + ├── App.tsx + ├── lib/ + │ ├── client.ts # Typed REST client + │ ├── sse.ts # SSE hook + │ ├── auth.ts # API key load from ?key= or localStorage + │ └── confirm.ts # Confirmation modal flow + ├── hooks/ + │ ├── useAccount.ts + │ ├── usePositions.ts # SSE-backed + │ ├── useTrades.ts # Paginated + │ └── useBacktest.ts # Job polling + ├── components/ + │ ├── PnLTile.tsx + │ ├── PositionsTable.tsx + │ ├── TradesTable.tsx + │ ├── ConfirmModal.tsx + │ └── KillSwitchPanel.tsx + └── pages/ + ├── Dashboard.tsx + ├── Positions.tsx + ├── Trades.tsx + ├── Settings.tsx + └── Backtest.tsx +``` + +Build output: `ui/dist/` → copied to `src/static/dist/` during packaging. + +### Stack + +- Vite + React 18 + TypeScript strict +- TanStack Query for data fetching/caching +- Recharts for P&L curves +- Tailwind for styling (lightweight, no design system needed for v2) +- Vitest for tests + +### Tests + +- `ui/src/__tests__/` — hook tests (Vitest + jsdom), component snapshot tests for key UI +- `tests/ui/test_static_serving.py` — Python side: FastAPI serves built assets correctly + +### Acceptance criteria + +- [ ] 5 pages render with mocked API +- [ ] Dashboard updates live via SSE +- [ ] Trades page paginates without flicker +- [ ] Confirmation modal blocks destructive actions until user confirms +- [ ] First-load flow: reads `?key=` from URL → stores in `localStorage` under `lambdaforge.apiKey` → calls `history.replaceState` to scrub URL +- [ ] All API calls include `Authorization: Bearer ` header +- [ ] `pnpm build` produces `ui/dist/` with bundled, minified assets +- [ ] Lighthouse: Performance ≥90, Accessibility ≥95, Best Practices ≥95 + +### Dependencies blocked by this phase +Phase 7. + +--- + +## Phase 7 — `v2/cli-and-packaging` + +**Goal:** one command, one process, browser opens. + +### Files + +| Path | Purpose | +|---|---| +| `src/cli/__main__.py` | `python -m lambdaforge` entry | +| `src/cli/serve.py` | Full `serve` command — orchestrates MCP stdio + FastAPI + UI static | +| `src/cli/browser.py` | `open_browser(url, key)` — uses `webbrowser` stdlib | +| `pyproject.toml` | `[project.scripts] lambdaforge = "src.cli:main"`, package_data for `src/static/dist/**` | +| `MANIFEST.in` | Include UI bundle | +| `README.md` | Quickstart updates (Install → Run → Open browser) | + +### CLI surface + +``` +lambdaforge serve # everything: MCP stdio + REST + UI, opens browser +lambdaforge serve --headless # no browser auto-open +lambdaforge serve --mcp-only # stdio MCP only (for Claude Desktop embedding) +lambdaforge serve --no-mcp # REST + UI only +lambdaforge serve --port 8787 # override default port +lambdaforge regenerate-key # rotate API key +lambdaforge --version +``` + +### Tests + +- `tests/cli/test_serve.py` — flag combinations select correct subset of services +- `tests/cli/test_browser_open.py` — URL constructed correctly with key, `--headless` skips +- `tests/cli/test_packaging.py` — `importlib.resources` can find bundled UI assets + +### Acceptance criteria + +- [ ] `pip install -e .` installs the `lambdaforge` script onto PATH +- [ ] `lambdaforge serve` starts, prints API key location, opens browser to `http://127.0.0.1:8787/?key=` +- [ ] Browser opens with key in URL once; UI scrubs URL after reading +- [ ] SIGTERM/SIGINT shuts down cleanly (no orphaned uvicorn workers) +- [ ] UI assets are inside the wheel — verifiable with `unzip -l dist/lambdaforge-*.whl | grep static` +- [ ] README quickstart works end-to-end on a fresh machine + +### Dependencies blocked by this phase +None. v2 is shippable. + +--- + +## Out of scope for v2 (deferred) + +- Live trading from `place_manual_order` (paper only in v2; lifted in v3 with stricter gating) +- Self-hosting on user's AWS — entire scope of v3 +- Persistent job store — v3 (jobs use DynamoDB once that exists) +- Multi-user RBAC — v3 +- Tauri native wrapper — v4 +- WebSocket transport (SSE covers v2 needs) +- Mobile UI + +## Critical risks + +1. **MCP SDK API stability** — `mcp` Python SDK is pre-1.0. Pin to a specific version in `pyproject.toml` and revisit at each phase boundary. +2. **SSE behind corporate proxies** — some users may have proxies that buffer SSE. Document polling fallback in README; don't engineer for it in v2. +3. **Confirmation token UX in CLI/Postman** — power users hitting the API directly will hit the 409→token flow. Document clearly in `/openapi.json` description. +4. **trades.db cache staleness** — 30s TTL means UI can show data up to 30s old. Acceptable for trade history (historical); positions/P&L use Alpaca live data, not SQLite, so no staleness there. + +## Order of execution + +``` +Phase 1 ─┬─→ Phase 2 ─┬─→ Phase 3 ──┐ + │ └─→ Phase 4 ──┼─→ Phase 5 ──→ Phase 6 ──→ Phase 7 + │ │ + └──────────────────────────┘ +``` + +Phases 3 and 4 can be developed in parallel once 1 and 2 land. Everything else is sequential. + +**Estimated effort:** 1.5–2 weekends of focused work for a single developer who knows the existing codebase. Phase 6 (UI) is the longest single phase; Phases 1, 4, 5 are medium; Phases 2, 3, 7 are short.