Performance harden call drilldown diagnostics

.github/workflows/ci.yml

@@ -43,6 +43,7 @@ jobs:
         run: |
           node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_format.js
           node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_data.js
+          node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_call_investigator.js
           node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard.js
           node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_state.js
       - name: Release readiness

AGENTS.md

@@ -144,6 +144,7 @@ python -m pytest --cov=codex_usage_tracker --cov-report=term-missing
 python -m compileall src
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_format.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_data.js
+node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_call_investigator.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_state.js
 python scripts/check_release.py
@@ -218,6 +219,7 @@ python -m pytest --cov=codex_usage_tracker --cov-report=term-missing
 python -m compileall src
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_format.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_data.js
+node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_call_investigator.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_state.js
 python scripts/check_release.py

CHANGELOG.md

@@ -2,9 +2,21 @@
 
 ## Unreleased
 
+- Remove low-value call/thread anchor diagnostics from the experimental call investigator to avoid an extra source-log scan per context load.
+- Persist call-origin metadata as categorical aggregate fields during indexing so normal dashboard payloads do not reopen source JSONL logs to infer user-vs-Codex initiation.
+- Persist archived-session scope, conservative thread keys, and per-thread previous/next call links as aggregate helper fields for faster dashboard filtering and investigator navigation.
+- Add opt-in localhost API timing diagnostics for `/api/usage` and `/api/context` without exposing raw transcript content.
+- Reduce explicit context loading to a quick default mode that omits tool output and serialized buckets, with full serialized JSONL bucket analysis still available on demand.
+- Add source-log-aware synthetic benchmark coverage that verifies normal dashboard payload assembly does not open generated source JSONL files.
+- Add SQL-backed live dashboard API slices for status, calls, one call, threads, thread calls, summary, and recommendations while preserving the compatibility `/api/usage` endpoint.
+- Materialize active and all-history thread summaries in SQLite so live thread APIs can read pre-aggregated totals.
+- Add source-file refresh cursors so live refresh skips unchanged logs, seeks to appended JSONL bytes when safe, and safely replaces aggregate rows for changed or truncated source logs.
+- Hydrate direct call-investigator links from the aggregate `/api/call` endpoint when the selected record is outside the currently loaded table slice or filter state.
+- Replace placeholder non-English dashboard locale catalogs with translated UI catalogs and add regression coverage for core visible labels.
+
 ## 0.5.0 - 2026-06-10
 
-- Add the dashboard localization foundation, including starter locale catalogs, language metadata, local browser language selection, `--lang`, and `CODEX_USAGE_TRACKER_LANG`.
+- Add the dashboard localization foundation, including initial locale catalogs, language metadata, local browser language selection, `--lang`, and `CODEX_USAGE_TRACKER_LANG`.
 - Add Vietnamese dashboard localization and focused validation coverage for translated dashboard labels.
 - Keep the README landing page focused on dashboard screenshots and companion usage workflows before detailed localization guidance.
 - Stabilize the CI synthetic benchmark smoke so coverage instrumentation does not create false release failures.

README.md

@@ -155,7 +155,7 @@ The tracker cannot read your logged-in ChatGPT plan or live remaining usage auto
 
 ## Dashboard Language
 
-The dashboard supports localized UI text. English is the canonical catalog, and the project includes starter locale catalogs for common dashboard languages.
+The dashboard supports localized UI text. English is the canonical catalog, and the project includes translated locale catalogs for common dashboard languages.
 
 Set the initial dashboard language with `--lang`:
 
@@ -171,7 +171,7 @@ CODEX_USAGE_TRACKER_LANG=vi codex-usage-tracker serve-dashboard --open
 
 The dashboard also includes a language selector. Browser selections are stored locally and can override the generated default for that browser.
 
-Supported starter locales include English, Vietnamese, Spanish, French, German, Portuguese, Japanese, Simplified Chinese, Korean, Russian, Italian, and Arabic. This localizes dashboard UI text, not the full CLI output or data exports.
+Supported dashboard locales include English, Vietnamese, Spanish, French, German, Portuguese, Japanese, Simplified Chinese, Korean, Russian, Italian, and Arabic. This localizes dashboard UI text, not raw Codex log content, thread names, project names, paths, full CLI output, or data exports.
 
 ### Adding A Dashboard Language
 

docs/architecture.md

@@ -4,31 +4,32 @@ Codex Usage Tracker is a local sidecar app. It reads aggregate token counters fr
 
 ## Boundaries
 
-- `parser.py` converts local JSONL events into aggregate `UsageEvent` records. It must not persist prompts, assistant text, tool output, or transcript snippets.
+- `parser.py` converts local JSONL events into aggregate `UsageEvent` records. It also attaches metadata-only call-origin categories, archived-session flags, and conservative thread keys. It must not persist prompts, assistant text, tool output, or transcript snippets.
+- `call_origin.py` owns the pure call-origin classifier and migrated-row fallback. It must not open source JSONL files; source-log reads belong in parser refresh or explicit context loading only.
 - `schema.py` owns persisted `usage_events` columns. Add columns there before changing SQLite migrations or export behavior.
-- `store.py` owns SQLite setup, refresh, rebuild, and query access. Keep filesystem scanning, database writes, SQL prefilters, counts, limits, and offsets here.
+- `store.py` owns SQLite setup, refresh, rebuild, query access, persisted per-thread previous/next call links, materialized thread summaries, source-file refresh cursors, and SQL-backed live dashboard API slices. Keep filesystem scanning, database writes, SQL prefilters, counts, limits, offsets, and incremental refresh decisions here.
 - `reports.py` is the application-service layer for summaries, expensive-call reports, recommendations, pricing coverage, and filtered query payloads. CLI and MCP should call this layer instead of duplicating report assembly.
-- `api_payloads.py` owns stable JSON payload helpers shared by CLI and MCP. `json_contracts.py` owns the lightweight contract checks for schema-versioned CLI/MCP payloads. Add payload builders and contract entries together when both surfaces need the same shape.
+- `api_payloads.py` owns stable JSON payload helpers shared by CLI and MCP. `json_contracts.py` owns the lightweight contract checks for schema-versioned CLI/MCP payloads and localhost live API payloads. Add payload builders and contract entries together when surfaces need the same shape.
 - `costing.py`, `pricing_config.py`, `pricing_openai.py`, `pricing_estimates.py`, and `allowance.py` own cost, credit, rate-card, and allowance annotation. Keep estimate confidence and source metadata attached to rows.
 - `projects.py`, `threads.py`, and `recommendations.py` annotate aggregate rows with project identity, thread relationships, and actionable signals. Project privacy redaction also belongs in `projects.py` so CLI, MCP, dashboard, CSV, and support-bundle surfaces share the same behavior.
-- `dashboard.py` builds aggregate-only dashboard payloads and writes HTML/assets. `server.py` adds localhost refresh and explicit lazy context loading.
+- `dashboard.py` builds aggregate-only static dashboard payloads and writes HTML/assets. `server.py` adds localhost refresh, the compatibility `/api/usage` endpoint, SQL-backed live API slices, and explicit lazy context loading.
 - `plugin_data/dashboard/dashboard_format.js` owns dashboard formatting primitives. `dashboard_data.js` owns row payload and thread relationship helpers. `dashboard_state.js` owns URL, CSV, and download state utilities. `dashboard.js` owns DOM rendering, event handling, API refresh, and detail-panel behavior.
-- `context.py` is the only normal path that reads raw log context, and it does so only for one selected record on demand with redaction and size limits.
+- `context.py` is the only normal path that reads raw log context, and it does so only for one selected record on demand with redaction and size limits. Its default quick mode omits tool output and serialized groups; full serialized JSONL group analysis is explicit.
 - `plugin_installer.py`, `.mcp.json`, `skills/`, and `scripts/check_release.py` own install and packaging behavior.
-- `scripts/benchmark_synthetic_history.py` owns generated large-history query timing and threshold enforcement for 10k, 100k, and 500k aggregate-row fixtures. It must stay synthetic-only and must not read real Codex logs.
+- `scripts/benchmark_synthetic_history.py` owns generated large-history query timing and threshold enforcement for 10k, 100k, and 500k aggregate-row fixtures. Its optional `--with-source-logs` mode writes synthetic JSONL source logs to time explicit context loading and to guard normal dashboard payload assembly against source-log reads. It must stay synthetic-only and must not read real Codex logs.
 - `skills/codex-usage-tracker/` is the source copy for the operational Codex skill. It should stay focused on setup, dashboard, export, doctor, and direct MCP workflows.
 - `skills/codex-usage-api/` is the source copy for the conversational analyst skill. It should stay focused on aggregate-only API routing, interpretation, and limitations.
 - `src/codex_usage_tracker/plugin_data/skills/` contains the wheel-bundled copies installed by `codex-usage-tracker install-plugin`.
 
 ## Extension Rules
 
-1. Add new persisted metrics through `UsageEvent`, `schema.py`, migrations, store queries, dashboard payload tests, and CSV/export checks.
+1. Add new persisted usage-event metrics through `UsageEvent`, `schema.py`, migrations, store queries, dashboard payload tests, and CSV/export checks. Add auxiliary aggregate tables such as `thread_summaries` or `source_files` through `store.py` migrations plus focused migration/privacy tests.
 2. Add new report views through `reports.py` first, then wire CLI and MCP wrappers to that shared service.
 3. Add new machine-readable outputs through `api_payloads.py` or report payload methods with a `schema` value, a `json_contracts.py` entry, and focused tests.
 4. Add dashboard-only interactions in `plugin_data/dashboard/dashboard.js` and keep URL state in `dashboard_state.js`.
 5. Keep all examples, screenshots, mocks, and tests synthetic. Never derive fixtures from real logs.
 6. When editing skill instructions, update both the source `skills/...` file and the bundled `src/codex_usage_tracker/plugin_data/skills/...` copy. `scripts/check_release.py` verifies that installable plugin assets stay complete and synced.
-7. When adding fields derived from `cwd`, Git metadata, or source paths, decide how they behave in `normal`, `redacted`, and `strict` privacy modes before exposing them in dashboard, JSON, CSV, MCP, or support-bundle output.
+7. When adding fields derived from `cwd`, Git metadata, source paths, or log-event metadata, decide how they behave in `normal`, `redacted`, and `strict` privacy modes before exposing them in dashboard, JSON, CSV, MCP, or support-bundle output.
 
 ## Validation
 
@@ -40,6 +41,7 @@ python -m compileall src
 python -m mypy
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_format.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_data.js
+node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_call_investigator.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard.js
 node --check src/codex_usage_tracker/plugin_data/dashboard/dashboard_state.js
 python scripts/check_release.py
@@ -50,4 +52,4 @@ git diff --check
 
 Dashboard UI changes should also be opened in a browser and checked on desktop and mobile widths for overlap, stale state, and aggregate-only output.
 
-Run `python scripts/benchmark_synthetic_history.py --rows 10000 100000 --json --enforce-thresholds` after changing SQLite filters, dashboard payload loading, or indexes. Run the 500k benchmark before release work when practical.
+Run `python scripts/benchmark_synthetic_history.py --rows 10000 100000 --json --enforce-thresholds` after changing SQLite filters, dashboard payload loading, or indexes. Run `python scripts/benchmark_synthetic_history.py --rows 1000 --with-source-logs --json --enforce-thresholds` after changing explicit context loading or source-log diagnostics. Run the 500k benchmark before release work when practical.