Skip to content

feat(stats): golden-record facts for the Query Results breakdown (slice 67E, backend) - #270

Merged
neomatrix369 merged 6 commits into
mainfrom
slice/67e-results-breakdown-backend
Nov 5, 2026
Merged

neomatrix369 merged 6 commits into
mainfrom
slice/67e-results-breakdown-backend

Conversation

@neomatrix369

@neomatrix369 neomatrix369 commented Nov 5, 2026 •

Copy link
Copy Markdown
Owner

What and why

Slice 67E, backend half. The stats tile for "Query Results" shows a 6-7 digit number with no way to see what it is made of. This adds the backend facts needed to break it down by golden-dataset record; the tile that displays it is the frontend half (separate branch, after this).

  • Each stored result row is one query answered in one run, so total_results is the sum over runs of queries answered (runs × golden queries when every run finished). top_k only changes how many chunks each row holds. unique_queries is not the golden count: it equals total_results because a fresh id is generated for every query in every run.
  • New per-experiment stats: result_runs, golden_queries, golden_queries_source (config or max_per_run), partial_runs. New group totals: total_result_runs, golden_queries_min, golden_queries_max. All are assembled once in server/db/ports/stats_common.py (golden_query_facts) for Mongo, Postgres and SQLite; adapters only supply per-run row counts they already build, so the single-experiment paths issue no extra query.
  • The golden count is the number of records in the golden dataset, counted with the pipeline's own load_queries and saved as sweep_summary.golden_queries the first time a run loads the queries (once per process, best effort, idempotent). Older experiments are inferred from stored results and tagged max_per_run.
  • Redis and Elasticsearch report nothing new in the grouped view (no overlay, as decided). The detail view overlays the run-state facts for every backend.
  • Failure isolation: an optional per-run query that fails logs one warning and reports unknown (None), never zero, and never fails the endpoint.

Verification

  • ./scripts/ci/quality-gates.sh: all gates passed (1015 backend tests, coverage 79.31 / 66.48 / 79.31 against the 72/59/72 floors, ruff, mypy, xenon, bandit, audits).
  • Seam test drives the real _run_single against a real SQLite store with a 3-record, two-persona, duplicate-text queries file and reads sweep_summary.golden_queries == 3 and exactly 3 result rows back; deleting the persist call or passing a wrong length makes it fail.
  • Live runs against throwaway containers (the live suites are skipped in PR CI): Postgres integration tests 20 passed; new live Mongo test 2 passed (two-stage grouped pipeline, single and grouped paths agree).
  • Three independent reviews (code quality, system design, acceptance coverage) approved after one round of changes: failure isolation and duplicate aggregates, an unwitnessed orchestrator seam, a Postgres test that had never executed, and a missing import in an uncommitted edit.

Notes for the reviewer

  • Known limits: under the default split-store setup the grouped view has no run-state overlay, so the group golden facts stay empty (the same pre-existing gap total_results has); partial_runs also counts runs still in flight; the stored count is write-once and goes stale if queries_file changes between pause and resume; the Mongo and Postgres live tests do not run in PR CI.
  • Follow-ups (listed in the spec): an atomic set-field operation for sweep_summary (the SQLite read-modify-write can lose a concurrent update; a lost count self-heals through the fallback); merge the two Postgres grouped scans; the Mongo per-experiment query_ids document-size ceiling; a Mongo index on results(experiment_id, run_id).
  • Docs: spec, PROGRESS, CHANGELOG, GROWTH, DECISIONS #313. This PR and fix(dashboard): numbers that add up: one Bayesian denominator, pure label helpers (slice 67D, first bundle) #269 (67D) both add lines at the same spots in those files; whichever merges second needs a trivial rebase.
  • No attribution trailers.

Complexity

Automatically refreshed by CI for product-code changes.

Scope Tool Result
Python Radon highest CC 40 (rank E), 900 blocks
JS/TS ESLint 18 function(s) above CC 10
JS/TS ESLint sonarjs 7 function(s) above cognitive 15
Functions above the JavaScript threshold
  • ExperimentDetailScreen.tsx:53 — CC 118
  • SearchExplorerScreen.tsx:35 — CC 38
  • ExplorePanels.tsx:415 — CC 29
  • LoadingFeedbackPanel.tsx:41 — CC 28
  • labels.ts:36 — CC 25
  • ExperimentsScreen.tsx:64 — CC 21
  • VectorDbStatsPanel.tsx:84 — CC 18
  • ExperimentControlButtons.tsx:54 — CC 14
  • loadWalkthruResults.ts:37 — CC 14
  • ExperimentsScreen.tsx:465 — CC 13
Functions above the cognitive-complexity threshold
  • ExperimentDetailScreen.tsx:53 — cognitive 60
  • LoadingFeedbackPanel.tsx:41 — cognitive 27
  • labels.ts:36 — cognitive 25
  • ExplorePanels.tsx:415 — cognitive 23
  • markdown.ts:59 — cognitive 20
  • SearchExplorerScreen.tsx:35 — cognitive 17
  • steps.ts:63 — cognitive 16

Expose result_runs, golden_queries (+source) and partial_runs so the
dashboard can explain total_results as runs x golden queries instead of
implying top_k drives the row count. The golden count is persisted lazily
by the orchestrator from the same load_queries the pipeline uses; older
experiments infer it from the largest per-run result count.
Derive per-run result counts from the run breakdown the single-experiment
paths already build (no extra results scan), fold them into the existing
Mongo grouped pipeline, and degrade the Postgres grouped per-run query to an
empty result with a logged warning so an optional breakdown cannot take the
stats endpoints down. Memoise the lazy golden-count write per process, log
persistence failures with the traceback, and document partial_runs, write-once
golden_queries and the grouped-view limitation. Add a real-run seam test and
live-Postgres assertions.
A failed optional per-run query now yields None (unknown) for result_runs,
partial_runs and the golden facts instead of a misleading 0, and group totals
skip it. Log the persistence traceback only on the first failure, apply the
same >0 run filter in the SQLite adapter, and add a live Mongo test of the
grouped results pipeline plus stricter seam and edge-case tests.
…cision 313, progress

Records the owner decisions on the golden count, the verified one-row-per-query
fact, the failure isolation and what is still open for the frontend half.
Resolves the four documentation conflicts (CHANGELOG, DECISIONS, GROWTH,
PROGRESS) by keeping both slices' entries; 67D first, then 67E.
@neomatrix369
neomatrix369 merged commit 9354d02 into main Nov 5, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant