Skip to content

MC-30: Sync the pipeline-records schema#14

Draft
touchdownllc wants to merge 4 commits into
Ed-Fi-Alliance-OSS:mainfrom
touchdownllc:MC-30-fact-run-identity-schema-sync
Draft

MC-30: Sync the pipeline-records schema#14
touchdownllc wants to merge 4 commits into
Ed-Fi-Alliance-OSS:mainfrom
touchdownllc:MC-30-fact-run-identity-schema-sync

Conversation

@touchdownllc

@touchdownllc touchdownllc commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

In plain English

This PR carries two byte-identical schema syncs to docs/design/schemas/pipeline-records.schema.json, both documented in Database-entity-details.md:

  1. Fact-run identityfact_runs gains two digest columns so the landing side can tell a corrected fact run from one it already landed. Tracked as MC-30; producer change nachos-ai-poc-3#346 (issue #345).
  2. base_nachos_scorescore_records gains the base NACHOS methodology tier as a first-class landed field (it was reachable only inside the open dimensions blob). No producer emit change; the matching pipeline-side contract update is filed as nachos-ai-poc-3#349.

1. Fact-run identity

Why

A fact-run header carries mode, model, prompt_version, status, cost_cap_hit and the run counts. Nothing the landing side dedups on covered any of it: ingestion_runs.run_id is derived purely from the ingestion inputs, and the per-(state, lens) release_id covers scored records.

That gap was observed, not theorised. Correcting fact_run.mode from "api" to "llm" (MC-24 §8.8) changed 135 fact-run headers while every identity the loader compares stayed byte-identical. The load concluded the run was already landed and skipped the correction.

What changes

Two digests, added to $defs.fact_artifact_header and $defs.fact_run (both properties and required), with three new supporting $defs and a top-level x-fact-identity block:

  • observations_digest — sha256 over the observation rows. What was extracted.
  • fact_run_digest — sha256 over the identity-recipe version, every non-volatile header field, and observations_digest. The run, and the dedup key.

Dedup fact runs on fact_run_digest. Equal means the held row is current, skip. Different means re-land, including when ingestion_run_id and the snapshot digests are unchanged — the case that broke. Comparing the pair separates a header-only correction (observations equal, run digest moved) from changed data (both moved).

The digest does not cover scored_at or the token/cost/cache counters. Those describe the occurrence, and a cache-warm replay of identical facts reports $0 and different token counts without being new data. The carve-out, the recipe, and the ordering tiebreak are published in the x-fact-identity block so both sides agree on exactly what the digest covers.


2. base_nachos_score — the base half of the NACHOS score

adjusted_nachos_score is already first-class in score_record_fields, but the base NACHOS score — the raw pre-adjustment methodology tier (0–3) — sat only inside the open dimensions block (dimensions.nachos_score.value). So the base half of the base/adjusted headline required a JSON dig while the adjusted half was a plain column. That's a contract miss for a core scored value.

Added to $defs.score_record_fields.properties as ["integer","null"], 0–3, mirroring complexity_score. The landing transform derives it from dimensions.nachos_score.value the same way documentation_gap is derived — no producer emit change, since the value is already emitted inside dimensions.

Distinct from complexity_score. nachos_score (pipeline src/poc3/score/rules.py, Phase F) is the NACHOS methodology tier that feeds adjusted_nachos_score; complexity_score is the business_logic_complexity comprehension-cost signal surfaced for the analyst's Complex Business Logic column. Same input facts, different rubric — they coincide on ~99% of rows but diverge where the methodology rules differ (e.g. tier_0_natural_key_format scores a state key-format to tier 0; tier_3_concatenation scores concatenation tier 3): 238 / 24,990 rows in release r-37334d4a830f060d.


Files

  • docs/design/schemas/pipeline-records.schema.json — byte-identical to the pipeline's docs/contracts/pipeline-records.schema.json at nachos-ai-poc-3#346, plus base_nachos_score in score_record_fields.
  • docs/design/Database-entity-details.md — the two fact_runs digest columns (+ a Fact-run identity section), and the new base_nachos_score row in the score_records field table.

Note on enforcement

The two copies of this schema aren't gated by CI on either side yet — MC-18 (the contract test) is still open, and the pipeline repo has no equivalent of its transport contract-drift job. So this sync is currently a manual discipline. A drift job on the pipeline side is filed as nachos-ai-poc-3#347; the base_nachos_score producer-copy update is nachos-ai-poc-3#349; MC-18 remains the landed-row half.

touchdownllc and others added 2 commits July 23, 2026 21:46
A fact-run header carries mode, model, prompt_version, status, cost_cap_hit
and the run counts. Nothing the landing side dedups on covered any of it:
ingestion_runs.run_id is derived purely from ingestion inputs, and the
per-(state, lens) release_id covers scored records.

The consequence was observed, not theorised. Correcting fact_run.mode from
"api" to "llm" (MC-24 §8.8) changed 135 fact-run headers while every identity
the loader compares stayed byte-identical, so the load concluded the run was
already landed and skipped the correction.

The pipeline now stamps two digests into every phase-A artifact header:
observations_digest over the observation rows, and fact_run_digest over the
identity-recipe version, every non-volatile header field, and the observations
digest. fact_run_digest is the dedup key; comparing the pair separates a
header-only correction from changed data.

Syncs docs/design/schemas/pipeline-records.schema.json byte-identically with
the generating repo, and documents the two columns plus the dedup rule in
Database-entity-details so the schema and the table doc do not disagree.

Producer: nachos-ai-poc-3#346 (issue #345, ADR 0023).
Rides alongside the fact-run identity sync in this PR. adjusted_nachos_score is
first-class in score_record_fields, but the base NACHOS score — the raw
pre-adjustment methodology tier (0-3) — sat only inside the open `dimensions`
block (dimensions.nachos_score.value), so the base half of the base/adjusted
headline required a JSON dig. Surface it as a landed field (integer, 0-3,
mirroring complexity_score); the landing transform derives it from
dimensions.nachos_score.value the same way documentation_gap is derived. No
pipeline emit change — the value is already emitted inside dimensions.

Distinct from complexity_score: nachos_score (rules.py Phase F) is the NACHOS
methodology tier that feeds adjusted_nachos_score, while complexity_score is the
business_logic_complexity comprehension-cost signal. Same input facts, different
rubric — they coincide on ~99% of rows but diverge where the methodology rules
differ (e.g. tier_0_natural_key_format, tier_3_concatenation): 238 / 24,990 in
release r-37334d4a830f060d.

Database-entity-details.md documents the new score_records column.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@touchdownllc
touchdownllc marked this pull request as draft July 24, 2026 17:45
The pipeline gained two operator-supplied element-record fields (registry
data/source_documents.yaml): source_document_description, a REQUIRED free-text
framing of the state documentation package this artifact was built from (e.g.
"TEA publication Jan 2026"), and school_year, an OPTIONAL year the publication
targets. Both are uniform across every record of one artifact — display-only
provenance, never scored, structurally kept out of prompt text and scoring
rules (no SCORING_PLAN_VERSION bump on the producer side).

Syncs docs/design/schemas/pipeline-records.schema.json byte-identically with
the generating repo: both properties added to the element_record def (emit
core and the landed variant), each declared last and appended to required so
serialization order is stable; the def description tracks 30 -> 32 fields.
Documents the two source_elements columns in Database-entity-details so the
schema and the table doc do not disagree.

Producer: nachos-ai-poc-3#351 (ADR 0016).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@touchdownllc touchdownllc changed the title MC-30: fact-run content identity — fact_runs gains the dedup key MC-30: sync pipeline-records schema — fact-run identity, base_nachos_score, source-document metadata Jul 24, 2026
@touchdownllc touchdownllc changed the title MC-30: sync pipeline-records schema — fact-run identity, base_nachos_score, source-document metadata MC-30: Sync the pipeline-records schema Jul 24, 2026
The pipeline puts `documented` — THE headline predicate — on the score
envelope. Aggregate computes every reported mean, dimension distribution, and
review count over the rows where documented is true (the header's
in_scope_count is their count), so no per-record field previously let a
consumer reproduce that population from a sidecar alone. documentation_source
looks like it should serve but does not: on the API-model lens every row is
source_doc, so a consumer using it computes the headline population correctly
on the source lens and silently wrongly on the other — the defect class that
would land at the MC contract boundary.

Syncs docs/design/schemas/pipeline-records.schema.json byte-identically with
the generating repo: `documented` (boolean) added to the score sidecar
envelope def (properties and required) and the landed score variant, and the
sidecar contract knob bumped 3 -> 4 (x-sidecar-contract-version and the
envelope contract_version const). Documents the new score_records column and
refreshes the two "currently 3" sidecar-version references in
Database-entity-details so the schema and the table doc do not disagree. Per
ADR 0017 the contract-version bump does not stale adjudications; no
SCORING_PLAN_VERSION bump, no re-extraction.

Producer: nachos-ai-poc-3#366 (issue #364, ADR 0025).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant