From bb27e64c34fa3f8cae440183c4d9cd6b3cba77e8 Mon Sep 17 00:00:00 2001 From: Chris Moffatt Date: Mon, 27 Jul 2026 17:44:04 -0700 Subject: [PATCH 1/2] MC-30: sync Database-entity-details with five landed pipeline changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documents five changes that have landed on the pipeline side, so the entity and field tables describe what the pipeline now emits. Fact-run identity (nachos-ai-poc-3#345, ADR 0023). fact_runs gains observations_digest and fact_run_digest, plus a Fact-run identity section naming fact_run_digest as the dedup key. Nothing the landing side deduped on covered a fact-run header: correcting fact_run.mode from "api" to "llm" (MC-24 §8.8) changed 135 headers while every identity the loader compares stayed byte-identical, so the load decided the run was already there and skipped the correction. base_nachos_score on score_records. adjusted_nachos_score was already a column; the base methodology tier it is built from was reachable only inside the open dimensions blob, so one half of the base/adjusted pair needed a JSON dig. Landing-derived from dimensions.nachos_score.value, the same way documentation_gap is, so nothing changes on the emit side. source_document_description and school_year on source_elements (nachos-ai-poc-3#351, ADR 0016) — operator-supplied provenance for the documentation package an artifact was built from, sourced from the pipeline's committed data/source_documents.yaml registry. Display-only, never scored. documented on score_records (nachos-ai-poc-3#364, ADR 0025) — the headline predicate aggregate computes every reported mean, dimension distribution, and review count over. documentation_source cannot stand in for it: on the API- model lens every row is source_doc. Sidecar contract version 3 -> 4, which per ADR 0017 does not stale adjudications. prompt_fingerprint and answered_by_host on fact_runs (nachos-ai-poc-3#378 and #381, ADR 0027; host tags #392, refined by #394); fact identity version 1 -> 2 as an epoch marker, both fields nullable and covered by the ADR 0023 deny-list so the digest recipe is unchanged. Also corrects the fact_runs.contract_version row, which described the sidecar meaning — the exact touchdownllc/dataload_poc#27 conflation. Co-Authored-By: Claude Opus 5 (1M context) --- docs/design/Database-entity-details.md | 55 ++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 4 deletions(-) diff --git a/docs/design/Database-entity-details.md b/docs/design/Database-entity-details.md index a1c4cc0..38d76c0 100644 --- a/docs/design/Database-entity-details.md +++ b/docs/design/Database-entity-details.md @@ -62,7 +62,7 @@ shared number that bumps for everything conflates unrelated changes: | Version | Field | What bumps it | |---|---|---| -| Envelope shape | `contract_version` | A change to the emitted record structure — the envelope (this contract). Already implemented as `SIDECAR_CONTRACT_VERSION` (currently `3` — ADR 0017 split the knob; POC-3 ADR 0020 bumped it for the four #318 per-record additions); envelope evolution bumps this knob without invalidating adjudications. Transform migrations key off it. | +| Envelope shape | `contract_version` | A change to the emitted record structure — the envelope (this contract). Already implemented as `SIDECAR_CONTRACT_VERSION` (currently `4` — ADR 0017 split the knob; POC-3 ADR 0020 bumped it for the four #318 per-record additions, POC-3 ADR 0025 for the `documented` predicate); envelope evolution bumps this knob without invalidating adjudications. Transform migrations key off it. | | Scoring methodology | `scoring_plan_version` | Rules, prompts' meaning, adjustment magnitudes — methodology semantics only (ADR 0017 moved sidecar-shape changes to `contract_version`). Reporting, score labels, and curation staleness key off it. | | State source documentation | `snapshot_id` + `snapshot_digest` | Content identity of the elements artifact a run consumed. Already implemented (ADR 0018): `snapshot_id` is `{filename stem}@{digest[:16]}`; `snapshot_digest` is sha256 over a canonical form of the artifact (volatile run timestamps stripped), so same content means same digest across re-ingests and formatting differences. This is the PRD's `state_requirements_tag` — the doc uses the pipeline's names. | @@ -74,6 +74,45 @@ Every score row's run traces to all of these: which envelope shape it was emitted under, which methodology produced it, and which edition of the state's documentation it scored — provably, via the digest. +### Fact-run identity + +`release_id` and the snapshot digests cover ingestion inputs and scored +records. Neither covers a **fact-run header** — the phase-A artifact's `mode`, +`model`, `prompt_version`, `status`, `cost_cap_hit`, and counts — so a +correction that changes only header metadata leaves every identity above +byte-identical. That is not hypothetical: the `fact_run.mode` `"api"` → `"llm"` +resolution (MC-24 §8.8) changed 135 headers with no change to any run identity, +and a loader deduping on `ingestion_run_id` would silently keep the stale +header. + +Two digests on `fact_runs` close that gap (nachos-ai-poc-3#345): + +- **`observations_digest`** — sha256 over the artifact's observation rows. + *What was extracted.* +- **`fact_run_digest`** — sha256 over the identity-recipe version, every + non-volatile header field, and `observations_digest`. *The run.* **Dedup fact + runs on this.** Equal digest ⇒ the held row is current, skip. Different digest + ⇒ re-land, *even when `ingestion_run_id` and the snapshot digests are + unchanged*. Comparing the pair distinguishes a header-only correction + (observations equal, run digest moved) from changed data (both moved). + +The digest deliberately does not cover `scored_at` or the token/cost/cache +counters — those describe the *occurrence*, and a cache-warm replay of identical +facts legitimately reports `$0` and different token counts. The exact carve-out, +the recipe, and the ordering tiebreak are published in the schema's +`x-fact-identity` block (`docs/design/schemas/pipeline-records.schema.json`), so +both sides agree on precisely what the digest covers rather than inferring it. + +ADR 0027 (nachos-ai-poc-3#378/#381) adds two non-volatile header fields — +`prompt_fingerprint` (the prompt bytes that produced the observations) and +`answered_by_host` (which endpoint served the run). The deny-list covers both by +construction, so the recipe is unchanged; the identity-recipe version moves +1 → 2 purely as an epoch marker, published in `x-fact-identity.version_history`. +The consequence is one-time and expected: every `fact_run_digest` in the tree +changes at once while every `observations_digest` stays put, so the load reads it +as a single producer-side recipe epoch — re-land every fact run once — rather +than as changed data. Both fields are nullable and carry no backfill obligation. + ### Assessor identity A scoring run is one assessor's sweep over a whole snapshot. The run record @@ -278,6 +317,8 @@ state explicitly authored or backfilled from the Ed-Fi swagger. | `source_page_or_section` | Location within the source document | TEXT | | `documented` | Whether the state source explicitly documents this element | BOOLEAN | | `documentation_source` | Provenance label: `source_doc`, `swagger`, or `swagger_leaf` | TEXT | +| `source_document_description` | *(nachos-ai-poc-3#351)* Operator-supplied framing of the state documentation package this artifact was built from — e.g. "TEA publication Jan 2026" — sourced from the pipeline's committed `data/source_documents.yaml` registry. Required non-blank (schema `minLength: 1`) and uniform across every record of one artifact; display-only provenance, never scored. | TEXT | +| `school_year` | *(nachos-ai-poc-3#351)* Operator-supplied school year the documentation targets, e.g. `2025-2026`. NULL when the publication does not target a single year. Uniform across every record of one artifact, alongside `source_document_description`. | TEXT | **Does ingest erase and recreate all rows for a state?** Under retain-by-run, each run inserts a fresh snapshot keyed by `ingestion_run_id`; rows from prior @@ -419,7 +460,7 @@ progresses/completes. | `lens` | `source` or `spine` | TEXT | | `ingestion_run_id` | The ingestion snapshot this extraction ran against | BIGINT | | `snapshot_id` | Snapshot identifier of that ingestion run | TEXT | -| `contract_version` | Envelope-shape version of the emitted records (see the version numbers) | TEXT | +| `contract_version` | *(meaning clarified in the contract — touchdownllc/dataload_poc#27)* The **fact-identity recipe epoch** this run was landed under: the string form of `x-fact-identity.version`, read from the vendored contract at landing time rather than hand-typed. **Not** the sidecar envelope version that `scoring_runs.contract_version` carries — a fact run has no sidecar, and the two families version on independent lifecycles (the fact-identity recipe is ADR 0023's; the sidecar envelope is ADR 0017's). Landing-supplied by design: the phase-A header carries no `contract_version` to project. | TEXT | | `fact_name` | Fact label, e.g. `business_rules_present`, `has_conditional_logic`, or the v30 extension-necessity pair `state_requirement_basis` / `core_can_express_requirement` (which replaced the retired `extension_is_necessary`). Emitted as `fact` in the artifact header. | TEXT | | `artifact_name` | Short label for the run, e.g. `AZ_source_business_rules_present` | TEXT | | `artifact_path` | Path to the emitted artifact file (optional but useful for replay/debug) | TEXT | @@ -430,7 +471,11 @@ progresses/completes. | `cost_cap_hit` | Whether the run hit its configured cost cap | BOOLEAN | | `schema_error` | Header-level schema validation error text, when present | TEXT | | `scored_at` | Header timestamp for artifact completion | TIMESTAMPTZ | -| `source_hash` | Optional checksum for deduplicating repeated submissions of the same job | TEXT | +| `observations_digest` | sha256 over the artifact's observation rows — what was extracted. Equal digests mean the same facts | TEXT | +| `fact_run_digest` | sha256 over the identity-recipe version, every non-volatile header field, and `observations_digest`. **The dedup key for fact runs** — see *Fact-run identity* above | TEXT | +| `source_hash` | Optional checksum for deduplicating repeated submissions of the same job. Predates `fact_run_digest` and is not the dedup key; dedup on `fact_run_digest` | TEXT | +| `prompt_fingerprint` | *(fact identity v2 — nachos-ai-poc-3#378, POC-3 ADR 0027)* sha256 over every rendered prompt batch of the pair — the prompt bytes that produced the observations. Nullable: NULL/absent on artifacts extracted before the field existed and where a cache replay could not prove the answers match the current prompt (absent means "not established", never "unchanged"); omitted on deterministic-fact runs (no prompt). Non-volatile, so it rides inside `fact_run_digest`; consumers land it as provenance and never recompute it. | TEXT | +| `answered_by_host` | *(fact identity v2 — nachos-ai-poc-3#381, POC-3 ADR 0027)* Transport identity of the endpoint that served the run: `anthropic` (direct Messages API), `azure:/`, `deterministic`, `cache:` (every batch replayed and every entry names the same producing host), `cache` (every batch replayed but the producer isn't recorded or the entries disagree), `mixed:` (some batches replayed and some answered live, every recorded producer being that same host), bare `mixed` (a replayed entry names a *different* producer, so naming the live host would wrongly credit it — split on `:` to read the host; bare `mixed` yields none), or `unknown` on artifacts migrated from before the field. The `cache:`/`mixed:` producer tags were added by nachos-ai-poc-3#392 and refined by #394; the prefixed host is the recorded producer of the replayed answers, never an inference. Records what `model` cannot — an Azure-hosted run pins the production cache namespace yet still reports the base `model` id. Nullable; non-volatile, so it rides inside `fact_run_digest`. | TEXT | | `record_count` | Number of observation rows in the run | INTEGER | | `scored_count` | Number of rows scored in the run | INTEGER | | `skipped_count` | Number of rows skipped in the run | INTEGER | @@ -532,7 +577,7 @@ load, with `assessor_type = 'human'`). | `scored_at` | Timestamp when the scoring pipeline produced the result set | TIMESTAMPTZ | | `status` | Run status, e.g. `complete` | TEXT | | `scoring_plan_version` | Methodology version that produced this run's scores | TEXT | -| `contract_version` | Envelope-shape version of the emitted records (`SIDECAR_CONTRACT_VERSION`, currently `3`) | TEXT | +| `contract_version` | Envelope-shape version of the emitted records (`SIDECAR_CONTRACT_VERSION`, currently `4`) | TEXT | | `snapshot_id` | Ingestion snapshot this run scored — `{artifact stem}@{digest[:16]}`, already emitted in the sidecar header (ADR 0018). The PRD's `state_requirements_tag`. | TEXT | | `snapshot_digest` | sha256 canonical-form digest of that snapshot — proves the content, not just the edition. Already emitted. | TEXT | | `release_id` | Deterministic hash over the full run identity — the landing zone's idempotency key. Already emitted. | TEXT | @@ -599,6 +644,7 @@ together — not from `fact_observations` alone. | `tier_name` | *(emitted since sidecar contract v3 — nachos-ai-poc-3#318, POC-3 ADR 0020)* Name of the rule-cascade tier that matched (the cascade token, e.g. `tier_3_aggregation`); NULL when the dimension could not evaluate. Display labels and thresholds for it live in `score_labels`, keyed by `scoring_plan_version` and joined on this token. | TEXT | | `confidence_composite` | Minimum confidence across all dimensions: `high`, `medium`, or `low` | TEXT | | `adjusted_nachos_score` | Final NACHOS score after extension and multi-entity adjustments, capped at 4.5 | DOUBLE PRECISION | +| `base_nachos_score` | Raw NACHOS complexity tier the scoring rules assigned this element *before* extension/multi-entity adjustment: 0–3. Landing-derived from `dimensions.nachos_score.value` (the value the rule cascade in `tier_name` produced), the same way `documentation_gap` is surfaced from the `dimensions` payload. The base half of the base/adjusted headline; `adjusted_nachos_score` is the adjusted half. Distinct from `complexity_score`: the two coincide on most rows but diverge where a rule (e.g. concatenation) sets the NACHOS tier independently of the business-logic complexity tier. | INTEGER | | `in_scope` | Whether this element is in scope for NACHOS methodology scoring (the value the run consumed; classification originates on the element record) | BOOLEAN | | `documentation_style` | *(payload field; typed-column materialization is MC-6's call)* `prescriptive`, `conceptual`, `cross_reference`, `regulatory`, or `unspecified` — the extracted fact behind the `documentation_style_tier` dimension. (The PRD calls this business_logic_complexity and its first value "Conditional"; the pipeline's fact name and tokens are used here.) | TEXT | | `adjustment_drivers` | *(emitted since sidecar contract v3 — nachos-ai-poc-3#318, POC-3 ADR 0020)* Array of the pipeline's canonical adjustment tokens: `unnecessary_ext`, `necessary_ext`, `multi_entity`, `fidelity_divergent_explained`, `fidelity_divergent_unclear` — built at the same rule sites as the rendered labels (no string parsing); the byte-pinned substrings of `nachos_justification` remain for display. | JSONB | @@ -608,6 +654,7 @@ together — not from `fact_observations` alone. | `nachos_justification` | Human-readable rule label and adjustment breakdown string | TEXT | | `discovery_lens` | Provenance: `source` for source-doc rows, `spine_anchored` for gap-recovered rows | TEXT | | `documentation_source` | Provenance label: `source_doc`, `swagger`, or `swagger_leaf` | TEXT | +| `documented` | *(emitted since sidecar contract v4 — nachos-ai-poc-3#364, POC-3 ADR 0025)* Whether the state source explicitly documents this element — the headline predicate. Aggregate computes every reported mean, dimension distribution, and review count over the rows where this is true (the run header's `in_scope_count` is their count), so carrying it per-record lets a consumer reproduce the headline population from the score envelope alone, without a cross-artifact join. Distinct from `documentation_source`, which discriminates only on the source lens (every API-model-lens row is `source_doc`) and so cannot stand in for it. | BOOLEAN | | `dimensions` | Per-dimension score objects (value, rule_matched, inputs_used, confidence). The full firing-rule path for a row is the per-dimension `rule_matched` entries plus `nachos_justification`; it stays inspectable without re-running a model. | JSONB | | `fact_provenance` | Per-fact audit trail (value, confidence, downgraded, downgrade_reason, spans). When an analyst fact correction replaced the extracted value at aggregate time, the entry additionally carries `provenance = 'human_corrected'` with confidence forced to `high`, downgrade flags cleared, and spans dropped; the key is emitted only when set, so uncorrected facts are byte-identical to before. Absent `provenance` means model-extracted. NULL on `legacy_import` rows. | JSONB | | `review` | Review block (needs_review, reasons, route) | JSONB | From 8e6b020b8a75d289776f28414e9a87a20f5a729b Mon Sep 17 00:00:00 2001 From: Chris Moffatt Date: Mon, 27 Jul 2026 17:44:58 -0700 Subject: [PATCH 2/2] MC-30: drop the vendored pipeline-records schema copy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit docs/design/schemas/pipeline-records.schema.json was a second copy of an artifact the pipeline generates and owns (publish/pipeline_records_contract.py), kept byte-identical by hand because the generator and the consumer of the contract lived in separate repos. The POC-3 migration brings the generator into this repo, so the copy stops being a sync target — it would be a stale sibling of a file generated a directory away. Delete it and let the migration land the one copy. Database-entity-details pointed at it by repo path in five places. Those now name the companion schema without pinning a path, since the migration settles where it lands, and the two-repo byte-identity discipline goes with the copy. MC-18 still validates emitted records against the schema; what it no longer has is a second copy to enforce identity against. Co-Authored-By: Claude Opus 5 (1M context) --- docs/design/Database-entity-details.md | 60 +- .../schemas/pipeline-records.schema.json | 2703 ----------------- 2 files changed, 29 insertions(+), 2734 deletions(-) delete mode 100644 docs/design/schemas/pipeline-records.schema.json diff --git a/docs/design/Database-entity-details.md b/docs/design/Database-entity-details.md index 38d76c0..1feecb1 100644 --- a/docs/design/Database-entity-details.md +++ b/docs/design/Database-entity-details.md @@ -12,12 +12,12 @@ > the PRD's adjudication field list contradicting its own routing section — is > tracked on the PRD follow-ups list. > -> A machine-readable companion, `schemas/pipeline-records.schema.json`, mirrors every -> entity here as a JSON Schema — emit cores plus landed `*_row` variants derived -> from them. It is generated from POC-3's `publish/pipeline_records_contract.py` -> and committed byte-identical in both repos. It is the artifact the CI contract -> test validates pipeline-emitted records against, so drift between the pipeline -> and the DDL fails a build instead of surfacing at integration. +> A machine-readable companion JSON Schema mirrors every entity here — emit +> cores plus landed `*_row` variants derived from them. It is generated from +> the pipeline's `publish/pipeline_records_contract.py` and lands in this repo +> with the POC-3 migration. It is the artifact the CI contract test validates +> pipeline-emitted records against, so drift between the pipeline and the DDL +> fails a build instead of surfacing at integration. This document lists the entities and fields the pipeline hands the storage database. It is the **logical contract** between the pipeline (producer) and the @@ -99,9 +99,9 @@ Two digests on `fact_runs` close that gap (nachos-ai-poc-3#345): The digest deliberately does not cover `scored_at` or the token/cost/cache counters — those describe the *occurrence*, and a cache-warm replay of identical facts legitimately reports `$0` and different token counts. The exact carve-out, -the recipe, and the ordering tiebreak are published in the schema's -`x-fact-identity` block (`docs/design/schemas/pipeline-records.schema.json`), so -both sides agree on precisely what the digest covers rather than inferring it. +the recipe, and the ordering tiebreak are published in the companion schema's +`x-fact-identity` block, so both sides agree on precisely what the digest +covers rather than inferring it. ADR 0027 (nachos-ai-poc-3#378/#381) adds two non-volatile header fields — `prompt_fingerprint` (the prompt bytes that produced the observations) and @@ -179,11 +179,10 @@ views. This is the lowest-effort shape for the pipeline port (no per-table typed writes to maintain) and keeps every physical choice on the database side of the boundary. What the contract records about it: -- **The emit contract is the payload shape**, pinned by - `schemas/pipeline-records.schema.json`. A JSONB landing zone accepts - anything, so drift would otherwise surface as a broken transform or a - silently-NULL dashboard column. The CI schema check is the primary - drift gate. +- **The emit contract is the payload shape**, pinned by the companion + schema. A JSONB landing zone accepts anything, so drift would otherwise + surface as a broken transform or a silently-NULL dashboard column. The CI + schema check is the primary drift gate. - **Run identity is stamped at emit, or never.** `snapshot_id` / `snapshot_digest`, `release_id`, assessor identity — the transform cannot reconstruct these later; they arrive on every landed payload, and the @@ -899,26 +898,25 @@ to physical tables. ## The CI contract test -The companion `schemas/pipeline-records.schema.json` (JSON Schema, draft 2020-12) -defines every record shape above under `$defs`. POC-3 already commits its own +The companion pipeline-records schema (JSON Schema, draft 2020-12) defines +every record shape above under `$defs`. POC-3 already commits its own contract artifact for the score sidecar — `docs/contracts/assessment-release.schema.json`, generated from `score/release_contract.py`, with an envelope/payload split — and states that the Metadata Catalog CI validates emits against it. The two -must not drift: for score records this file defers to the release contract -(same envelope roster), and adds what it does not cover — element records, -the ingestion run ledger, fact artifacts, curation sidecars, and gap logs. -This file is generated from POC-3's `publish/pipeline_records_contract.py` and -committed **byte-identical** in both repos — POC-3's copy at -`docs/contracts/pipeline-records.schema.json` is validated against every -artifact family on every publish run (POC-3 ADR 0021). Emit cores state what -the pipeline emits; the landed `*_row` variants derive from them by -construction — identical emit-field constraints plus the landing-generated -additions (surrogate keys, import audit stamps, denormalized snapshot -identity) the pipeline can never emit — so emit/landed alignment is -mechanical, not a change-flow discipline. The CI contract test (MC-18) -enforces the byte identity on this side. The contract test validates a -sample of pipeline-emitted records against it on every build, and the -transform/DDL review checks the schema file against what the database extracts. +must not drift: for score records the pipeline-records schema defers to the +release contract (same envelope roster), and adds what it does not cover — +element records, the ingestion run ledger, fact artifacts, curation sidecars, +and gap logs. It is generated from POC-3's `publish/pipeline_records_contract.py` +and validated there against every artifact family on every publish run (POC-3 +ADR 0021); it lands in this repo with the POC-3 migration, which settles the +path it lives at. Emit cores state what the pipeline emits; the landed `*_row` +variants derive from them by construction — identical emit-field constraints +plus the landing-generated additions (surrogate keys, import audit stamps, +denormalized snapshot identity) the pipeline can never emit — so emit/landed +alignment is mechanical, not a change-flow discipline. The CI contract test +(MC-18) validates a sample of pipeline-emitted records against the schema on +every build, and the transform/DDL review checks the schema against what the +database extracts. A field added to the emit without a schema update — or a schema update without a doc update — fails the build. Under the JSONB-landing delivery model the landing zone accepts any payload, so this check is the only place drift fails diff --git a/docs/design/schemas/pipeline-records.schema.json b/docs/design/schemas/pipeline-records.schema.json deleted file mode 100644 index bb51d0e..0000000 --- a/docs/design/schemas/pipeline-records.schema.json +++ /dev/null @@ -1,2703 +0,0 @@ -{ - "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://github.com/touchdownllc/nachos-ai-poc-3/blob/dev/docs/contracts/pipeline-records.schema.json", - "title": "Pipeline records contract — emit cores and landed variants", - "description": "The single pipeline ↔ database contract for every non-score-sidecar artifact family (the score sidecar envelope is assessment-release.schema.json, which the landed score defs defer to). Generated from POC-3 publish/pipeline_records_contract.py and committed byte-identical in BOTH repos: POC-3 docs/contracts/pipeline-records.schema.json (validated against every artifact family on every publish run — ADR 0021) and Metadata-Catalog docs/design/schemas/pipeline-records.schema.json (the MC-18 CI contract test enforces the byte identity and validates landed rows against the *_row / landed defs). Emit cores state what the pipeline emits; landed variants derive from them by construction — identical emit-field constraints plus the landing-generated additions (surrogate keys, import audit stamps, denormalized snapshot identity) the pipeline can never emit. Record types set additionalProperties:false deliberately — an emitted field with no contract entry is drift, and drift must fail; writer-variant payloads (fact headers' long tail, gap-log verbatim extras, evidence payloads: dimensions, fact_provenance, review, raw_payload, header) stay open — their owners' internals churn freely. Regenerate via scripts/refresh_pipeline_records_schema.py; tests/test_pipeline_records_contract.py enforces the same-diff discipline.", - "x-curation-version": 5, - "x-sidecar-contract-version": "3", - "$defs": { - "state_code": { - "type": "string", - "pattern": "^[A-Z]{2}$", - "description": "Two-letter state code, e.g. AZ, WI, MN, TX, IN" - }, - "lens": { - "type": "string", - "enum": [ - "source", - "spine" - ], - "description": "The two parallel, never-averaged views; part of the grain on every scored table" - }, - "record_key": { - "type": "string", - "pattern": "^[A-Z]{2}\\|[^|]+\\|.+$", - "description": "STATE|Entity|element_name - the join key across scored and curated data" - }, - "confidence_level": { - "type": "string", - "enum": [ - "high", - "medium", - "low" - ] - }, - "element_source": { - "type": "string", - "enum": [ - "core", - "extension", - "unknown", - "filtered" - ], - "description": "Element origin" - }, - "documentation_source": { - "type": "string", - "enum": [ - "source_doc", - "swagger", - "swagger_leaf" - ] - }, - "match_confidence": { - "type": "string", - "enum": [ - "high", - "medium", - "low", - "unmatched" - ], - "description": "Entity-to-standard-domain match confidence; unmatched entities are flagged for staff review, never silently dropped" - }, - "cardinality": { - "type": "string", - "enum": [ - "keys", - "required", - "optional", - "optional_conditional" - ] - }, - "assessor_type": { - "type": "string", - "description": "Open string, deliberately not a closed enum (ADR 0018): 'engine' today; 'second-model', 'human', 'synthesis' are ordinary future values, not schema changes. A population-scale sweep over a snapshot; adjudication is not an assessor type." - }, - "data_completeness": { - "type": "string", - "enum": [ - "pipeline_full", - "pipeline_machine_only", - "legacy_import" - ], - "description": "Distinguishes 'absent because legacy' from 'absent because something broke'. legacy_import rows cannot claim full provenance." - }, - "documentation_style": { - "type": "string", - "enum": [ - "prescriptive", - "conceptual", - "cross_reference", - "regulatory", - "unspecified" - ], - "description": "The extracted documentation_style fact behind the documentation_style_tier dimension (the PRD's 'business_logic_complexity'; the PRD's 'Conditional' value is the pipeline's 'prescriptive')." - }, - "adjustment_driver": { - "type": "string", - "enum": [ - "unnecessary_ext", - "necessary_ext", - "multi_entity", - "fidelity_divergent_explained", - "fidelity_divergent_unclear" - ], - "description": "The pipeline's canonical adjustment tokens (aggregate.ADJ_LABEL_*, byte-pinned inside nachos_justification). Emitted as the structured adjustment_drivers array since sidecar contract v3 (nachos-ai-poc-3#318, POC-3 ADR 0020); these are the agreed token spellings." - }, - "state_requirement_basis": { - "type": "string", - "enum": [ - "law_or_regulation", - "state_reporting_mandate", - "state_program_requirement", - "none", - "unresolved" - ], - "description": "Q5 provenance fact (v30): does the state document an independent obligation for this element? The first three values are the positive bases (POC-3 src/poc3/score/rules.py _POSITIVE_STATE_REQUIREMENT_BASES); 'none' is an explicit no-obligation finding; 'unresolved' is incomplete evidence." - }, - "core_can_express_requirement": { - "type": "string", - "enum": [ - "yes", - "no", - "unresolved" - ], - "description": "Q5 core-fit fact (v30): can core Ed-Fi faithfully express the requirement? Candidates come from the state's actual spine catalog. 'no' is a demonstrated core gap; 'unresolved' is absent, incomplete, hallucinated, or wrong-channel evidence." - }, - "extension_necessity": { - "type": "string", - "enum": [ - "necessary", - "unnecessary", - "unresolved" - ], - "description": "Resolved determination since scoring plan v30 (POC-3 PR #284): not an extracted fact - it resolves from state_requirement_basis and core_can_express_requirement via a pure truth table (rules.py resolve_extension_necessity_values): a positive state basis OR core fit 'no' => necessary; basis 'none' AND core fit 'yes' => unnecessary; every other combination => unresolved. The retired extension_is_necessary fact must not appear in any emit." - }, - "extension_necessity_basis": { - "type": "string", - "enum": [ - "state_requirement_and_core_gap", - "state_requirement", - "core_gap", - "core_supported_without_state_requirement", - "unresolved" - ], - "description": "Which truth-table path resolved the necessity. Recorded by the rule stage in the dimensions payload as the extension_necessity_resolution input; listed here so consumers use the exact labels." - }, - "snapshot_id": { - "type": [ - "string", - "null" - ], - "pattern": "^.+@[a-f0-9]{16}$", - "description": "Content identity of the elements artifact a run consumed (ADR 0018): {artifact filename stem}@{sha256 canonical digest, first 16 hex}. The PRD's state_requirements_tag. Null when no elements artifact existed (hermetic fixtures)." - }, - "snapshot_digest": { - "type": [ - "string", - "null" - ], - "pattern": "^[a-f0-9]{64}$", - "description": "sha256 over the canonical form of the input artifact (parsed JSON, volatile run-timestamp keys stripped, sorted keys, compact separators) - same content means same digest across re-ingests and formatting differences. Never recompute from a stored jsonb payload." - }, - "release_id": { - "type": "string", - "description": "Deterministic hash over the full run identity (ADR 0018) - the import adapter's idempotency key; the landing zone dedupes on it and curation decision stamps record it as release_id_at_decision." - }, - "timestamp": { - "type": "string", - "format": "date-time" - }, - "nullable_text": { - "type": [ - "string", - "null" - ] - }, - "fact_name": { - "type": "string", - "minLength": 1, - "not": { - "const": "extension_is_necessary" - }, - "description": "Fact label, e.g. business_rules_present, has_conditional_logic, state_requirement_basis, core_can_express_requirement. The retired extension_is_necessary fact (replaced in scoring plan v30 by the state_requirement_basis / core_can_express_requirement pair) is rejected by contract." - }, - "element_record": { - "type": "object", - "description": "One element record (both lenses; 30 fields — the tests/test_record_schema.py EXPECTED_FIELDS contract). The scope if/then mirrors the MC contract: an out-of-scope classification must carry its rationale.", - "properties": { - "state": { - "type": "string" - }, - "edfi_version": { - "type": "string" - }, - "domain": { - "type": "string", - "description": "Source Area - the state's own grouping vocabulary; empty for swagger-backfill rows. Not the Ed-Fi domain." - }, - "edfi_domain": { - "type": [ - "string", - "null" - ], - "description": "Cross-state Ed-Fi domain, spine-derived; multi-domain entities joined with '; '. Null when the entity resolves to no spine domain." - }, - "match_confidence": { - "enum": [ - "high", - "medium", - "low", - "unmatched" - ] - }, - "match_notes": { - "type": [ - "string", - "null" - ] - }, - "entity": { - "type": "string", - "description": "Normalized entity name - the best-matched Ed-Fi Data Standard entity (the PRD's standard_entity)" - }, - "raw_entity": { - "type": [ - "string", - "null" - ] - }, - "element_name": { - "type": "string" - }, - "element_path": { - "type": [ - "string", - "null" - ] - }, - "data_type": { - "type": [ - "string", - "null" - ] - }, - "cardinality": { - "enum": [ - "keys", - "required", - "optional", - "optional_conditional", - null - ] - }, - "definition_text": { - "type": "string" - }, - "source": { - "enum": [ - "core", - "extension", - "unknown", - "filtered" - ] - }, - "extension_name": { - "type": [ - "string", - "null" - ] - }, - "business_rules_text": { - "type": [ - "string", - "null" - ] - }, - "element_specific_rules": { - "type": [ - "string", - "null" - ] - }, - "regulatory_citations": { - "type": "array", - "items": { - "type": "string" - } - }, - "related_entities": { - "type": "array", - "items": { - "type": "string" - } - }, - "descriptor_table_code": { - "type": [ - "string", - "null" - ] - }, - "descriptor_table_values": { - "type": "array", - "items": { - "type": "object" - } - }, - "collections_text": { - "type": [ - "string", - "null" - ] - }, - "edfi_standard_definition": { - "type": [ - "string", - "null" - ] - }, - "source_document": { - "type": [ - "string", - "null" - ] - }, - "source_page_or_section": { - "type": [ - "string", - "null" - ] - }, - "documented": { - "type": "boolean" - }, - "in_scope": { - "type": [ - "boolean", - "null" - ] - }, - "scope_confidence": { - "enum": [ - "high", - "medium", - "low", - null - ] - }, - "scope_rationale": { - "type": [ - "string", - "null" - ], - "description": "Required (non-null) whenever in_scope is false" - }, - "documentation_source": { - "enum": [ - "source_doc", - "swagger", - "swagger_leaf" - ] - } - }, - "required": [ - "state", - "edfi_version", - "domain", - "edfi_domain", - "match_confidence", - "match_notes", - "entity", - "raw_entity", - "element_name", - "element_path", - "data_type", - "cardinality", - "definition_text", - "source", - "extension_name", - "business_rules_text", - "element_specific_rules", - "regulatory_citations", - "related_entities", - "descriptor_table_code", - "descriptor_table_values", - "collections_text", - "edfi_standard_definition", - "source_document", - "source_page_or_section", - "documented", - "in_scope", - "scope_confidence", - "scope_rationale", - "documentation_source" - ], - "additionalProperties": false, - "allOf": [ - { - "if": { - "properties": { - "in_scope": { - "const": false - } - }, - "required": [ - "in_scope" - ] - }, - "then": { - "properties": { - "scope_rationale": { - "type": "string", - "minLength": 1 - } - }, - "required": [ - "scope_rationale" - ] - } - } - ] - }, - "source_element": { - "$ref": "#/$defs/element_record" - }, - "spine_element": { - "$ref": "#/$defs/element_record" - }, - "elements_artifact": { - "type": "object", - "description": "The {state}_elements_{lens}.json wrapper (models.element.StateElements).", - "properties": { - "state": { - "type": "string" - }, - "edfi_version": { - "type": "string" - }, - "extracted_at": { - "type": "string" - }, - "element_count": { - "type": "integer" - }, - "elements": { - "type": "array", - "items": { - "$ref": "#/$defs/element_record" - } - } - }, - "required": [ - "state", - "edfi_version", - "extracted_at", - "element_count", - "elements" - ], - "additionalProperties": false - }, - "ingestion_run": { - "type": "object", - "description": "One data/out/ingestion_runs.jsonl ledger row (models.ingestion_run.IngestionRunRecord; issue #318 W2). Top-level snapshot identity = the source-lens artifact; the additive spine_snapshot_* pair is POC-3's extension adopted on MC-17.", - "properties": { - "run_id": { - "type": "string", - "pattern": "^[a-f0-9]{16}$", - "description": "Deterministic per-occurrence run identity: sha256 over state|edfi_version|source_digest|spine_digest|prev_run_id, first 16 hex - an EVENT identity chained to the state's previous ledger row. Content idempotency lives in the digest pair: a re-run with unchanged content never appends; a rollback to earlier content appends a new run row (fresh run_id, same digests), so the latest row per state always describes the current artifacts and a run_id never repeats. The landing-side ingestion_run_id surrogate is not emitted (same posture as fact_observation's landing-side keys)." - }, - "state": { - "type": "string", - "pattern": "^[A-Z]{2}$" - }, - "edfi_version": { - "type": [ - "string", - "null" - ] - }, - "snapshot_id": { - "type": [ - "string", - "null" - ], - "pattern": "^.+@[a-f0-9]{16}$" - }, - "snapshot_digest": { - "type": [ - "string", - "null" - ], - "pattern": "^[a-f0-9]{64}$" - }, - "spine_snapshot_id": { - "type": [ - "string", - "null" - ], - "pattern": "^.+@[a-f0-9]{16}$", - "description": "Spine-lens elements artifact identity. One ingestion run produces two lens artifacts, each with its own content identity, and scoring pins per-lens - the ingestion-scoring join is scoring.snapshot_digest IN (snapshot_digest, spine_snapshot_digest). The top-level pair names the source-lens artifact." - }, - "spine_snapshot_digest": { - "type": [ - "string", - "null" - ], - "pattern": "^[a-f0-9]{64}$" - }, - "source_inputs": { - "type": "array", - "items": { - "type": "string" - } - }, - "source_element_count": { - "type": "integer" - }, - "spine_element_count": { - "type": "integer" - }, - "status": { - "enum": [ - "running", - "completed", - "failed" - ] - }, - "started_at": { - "type": [ - "string", - "null" - ] - }, - "finished_at": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "run_id", - "state", - "edfi_version", - "snapshot_id", - "snapshot_digest", - "spine_snapshot_id", - "spine_snapshot_digest", - "source_inputs", - "source_element_count", - "spine_element_count", - "status", - "started_at", - "finished_at" - ], - "additionalProperties": false - }, - "fact_observation": { - "type": "object", - "description": "One phase-A JSONL artifact row (extract._process_payload): the model's raw value and the post-validation value stored separately, with downgrade provenance. The landed fact_observations row is the fact_observation_row variant.", - "properties": { - "record_key": { - "type": "string", - "pattern": "^[A-Z]{2}\\|[^|]+\\|.+$" - }, - "entity": { - "type": "string" - }, - "element_name": { - "type": "string" - }, - "llm_value": { - "type": [ - "boolean", - "integer", - "string", - "null" - ] - }, - "validated_value": { - "type": [ - "boolean", - "integer", - "string", - "null" - ], - "description": "null when the deterministic gate vetoed; scoring consumes this, not llm_value" - }, - "spans": { - "type": "array", - "items": { - "anyOf": [ - { - "type": "object", - "properties": { - "text": { - "type": "string" - }, - "valid": { - "type": "boolean" - } - }, - "required": [ - "text", - "valid" - ], - "additionalProperties": false - }, - { - "type": "string" - } - ] - } - }, - "confidence": { - "enum": [ - "high", - "medium", - "low", - null - ] - }, - "downgrade_reason": { - "type": [ - "string", - "null" - ] - }, - "any_invalid_spans": { - "type": "boolean" - }, - "model": { - "type": [ - "string", - "null" - ] - }, - "prompt_version": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "record_key", - "entity", - "element_name", - "llm_value", - "validated_value", - "spans", - "confidence", - "downgrade_reason", - "model", - "prompt_version" - ], - "additionalProperties": false - }, - "fact_artifact_header": { - "type": "object", - "description": "The header line of a phase-A JSONL artifact. Writers differ (extract / deterministic / peer-gap), so only the identity/status core is pinned and the def stays open. The landed fact_runs parent row is the fact_run def.", - "properties": { - "state": { - "type": "string" - }, - "fact": { - "$ref": "#/$defs/fact_name" - }, - "model": { - "type": "string" - }, - "prompt_version": { - "type": "string" - }, - "record_count": { - "type": "integer" - }, - "scored_count": { - "type": "integer" - }, - "status": { - "type": "string" - } - }, - "required": [ - "state", - "fact", - "model", - "prompt_version", - "record_count", - "scored_count", - "status" - ], - "additionalProperties": true - }, - "gap_log": { - "type": "object", - "description": "The {state}_gap_log.json diagnostic (ingest.shared.build_gap_log). Open beyond the required core: states merge free-form `extra` keys verbatim; alias_tier_histogram is optional. The landed gap_log_row closes the shape.", - "properties": { - "state": { - "type": "string" - }, - "spine_source": { - "type": "string" - }, - "spine_unique_element_keys": { - "type": "integer" - }, - "spine_extension_element_count": { - "type": "integer" - }, - "source_element_count": { - "type": "integer" - }, - "source_coverage": { - "type": "object", - "properties": { - "matched": { - "type": "integer" - }, - "total": { - "type": "integer" - }, - "pct": { - "type": "number" - }, - "note": { - "type": "string" - } - }, - "required": [ - "matched", - "total", - "pct" - ] - }, - "spine_coverage": { - "type": "object", - "properties": { - "matched_unique_keys": { - "type": "integer" - }, - "total_spine_keys": { - "type": "integer" - }, - "pct": { - "type": "number" - }, - "note": { - "type": "string" - } - }, - "required": [ - "matched_unique_keys", - "total_spine_keys", - "pct" - ] - }, - "unmatched_source_count": { - "type": "integer" - }, - "unflatten_recovered_count": { - "type": "integer" - }, - "unflatten_recovered": { - "type": "array" - }, - "unmatched_by_entity": { - "type": "object" - }, - "missing_from_docs_count": { - "type": "integer" - }, - "missing_from_docs_samples": { - "type": "array" - }, - "alias_tier_histogram": { - "type": [ - "object", - "null" - ], - "additionalProperties": { - "type": "integer" - }, - "description": "Optional (issue #213): spine-lens documented matches per alias tier - audits how much match rate rides on the fuzziest tiers" - } - }, - "required": [ - "state", - "spine_source", - "spine_unique_element_keys", - "spine_extension_element_count", - "source_element_count", - "source_coverage", - "spine_coverage", - "unmatched_source_count", - "unflatten_recovered_count", - "unflatten_recovered", - "unmatched_by_entity", - "missing_from_docs_count", - "missing_from_docs_samples" - ], - "additionalProperties": true - }, - "curated_value": { - "type": "object", - "description": "Wrapper for every workbook-band curated value - the sidecar entry `values` captures and the landed curation_records band columns", - "properties": { - "value": {}, - "author": { - "type": [ - "string", - "null" - ], - "description": "Null when the workbook ingest ran without --author (xlsx carries no cell-level authorship)" - }, - "ingested_at": { - "$ref": "#/$defs/timestamp" - }, - "source_workbook": { - "type": [ - "string", - "null" - ], - "description": "Workbook filename; null on CLI-direct captures (poc3 review confirm's band echo)" - }, - "workbook_generated": { - "type": [ - "string", - "null" - ], - "description": "Best-effort read of the workbook's 'Workbook generated' stamp - arbitrary cell text, not guaranteed date-time; null when the workbook lacks the stamp or on CLI-direct captures. Matches the confirmation_block typing." - }, - "lens": { - "$ref": "#/$defs/lens" - }, - "release_id_at_decision": { - "type": [ - "string", - "null" - ], - "description": "Curation v5: recorded on the two override-axis band captures" - } - }, - "required": [ - "value", - "author", - "ingested_at" - ], - "additionalProperties": false - }, - "confirmation_block": { - "type": "object", - "description": "Analyst agreement with the engine (curation v4: 'Reviewed? = Agree' band capture or CLI). A fresh confirmation retires the row's review-queue entry; staleness follows the same engine-value + plan-version model as adjudication.", - "properties": { - "status": { - "const": "confirmed" - }, - "lens": { - "$ref": "#/$defs/lens" - }, - "confirmed_by": { - "type": [ - "string", - "null" - ] - }, - "confirmed_at": { - "$ref": "#/$defs/timestamp" - }, - "source_workbook": { - "type": [ - "string", - "null" - ] - }, - "workbook_generated": { - "type": [ - "string", - "null" - ] - }, - "engine_score_at_confirmation": { - "type": "number" - }, - "plan_version_at_confirmation": { - "type": "string" - }, - "release_id_at_decision": { - "type": [ - "string", - "null" - ], - "description": "Curation v5 - absent on blocks written by pre-v5 writers (sidecars are additive-convergent, so those blocks persist); null when the release was unknown at decision time" - } - }, - "required": [ - "status", - "lens", - "confirmed_at", - "engine_score_at_confirmation", - "plan_version_at_confirmation" - ], - "additionalProperties": false - }, - "adjudication_block": { - "type": "object", - "description": "Emitted by poc3 review adjudicate (curation v5). agreed_by is the PRD's 'participants'. axis is 'adjusted' today; 'extension_necessity' is the PRD-proposed addition. Base tier deliberately excluded: consensus that a tier is wrong is a rule problem (methodology change + plan-version bump).", - "properties": { - "status": { - "const": "adjudicated" - }, - "axis": { - "type": "string", - "enum": [ - "adjusted", - "extension_necessity" - ] - }, - "value": {}, - "lens": { - "$ref": "#/$defs/lens" - }, - "agreed_by": { - "type": "array", - "items": { - "type": "string" - }, - "minItems": 1, - "description": "Who agreed - adjudication is a team determination" - }, - "decided_at": { - "$ref": "#/$defs/timestamp" - }, - "rationale": { - "type": "string", - "minLength": 1, - "description": "Documents the decision; never an input to scoring" - }, - "engine_score_at_decision": { - "type": [ - "number", - "string", - "boolean" - ] - }, - "plan_version_at_decision": { - "type": "string" - }, - "release_id_at_decision": { - "type": [ - "string", - "null" - ], - "description": "Which exact release was contested (curation v5) - additive precision, never a staleness input; absent on blocks written by pre-v5 writers, null on pre-ADR-0018 sidecars" - } - }, - "required": [ - "status", - "axis", - "value", - "lens", - "agreed_by", - "decided_at", - "rationale", - "engine_score_at_decision", - "plan_version_at_decision" - ], - "additionalProperties": false - }, - "fact_correction": { - "type": "object", - "description": "One poc3 review correct-fact block (curation v5 adds release_id_at_decision). LLM facts only; the resolved extension_necessity value is never corrected directly.", - "properties": { - "value": {}, - "lens": { - "$ref": "#/$defs/lens" - }, - "author": { - "type": "string" - }, - "corrected_at": { - "$ref": "#/$defs/timestamp" - }, - "rationale": { - "type": "string", - "minLength": 1 - }, - "prior_value": {}, - "prior_provenance": { - "type": "string", - "pattern": "^(llm|llm_downgraded:.+|human_corrected)$" - }, - "plan_version_at_correction": { - "type": "string" - }, - "release_id_at_decision": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "value", - "lens", - "author", - "corrected_at", - "rationale", - "prior_value", - "prior_provenance", - "plan_version_at_correction" - ], - "additionalProperties": false - }, - "curation_history_item": { - "type": "object", - "description": "One capped history entry. Band-column replacements carry author/source_workbook passthroughs (nullable - a CLI-direct capture had neither); whole-block replacements (column 'adjudication', 'confirmation', or 'fact:{name}') preserve the entire prior block as value with no passthrough keys.", - "properties": { - "column": { - "type": "string" - }, - "value": {}, - "author": { - "type": [ - "string", - "null" - ] - }, - "replaced_at": { - "$ref": "#/$defs/timestamp" - }, - "source_workbook": { - "type": [ - "string", - "null" - ] - } - }, - "required": [ - "column", - "value", - "replaced_at" - ], - "additionalProperties": false - }, - "curation_entry": { - "type": "object", - "description": "One sidecar entry (report/curation.py writers; every writer setdefaults values/history). The known blocks are typed - drift in a block the database consumes fails at publish; keys beyond them belong to future writers (the fact-header-tail posture).", - "properties": { - "values": { - "type": "object", - "description": "Band captures keyed by column (reviewed, required, recommendations, ...) - the landed curation_record enumerates the column roster", - "additionalProperties": { - "$ref": "#/$defs/curated_value" - } - }, - "history": { - "type": "array", - "items": { - "$ref": "#/$defs/curation_history_item" - } - }, - "confirmation": { - "$ref": "#/$defs/confirmation_block" - }, - "adjudication": { - "$ref": "#/$defs/adjudication_block" - }, - "facts": { - "type": "object", - "description": "Analyst corrections keyed by fact name", - "propertyNames": { - "$ref": "#/$defs/fact_name" - }, - "additionalProperties": { - "$ref": "#/$defs/fact_correction" - } - } - }, - "required": [ - "values", - "history" - ], - "additionalProperties": true - }, - "curation_sidecar": { - "type": "object", - "description": "The data/curation/{state}.json sidecar (report/curation.py, currently v5). Entries are typed since the MC-17 unification: the known blocks (values, history, confirmation, adjudication, facts) validate against the shared curation defs; keys beyond them stay open. Sidecars remain additive-convergent across versions.", - "properties": { - "version": { - "type": "integer", - "minimum": 1, - "maximum": 5 - }, - "state": { - "type": "string", - "pattern": "^[A-Z]{2}$" - }, - "updated_at": { - "type": [ - "string", - "null" - ] - }, - "entries": { - "type": "object", - "propertyNames": { - "pattern": "^[A-Z]{2}\\|" - }, - "additionalProperties": { - "$ref": "#/$defs/curation_entry" - } - } - }, - "required": [ - "version", - "state", - "updated_at", - "entries" - ], - "additionalProperties": false - }, - "gap_score_sidecar": { - "type": "object", - "properties": { - "state": { - "type": "string" - }, - "lens": { - "const": "spine_gap" - }, - "edfi_version": { - "type": [ - "string", - "null" - ] - }, - "scored_at": { - "type": "string" - }, - "model": { - "type": "string" - }, - "prompt_version": { - "type": "string" - }, - "scoring_plan_version": { - "type": "string" - }, - "contract_version": { - "const": "3" - }, - "assessor_type": { - "type": "string" - }, - "assessor_id": { - "type": "string" - }, - "snapshot_id": { - "type": [ - "string", - "null" - ] - }, - "snapshot_digest": { - "type": [ - "string", - "null" - ] - }, - "release_id": { - "type": "string" - }, - "record_count": { - "type": "integer" - }, - "scored_count": { - "type": "integer" - }, - "skipped_count": { - "type": "integer" - }, - "mean_quality_score": { - "type": [ - "number", - "null" - ] - }, - "needs_review_count": { - "type": "integer" - }, - "dimension_stats": { - "type": "object" - }, - "in_scope_count": { - "type": "integer" - }, - "nachos_score_histogram": { - "type": "object", - "additionalProperties": { - "type": "integer" - } - }, - "mean_nachos_score": { - "type": [ - "number", - "null" - ] - }, - "gap_source_generated_at": { - "type": [ - "string", - "null" - ] - }, - "gap_discovery_counts": { - "type": "object" - }, - "step": { - "type": [ - "integer", - "string" - ] - }, - "scores": { - "type": "array", - "items": { - "type": "object", - "properties": { - "record_key": { - "type": "string", - "pattern": "^[^|]+\\|[^|]+\\|.+$" - }, - "entity": { - "type": "string" - }, - "element_name": { - "type": "string" - }, - "complexity_score": { - "type": [ - "integer", - "null" - ] - }, - "adjusted_nachos_score": { - "type": [ - "number", - "null" - ] - }, - "in_scope": { - "type": "boolean" - }, - "confidence_composite": { - "enum": [ - "high", - "medium", - "low" - ] - }, - "discovery_lens": { - "type": "string" - }, - "documentation_source": { - "type": "string" - }, - "tier_name": { - "type": [ - "string", - "null" - ] - }, - "adjustment_drivers": { - "type": "array", - "items": { - "enum": [ - "unnecessary_ext", - "necessary_ext", - "multi_entity", - "fidelity_divergent_explained", - "fidelity_divergent_unclear" - ] - } - }, - "extension_necessity": { - "enum": [ - "necessary", - "unnecessary", - "unresolved", - null - ] - }, - "data_completeness": { - "enum": [ - "pipeline_full", - "pipeline_machine_only", - "legacy_import" - ] - }, - "dimensions": { - "description": "method-private payload — free to churn within a contract version" - }, - "fact_provenance": { - "description": "method-private payload — free to churn within a contract version" - }, - "review": { - "description": "method-private payload — free to churn within a contract version" - }, - "nachos_justification": { - "description": "method-private payload — free to churn within a contract version" - }, - "_quality_mean_diagnostic": { - "description": "method-private payload — free to churn within a contract version" - } - }, - "required": [ - "record_key", - "entity", - "element_name", - "complexity_score", - "adjusted_nachos_score", - "in_scope", - "confidence_composite", - "discovery_lens", - "documentation_source", - "tier_name", - "adjustment_drivers", - "extension_necessity", - "data_completeness", - "dimensions", - "fact_provenance", - "review", - "nachos_justification", - "_quality_mean_diagnostic" - ], - "additionalProperties": false - } - } - }, - "required": [ - "state", - "lens", - "edfi_version", - "scored_at", - "model", - "prompt_version", - "scoring_plan_version", - "contract_version", - "assessor_type", - "assessor_id", - "snapshot_id", - "snapshot_digest", - "release_id", - "record_count", - "scored_count", - "skipped_count", - "mean_quality_score", - "needs_review_count", - "dimension_stats", - "in_scope_count", - "nachos_score_histogram", - "mean_nachos_score", - "gap_source_generated_at", - "gap_discovery_counts", - "step", - "scores" - ], - "additionalProperties": false - }, - "element_row": { - "type": "object", - "description": "A landed source_elements / spine_elements row: the emitted element record (identical field constraints by construction) plus the landing-side run key and the snapshot identity denormalized from the elements-artifact envelope.", - "properties": { - "state": { - "type": "string" - }, - "edfi_version": { - "type": "string" - }, - "domain": { - "type": "string", - "description": "Source Area - the state's own grouping vocabulary; empty for swagger-backfill rows. Not the Ed-Fi domain." - }, - "edfi_domain": { - "type": [ - "string", - "null" - ], - "description": "Cross-state Ed-Fi domain, spine-derived; multi-domain entities joined with '; '. Null when the entity resolves to no spine domain." - }, - "match_confidence": { - "enum": [ - "high", - "medium", - "low", - "unmatched" - ] - }, - "match_notes": { - "type": [ - "string", - "null" - ] - }, - "entity": { - "type": "string", - "description": "Normalized entity name - the best-matched Ed-Fi Data Standard entity (the PRD's standard_entity)" - }, - "raw_entity": { - "type": [ - "string", - "null" - ] - }, - "element_name": { - "type": "string" - }, - "element_path": { - "type": [ - "string", - "null" - ] - }, - "data_type": { - "type": [ - "string", - "null" - ] - }, - "cardinality": { - "enum": [ - "keys", - "required", - "optional", - "optional_conditional", - null - ] - }, - "definition_text": { - "type": "string" - }, - "source": { - "enum": [ - "core", - "extension", - "unknown", - "filtered" - ] - }, - "extension_name": { - "type": [ - "string", - "null" - ] - }, - "business_rules_text": { - "type": [ - "string", - "null" - ] - }, - "element_specific_rules": { - "type": [ - "string", - "null" - ] - }, - "regulatory_citations": { - "type": "array", - "items": { - "type": "string" - } - }, - "related_entities": { - "type": "array", - "items": { - "type": "string" - } - }, - "descriptor_table_code": { - "type": [ - "string", - "null" - ] - }, - "descriptor_table_values": { - "type": "array", - "items": { - "type": "object" - } - }, - "collections_text": { - "type": [ - "string", - "null" - ] - }, - "edfi_standard_definition": { - "type": [ - "string", - "null" - ] - }, - "source_document": { - "type": [ - "string", - "null" - ] - }, - "source_page_or_section": { - "type": [ - "string", - "null" - ] - }, - "documented": { - "type": "boolean" - }, - "in_scope": { - "type": [ - "boolean", - "null" - ] - }, - "scope_confidence": { - "enum": [ - "high", - "medium", - "low", - null - ] - }, - "scope_rationale": { - "type": [ - "string", - "null" - ], - "description": "Required (non-null) whenever in_scope is false" - }, - "documentation_source": { - "enum": [ - "source_doc", - "swagger", - "swagger_leaf" - ] - }, - "ingestion_run_id": { - "type": "integer", - "description": "Landing-side surrogate (BIGSERIAL) - never emitted" - }, - "snapshot_id": { - "$ref": "#/$defs/snapshot_id" - } - }, - "required": [ - "state", - "edfi_version", - "domain", - "edfi_domain", - "match_confidence", - "match_notes", - "entity", - "raw_entity", - "element_name", - "element_path", - "data_type", - "cardinality", - "definition_text", - "source", - "extension_name", - "business_rules_text", - "element_specific_rules", - "regulatory_citations", - "related_entities", - "descriptor_table_code", - "descriptor_table_values", - "collections_text", - "edfi_standard_definition", - "source_document", - "source_page_or_section", - "documented", - "in_scope", - "scope_confidence", - "scope_rationale", - "documentation_source", - "ingestion_run_id", - "snapshot_id" - ], - "additionalProperties": false, - "allOf": [ - { - "if": { - "properties": { - "in_scope": { - "const": false - } - }, - "required": [ - "in_scope" - ] - }, - "then": { - "properties": { - "scope_rationale": { - "type": "string", - "minLength": 1 - } - }, - "required": [ - "scope_rationale" - ] - } - } - ] - }, - "ingestion_run_row": { - "type": "object", - "description": "A landed ingestion_runs row: the emitted ledger row (identical field constraints by construction) plus the landing-side surrogate key, the import audit stamp, and operator notes.", - "properties": { - "run_id": { - "type": "string", - "pattern": "^[a-f0-9]{16}$", - "description": "Deterministic per-occurrence run identity: sha256 over state|edfi_version|source_digest|spine_digest|prev_run_id, first 16 hex - an EVENT identity chained to the state's previous ledger row. Content idempotency lives in the digest pair: a re-run with unchanged content never appends; a rollback to earlier content appends a new run row (fresh run_id, same digests), so the latest row per state always describes the current artifacts and a run_id never repeats. The landing-side ingestion_run_id surrogate is not emitted (same posture as fact_observation's landing-side keys)." - }, - "state": { - "type": "string", - "pattern": "^[A-Z]{2}$" - }, - "edfi_version": { - "type": [ - "string", - "null" - ] - }, - "snapshot_id": { - "type": [ - "string", - "null" - ], - "pattern": "^.+@[a-f0-9]{16}$" - }, - "snapshot_digest": { - "type": [ - "string", - "null" - ], - "pattern": "^[a-f0-9]{64}$" - }, - "spine_snapshot_id": { - "type": [ - "string", - "null" - ], - "pattern": "^.+@[a-f0-9]{16}$", - "description": "Spine-lens elements artifact identity. One ingestion run produces two lens artifacts, each with its own content identity, and scoring pins per-lens - the ingestion-scoring join is scoring.snapshot_digest IN (snapshot_digest, spine_snapshot_digest). The top-level pair names the source-lens artifact." - }, - "spine_snapshot_digest": { - "type": [ - "string", - "null" - ], - "pattern": "^[a-f0-9]{64}$" - }, - "source_inputs": { - "type": "array", - "items": { - "type": "string" - } - }, - "source_element_count": { - "type": "integer" - }, - "spine_element_count": { - "type": "integer" - }, - "status": { - "enum": [ - "running", - "completed", - "failed" - ] - }, - "started_at": { - "type": [ - "string", - "null" - ] - }, - "finished_at": { - "type": [ - "string", - "null" - ] - }, - "ingestion_run_id": { - "type": "integer", - "description": "Landing-side surrogate (BIGSERIAL) - never emitted; the emitted run_id is the per-occurrence event identity and the digest pair is the content-idempotency key" - }, - "imported_at": { - "$ref": "#/$defs/timestamp", - "description": "When this row was written to PostgreSQL" - }, - "notes": { - "$ref": "#/$defs/nullable_text", - "description": "Free-form operator notes" - } - }, - "required": [ - "run_id", - "state", - "edfi_version", - "snapshot_id", - "snapshot_digest", - "spine_snapshot_id", - "spine_snapshot_digest", - "source_inputs", - "source_element_count", - "spine_element_count", - "status", - "started_at", - "finished_at", - "ingestion_run_id", - "imported_at" - ], - "additionalProperties": false - }, - "fact_observation_row": { - "type": "object", - "description": "A landed fact_observations row: the emitted artifact row (identical field constraints by construction) plus the landing-side surrogate, the fact_runs parent key, and the landing timestamp.", - "properties": { - "record_key": { - "type": "string", - "pattern": "^[A-Z]{2}\\|[^|]+\\|.+$" - }, - "entity": { - "type": "string" - }, - "element_name": { - "type": "string" - }, - "llm_value": { - "type": [ - "boolean", - "integer", - "string", - "null" - ] - }, - "validated_value": { - "type": [ - "boolean", - "integer", - "string", - "null" - ], - "description": "null when the deterministic gate vetoed; scoring consumes this, not llm_value" - }, - "spans": { - "type": "array", - "items": { - "anyOf": [ - { - "type": "object", - "properties": { - "text": { - "type": "string" - }, - "valid": { - "type": "boolean" - } - }, - "required": [ - "text", - "valid" - ], - "additionalProperties": false - }, - { - "type": "string" - } - ] - } - }, - "confidence": { - "enum": [ - "high", - "medium", - "low", - null - ] - }, - "downgrade_reason": { - "type": [ - "string", - "null" - ] - }, - "any_invalid_spans": { - "type": "boolean" - }, - "model": { - "type": [ - "string", - "null" - ] - }, - "prompt_version": { - "type": [ - "string", - "null" - ] - }, - "fact_observation_id": { - "type": "integer", - "description": "Landing-side surrogate (BIGSERIAL) - never emitted" - }, - "fact_run_id": { - "type": "integer", - "description": "The fact_runs parent row" - }, - "created_at": { - "$ref": "#/$defs/timestamp", - "description": "When this row was landed" - } - }, - "required": [ - "record_key", - "entity", - "element_name", - "llm_value", - "validated_value", - "spans", - "confidence", - "downgrade_reason", - "model", - "prompt_version", - "fact_observation_id", - "fact_run_id", - "created_at" - ], - "additionalProperties": false - }, - "gap_log_row": { - "type": "object", - "description": "A landed gap_logs row. The emit merges writer extras verbatim at top level (open); the landing transform normalizes every non-core key into `extra`, so the landed row is closed - the drift gate the open emit deliberately is not.", - "properties": { - "state": { - "type": "string" - }, - "spine_source": { - "type": "string" - }, - "spine_unique_element_keys": { - "type": "integer" - }, - "spine_extension_element_count": { - "type": "integer" - }, - "source_element_count": { - "type": "integer" - }, - "source_coverage": { - "type": "object", - "properties": { - "matched": { - "type": "integer" - }, - "total": { - "type": "integer" - }, - "pct": { - "type": "number" - }, - "note": { - "type": "string" - } - }, - "required": [ - "matched", - "total", - "pct" - ] - }, - "spine_coverage": { - "type": "object", - "properties": { - "matched_unique_keys": { - "type": "integer" - }, - "total_spine_keys": { - "type": "integer" - }, - "pct": { - "type": "number" - }, - "note": { - "type": "string" - } - }, - "required": [ - "matched_unique_keys", - "total_spine_keys", - "pct" - ] - }, - "unmatched_source_count": { - "type": "integer" - }, - "unflatten_recovered_count": { - "type": "integer" - }, - "unflatten_recovered": { - "type": "array" - }, - "unmatched_by_entity": { - "type": "object" - }, - "missing_from_docs_count": { - "type": "integer" - }, - "missing_from_docs_samples": { - "type": "array" - }, - "alias_tier_histogram": { - "type": [ - "object", - "null" - ], - "additionalProperties": { - "type": "integer" - }, - "description": "Optional (issue #213): spine-lens documented matches per alias tier - audits how much match rate rides on the fuzziest tiers" - }, - "ingestion_run_id": { - "type": "integer" - }, - "extra": { - "type": [ - "object", - "null" - ], - "description": "The emit's free-form top-level extras, collected at landing" - }, - "synced_at": { - "$ref": "#/$defs/timestamp" - } - }, - "required": [ - "state", - "spine_source", - "spine_unique_element_keys", - "spine_extension_element_count", - "source_element_count", - "source_coverage", - "spine_coverage", - "unmatched_source_count", - "unflatten_recovered_count", - "unflatten_recovered", - "unmatched_by_entity", - "missing_from_docs_count", - "missing_from_docs_samples", - "ingestion_run_id", - "synced_at" - ], - "additionalProperties": false - }, - "ingestion_status": { - "type": "object", - "description": "Landing-side per-state/lens ingestion rollup - no emit counterpart; maintained by the landing transform.", - "properties": { - "state": { - "$ref": "#/$defs/state_code" - }, - "lens": { - "$ref": "#/$defs/lens" - }, - "ingestion_run_id": { - "type": "integer" - }, - "snapshot_id": { - "$ref": "#/$defs/snapshot_id" - }, - "edfi_version": { - "$ref": "#/$defs/nullable_text" - }, - "extracted_at": { - "$ref": "#/$defs/timestamp" - }, - "element_count": { - "type": "integer" - }, - "source_inputs": { - "type": "array", - "items": { - "type": "string" - } - }, - "status": { - "type": "string" - }, - "synced_at": { - "$ref": "#/$defs/timestamp" - } - }, - "required": [ - "state", - "lens", - "ingestion_run_id", - "extracted_at", - "element_count", - "status", - "synced_at" - ], - "additionalProperties": false - }, - "fact_run": { - "type": "object", - "description": "A landed fact_runs row - the parent record for a phase-A extraction job, built from the artifact header at landing (no single emit core: headers vary by writer; fact_artifact_header pins the emit-side core).", - "properties": { - "fact_run_id": { - "type": "integer" - }, - "state": { - "$ref": "#/$defs/state_code" - }, - "lens": { - "$ref": "#/$defs/lens" - }, - "ingestion_run_id": { - "type": "integer" - }, - "snapshot_id": { - "$ref": "#/$defs/snapshot_id" - }, - "contract_version": { - "type": "string" - }, - "fact_name": { - "$ref": "#/$defs/fact_name", - "description": "The fact this run extracted - the Database-entity-details.md column name; populated from the artifact header's 'fact' key (fact_artifact_header) at landing" - }, - "artifact_name": { - "$ref": "#/$defs/nullable_text" - }, - "artifact_path": { - "$ref": "#/$defs/nullable_text" - }, - "prompt_version": { - "$ref": "#/$defs/nullable_text" - }, - "model_id": { - "type": "string", - "description": "Model identifier for LLM-based runs; the literal 'deterministic' for det-fact runs" - }, - "mode": { - "type": "string", - "enum": [ - "deterministic", - "llm" - ] - }, - "status": { - "type": "string" - }, - "cost_cap_hit": { - "type": "boolean" - }, - "schema_error": { - "$ref": "#/$defs/nullable_text" - }, - "scored_at": { - "anyOf": [ - { - "$ref": "#/$defs/timestamp" - }, - { - "type": "null" - } - ] - }, - "source_hash": { - "$ref": "#/$defs/nullable_text" - }, - "record_count": { - "type": "integer" - }, - "scored_count": { - "type": "integer" - }, - "skipped_count": { - "type": "integer" - }, - "entities_processed": { - "type": "integer" - }, - "total_tokens_in": { - "type": "integer" - }, - "total_tokens_out": { - "type": "integer" - }, - "total_usd": { - "type": "number" - }, - "cache_hit_count": { - "type": "integer" - }, - "downgrade_count": { - "type": "integer" - }, - "true_count": { - "type": "integer" - }, - "false_count": { - "type": "integer" - }, - "started_at": { - "$ref": "#/$defs/timestamp" - }, - "finished_at": { - "anyOf": [ - { - "$ref": "#/$defs/timestamp" - }, - { - "type": "null" - } - ] - }, - "header": { - "type": "object" - }, - "imported_at": { - "$ref": "#/$defs/timestamp" - }, - "notes": { - "$ref": "#/$defs/nullable_text" - } - }, - "required": [ - "fact_run_id", - "state", - "lens", - "ingestion_run_id", - "snapshot_id", - "contract_version", - "fact_name", - "model_id", - "mode", - "status", - "started_at" - ], - "additionalProperties": false - }, - "scoring_run": { - "type": "object", - "description": "A landed scoring_runs row - run identity from the score sidecar header plus landing context.", - "properties": { - "scoring_run_id": { - "type": "integer" - }, - "state": { - "$ref": "#/$defs/state_code" - }, - "lens": { - "$ref": "#/$defs/lens" - }, - "scored_at": { - "$ref": "#/$defs/timestamp" - }, - "status": { - "type": "string" - }, - "scoring_plan_version": { - "type": "string" - }, - "contract_version": { - "type": "string" - }, - "snapshot_id": { - "$ref": "#/$defs/snapshot_id" - }, - "snapshot_digest": { - "$ref": "#/$defs/snapshot_digest" - }, - "release_id": { - "$ref": "#/$defs/release_id" - }, - "ingestion_run_id": { - "type": "integer" - }, - "assessor_type": { - "$ref": "#/$defs/assessor_type" - }, - "assessor_id": { - "type": "string" - }, - "model": { - "$ref": "#/$defs/nullable_text" - }, - "prompt_version": { - "$ref": "#/$defs/nullable_text" - }, - "data_completeness": { - "$ref": "#/$defs/data_completeness" - }, - "record_count": { - "type": "integer" - }, - "scored_count": { - "type": "integer" - }, - "skipped_count": { - "type": "integer" - }, - "in_scope_count": { - "type": "integer" - }, - "needs_review_count": { - "type": "integer" - }, - "header": { - "type": "object" - }, - "notes": { - "$ref": "#/$defs/nullable_text" - }, - "imported_at": { - "$ref": "#/$defs/timestamp" - } - }, - "required": [ - "scoring_run_id", - "state", - "lens", - "scored_at", - "status", - "scoring_plan_version", - "contract_version", - "snapshot_id", - "snapshot_digest", - "release_id", - "ingestion_run_id", - "assessor_type", - "assessor_id", - "data_completeness" - ], - "additionalProperties": false, - "allOf": [ - { - "if": { - "properties": { - "data_completeness": { - "const": "legacy_import" - } - } - }, - "then": { - "properties": { - "assessor_type": { - "const": "human" - } - } - } - } - ] - }, - "score_record_fields": { - "type": "object", - "description": "The landed score-record payload (one per scored element per run). History storage and the typed current table the transform maintains are MC-6 storage design; this def specifies what lands. Note the landed row is the emitted sidecar record PLUS landing context: state/lens/scored_at come from the sidecar header at import, and scoring_run_id is the landing-side surrogate - the per-record EMIT contract is assessment-release.schema.json, which this def defers to.", - "properties": { - "state": { - "$ref": "#/$defs/state_code" - }, - "lens": { - "$ref": "#/$defs/lens" - }, - "record_key": { - "$ref": "#/$defs/record_key" - }, - "scoring_run_id": { - "type": "integer" - }, - "scored_at": { - "$ref": "#/$defs/timestamp" - }, - "entity": { - "type": "string" - }, - "element_name": { - "type": "string" - }, - "_quality_mean_diagnostic": { - "type": [ - "number", - "null" - ] - }, - "complexity_score": { - "type": [ - "integer", - "null" - ], - "minimum": 0, - "maximum": 3 - }, - "tier_name": { - "$ref": "#/$defs/nullable_text" - }, - "confidence_composite": { - "anyOf": [ - { - "$ref": "#/$defs/confidence_level" - }, - { - "type": "null" - } - ] - }, - "adjusted_nachos_score": { - "type": [ - "number", - "null" - ], - "minimum": 0, - "maximum": 4.5 - }, - "in_scope": { - "type": "boolean" - }, - "documentation_style": { - "anyOf": [ - { - "$ref": "#/$defs/documentation_style" - }, - { - "type": "null" - } - ] - }, - "adjustment_drivers": { - "type": [ - "array", - "null" - ], - "items": { - "$ref": "#/$defs/adjustment_driver" - } - }, - "extension_necessity": { - "anyOf": [ - { - "$ref": "#/$defs/extension_necessity" - }, - { - "type": "null" - } - ], - "description": "Null for core elements" - }, - "documentation_gap": { - "type": [ - "boolean", - "null" - ] - }, - "data_completeness": { - "$ref": "#/$defs/data_completeness" - }, - "nachos_justification": { - "$ref": "#/$defs/nullable_text" - }, - "discovery_lens": { - "type": "string", - "enum": [ - "source", - "spine_anchored" - ] - }, - "documentation_source": { - "$ref": "#/$defs/documentation_source" - }, - "dimensions": { - "type": [ - "object", - "null" - ] - }, - "fact_provenance": { - "type": [ - "object", - "null" - ] - }, - "review": { - "type": [ - "object", - "null" - ] - } - }, - "required": [ - "state", - "lens", - "record_key", - "scoring_run_id", - "scored_at", - "entity", - "element_name", - "in_scope", - "data_completeness" - ], - "additionalProperties": false, - "allOf": [ - { - "if": { - "properties": { - "data_completeness": { - "const": "legacy_import" - } - } - }, - "then": { - "properties": { - "fact_provenance": { - "type": "null" - } - }, - "description": "Loader validation: a legacy row cannot claim full provenance" - }, - "else": { - "properties": { - "dimensions": { - "type": "object" - }, - "fact_provenance": { - "type": "object" - } - }, - "required": [ - "dimensions", - "fact_provenance" - ] - } - } - ] - }, - "score_record": { - "$ref": "#/$defs/score_record_fields" - }, - "score_status": { - "type": "object", - "description": "Landing-side per-state/lens scoring rollup - no emit counterpart; maintained by the landing transform.", - "properties": { - "state": { - "$ref": "#/$defs/state_code" - }, - "lens": { - "$ref": "#/$defs/lens" - }, - "scoring_run_id": { - "type": "integer" - }, - "edfi_version": { - "$ref": "#/$defs/nullable_text" - }, - "scored_at": { - "$ref": "#/$defs/timestamp" - }, - "record_count": { - "type": "integer" - }, - "scored_count": { - "type": "integer" - }, - "skipped_count": { - "type": "integer" - }, - "mean_quality_score": { - "type": [ - "number", - "null" - ] - }, - "needs_review_count": { - "type": "integer" - }, - "scoring_plan_version": { - "type": "string" - }, - "model": { - "$ref": "#/$defs/nullable_text" - }, - "prompt_version": { - "$ref": "#/$defs/nullable_text" - }, - "in_scope_count": { - "type": "integer" - }, - "documentation_gap_count": { - "type": "integer" - }, - "header": { - "type": "object" - }, - "synced_at": { - "$ref": "#/$defs/timestamp" - } - }, - "required": [ - "state", - "lens", - "scoring_run_id", - "scored_at", - "record_count", - "scored_count", - "scoring_plan_version", - "synced_at" - ], - "additionalProperties": false - }, - "score_label": { - "type": "object", - "description": "Plan-version display metadata (score_labels rows) - seeded per scoring plan version, not emitted by runs.", - "properties": { - "scoring_plan_version": { - "type": "string" - }, - "label_type": { - "type": "string", - "enum": [ - "nachos_tier", - "adjusted_band", - "complexity_band", - "threshold" - ] - }, - "value_key": { - "type": "string", - "description": "The labeled key. For nachos_tier this is the rule-cascade token (tier_3_aggregation, ...) - the value score_records.tier_name carries, since two tier-3 rules would collide on a digit key; the numeric tier rides in min_value/max_value. Digit keys where no richer key exists (complexity_band)." - }, - "display_label": { - "type": "string" - }, - "description": { - "$ref": "#/$defs/nullable_text" - }, - "min_value": { - "type": [ - "number", - "null" - ] - }, - "max_value": { - "type": [ - "number", - "null" - ] - }, - "sort_order": { - "type": "integer" - } - }, - "required": [ - "scoring_plan_version", - "label_type", - "value_key", - "display_label", - "sort_order" - ], - "additionalProperties": false - }, - "curation_record": { - "type": "object", - "description": "A landed curation_records row - the sidecar entry flattened to band columns (jsonb per column); the blocks reuse the same defs the emit-side curation_entry validates.", - "properties": { - "state": { - "$ref": "#/$defs/state_code" - }, - "version": { - "type": "integer", - "minimum": 1 - }, - "updated_at": { - "$ref": "#/$defs/timestamp" - }, - "record_key": { - "$ref": "#/$defs/record_key" - }, - "entity": { - "type": "string" - }, - "element_name": { - "type": "string" - }, - "reviewed": { - "$ref": "#/$defs/curated_value" - }, - "analyst_adjusted_override": { - "$ref": "#/$defs/curated_value" - }, - "analyst_base_override": { - "$ref": "#/$defs/curated_value" - }, - "required": { - "$ref": "#/$defs/curated_value" - }, - "recommendations": { - "$ref": "#/$defs/curated_value" - }, - "edfi_comments": { - "$ref": "#/$defs/curated_value" - }, - "ds_next_steps": { - "$ref": "#/$defs/curated_value" - }, - "state_response": { - "$ref": "#/$defs/curated_value" - }, - "kb_reviewed": { - "$ref": "#/$defs/curated_value" - }, - "reviewed_with_state": { - "$ref": "#/$defs/curated_value" - }, - "validated_by": { - "$ref": "#/$defs/curated_value" - }, - "adoption_timeline": { - "$ref": "#/$defs/curated_value" - }, - "commitment_status": { - "$ref": "#/$defs/curated_value" - }, - "commitment_comments": { - "$ref": "#/$defs/curated_value" - }, - "adjudication": { - "$ref": "#/$defs/adjudication_block" - }, - "confirmation": { - "$ref": "#/$defs/confirmation_block" - }, - "facts": { - "type": "object", - "description": "Analyst corrections keyed by fact name; LLM facts only. The v30 pair state_requirement_basis / core_can_express_requirement are each independently correctable; the resolved extension_necessity value is never corrected directly - it recomputes through the truth table on the next aggregate run.", - "propertyNames": { - "$ref": "#/$defs/fact_name" - }, - "properties": { - "state_requirement_basis": { - "allOf": [ - { - "$ref": "#/$defs/fact_correction" - }, - { - "properties": { - "value": { - "$ref": "#/$defs/state_requirement_basis" - } - } - } - ] - }, - "core_can_express_requirement": { - "allOf": [ - { - "$ref": "#/$defs/fact_correction" - }, - { - "properties": { - "value": { - "$ref": "#/$defs/core_can_express_requirement" - } - } - } - ] - } - }, - "additionalProperties": { - "$ref": "#/$defs/fact_correction" - } - }, - "history": { - "type": "array", - "description": "Band-column entries capped at 50; replaced adjudication and fact-correction blocks exempt from the cap", - "items": { - "$ref": "#/$defs/curation_history_item" - } - } - }, - "required": [ - "state", - "version", - "record_key", - "entity", - "element_name" - ], - "additionalProperties": false - } - } -}