Skip to content

Commit b435dc1

Browse files
cschanhniemOpenCode
andauthored
feat: Phase X X5 phase X learning gate schema (XLG-) -- top-level gate asserting MBL+CIT+LPR+RCC all present; closes Phase X learning loop (#853) (#1012)
REQUIRED_X_COMPONENTS=(MBL,CIT,LPR,RCC), VALID_XLG_VERDICTS (3: learning_verified/learning_in_progress/learning_not_started). XComponentCheck helper; verdict: learning_verified (all 4), learning_in_progress (2-3), learning_not_started (0-1). Each artifact_id prefix-validated (MBL-/CIT-/LPR-/RCC-) when non-empty; dry_lab_only=True. 55 tests; BASELINE updated 10443->10498. Completes Phase X. Co-authored-by: OpenCode <opencode@example.com>
1 parent fa18d22 commit b435dc1

4 files changed

Lines changed: 476 additions & 2 deletions

File tree

‎docs/research/NEXT_100_PR_MAP.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -292,4 +292,4 @@ Track whether the pipeline actually improves across batches by capturing per-bat
292292
| X2 | Add calibration improvement tracker schema (CIT-) (complete). — src/openamp_foundry/evidence/calibration_improvement_tracker.py: VALID_CIT_TREND_DIRECTIONS (4: improving/stable/degrading/insufficient_data), VALID_CIT_SUMMARY_GRADES (5: A-D/N/A), MIN_BATCHES_FOR_TREND=2, IMPROVEMENT_THRESHOLD=0.05; BatchHitRateEntry helper; trend auto-computed from first→latest hit_rate delta; insufficient_data forces N/A grade; dry_lab_only=True; 49 tests. | Aggregates MBL records across batches; computes hit-rate trend direction (improving/stable/degrading/insufficient_data); minimum 2 batches required; flags when calibration is not producing measurable improvement. | C |
293293
| X3 | Add learning progress report schema (LPR-) (complete). — src/openamp_foundry/evidence/learning_progress_report.py: VALID_LPR_VERDICTS (4: learning_confirmed/learning_inconclusive/no_learning_signal/insufficient_data), VALID_FEATURE_PREDICTIVITY (3: predictive/not_predictive/uncertain), VALID_FEATURE_CATEGORIES (8); FeatureLearningEntry helper; verdict auto-computed: learning_confirmed (n_pred>n_non), learning_inconclusive (equal+>0), no_learning_signal (n_pred<n_non), insufficient_data (no batches or no definitive features); dry_lab_only=True; 58 tests in tests/evidence/test_learning_progress_report.py. | Human-readable summary of what the pipeline has learned from all batches to date; references CIT- for trend data; includes which candidate features proved predictive vs not; links to calibration decision logs. | C |
294294
| X4 | Add recalibration confidence certificate schema (RCC-) (complete). — src/openamp_foundry/evidence/recalibration_confidence_certificate.py: VALID_RCC_GRADES (A-D), VALID_RCC_VERDICTS (4: high_confidence/moderate_confidence/low_confidence/insufficient_data), VALID_CONSISTENCY_RATINGS (4); BatchCohortEntry helper; grade A (≥4 batches+consistent), B (≥2 batches+consistent/moderately_consistent), D (insufficient_data); cross_batch_consistency auto-computed; total_cohort_size auto-summed; dry_lab_only=True; cit_id/lpr_id prefix-validated; 76 tests. | Asserts with what confidence the current calibration weights are reliable based on cohort size, quality, and consistency across batches; A/B/C/D grade; prevents overconfident calibration claims. | C |
295-
| X5 | Add Phase X learning gate (XLG-). | Top-level gate asserting MBL + CIT + LPR + RCC all present; overall verdict: learning_verified/learning_in_progress/learning_not_started; closes Phase X; no calibration improvement claim is credible without passing this gate. | C |
295+
| X5 | Add Phase X learning gate (XLG-) (complete). — src/openamp_foundry/evidence/phase_x_learning_gate.py: REQUIRED_X_COMPONENTS=(MBL,CIT,LPR,RCC), VALID_XLG_VERDICTS (3: learning_verified/learning_in_progress/learning_not_started); XComponentCheck helper; verdict auto-computed: learning_verified (all 4 present), learning_in_progress (2-3), learning_not_started (0-1); artifact_id prefix-validated per component; dry_lab_only=True; 55 tests. | Top-level gate asserting MBL + CIT + LPR + RCC all present; overall verdict: learning_verified/learning_in_progress/learning_not_started; closes Phase X; no calibration improvement claim is credible without passing this gate. | C |
Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
"""XLG- Phase X learning gate schema.
2+
3+
Top-level gate asserting all four Phase X components are present and
4+
the learning loop is closed: MBL + CIT + LPR + RCC.
5+
6+
No calibration improvement claim is credible without passing this gate.
7+
Verdict: learning_verified / learning_in_progress / learning_not_started.
8+
"""
9+
10+
from __future__ import annotations
11+
12+
from dataclasses import dataclass
13+
14+
REQUIRED_X_COMPONENTS: tuple[str, ...] = ("MBL", "CIT", "LPR", "RCC")
15+
16+
VALID_XLG_VERDICTS: frozenset[str] = frozenset({
17+
"learning_verified",
18+
"learning_in_progress",
19+
"learning_not_started",
20+
})
21+
22+
LEARNING_VERIFIED_REQUIRED_PRESENT: int = 4
23+
LEARNING_IN_PROGRESS_MIN_PRESENT: int = 2
24+
25+
26+
@dataclass
27+
class XComponentCheck:
28+
component_type: str
29+
artifact_id: str
30+
present: bool
31+
32+
33+
@dataclass
34+
class PhaseXLearningGate:
35+
xlg_id: str
36+
pipeline_version: str
37+
component_checks: list[XComponentCheck]
38+
n_components_present: int
39+
xlg_verdict: str
40+
dry_lab_only: bool
41+
limitations: list[str]
42+
created_at: str
43+
44+
45+
def validate_phase_x_learning_gate(xlg: PhaseXLearningGate) -> None:
46+
if not xlg.xlg_id.startswith("XLG-"):
47+
raise ValueError(f"xlg_id must start with 'XLG-': {xlg.xlg_id!r}")
48+
if not xlg.pipeline_version:
49+
raise ValueError("pipeline_version must be non-empty")
50+
if len(xlg.component_checks) != len(REQUIRED_X_COMPONENTS):
51+
raise ValueError(
52+
f"component_checks must have exactly {len(REQUIRED_X_COMPONENTS)} entries"
53+
)
54+
for check in xlg.component_checks:
55+
if check.component_type not in REQUIRED_X_COMPONENTS:
56+
raise ValueError(
57+
f"component_type {check.component_type!r} not in REQUIRED_X_COMPONENTS"
58+
)
59+
expected_prefix = f"{check.component_type}-"
60+
if check.artifact_id and not check.artifact_id.startswith(expected_prefix):
61+
raise ValueError(
62+
f"artifact_id {check.artifact_id!r} must start with {expected_prefix!r}"
63+
)
64+
n_present = sum(1 for c in xlg.component_checks if c.present)
65+
if xlg.n_components_present != n_present:
66+
raise ValueError("n_components_present mismatch")
67+
if xlg.xlg_verdict not in VALID_XLG_VERDICTS:
68+
raise ValueError(
69+
f"xlg_verdict {xlg.xlg_verdict!r} not in VALID_XLG_VERDICTS"
70+
)
71+
if not xlg.dry_lab_only:
72+
raise ValueError("dry_lab_only must be True")
73+
if not xlg.limitations:
74+
raise ValueError("limitations must be non-empty")
75+
if not xlg.created_at:
76+
raise ValueError("created_at must be non-empty")
77+
78+
79+
def _compute_verdict(n_present: int) -> str:
80+
if n_present >= LEARNING_VERIFIED_REQUIRED_PRESENT:
81+
return "learning_verified"
82+
if n_present >= LEARNING_IN_PROGRESS_MIN_PRESENT:
83+
return "learning_in_progress"
84+
return "learning_not_started"
85+
86+
87+
def build_phase_x_learning_gate(
88+
*,
89+
xlg_id: str,
90+
pipeline_version: str,
91+
mbl_artifact_id: str = "",
92+
cit_artifact_id: str = "",
93+
lpr_artifact_id: str = "",
94+
rcc_artifact_id: str = "",
95+
limitations: list[str],
96+
created_at: str,
97+
) -> PhaseXLearningGate:
98+
"""Build a PhaseXLearningGate.
99+
100+
Pass non-empty artifact_id for each component that is present.
101+
An empty artifact_id means the component is absent (present=False).
102+
"""
103+
checks = [
104+
XComponentCheck(
105+
component_type="MBL",
106+
artifact_id=mbl_artifact_id,
107+
present=bool(mbl_artifact_id),
108+
),
109+
XComponentCheck(
110+
component_type="CIT",
111+
artifact_id=cit_artifact_id,
112+
present=bool(cit_artifact_id),
113+
),
114+
XComponentCheck(
115+
component_type="LPR",
116+
artifact_id=lpr_artifact_id,
117+
present=bool(lpr_artifact_id),
118+
),
119+
XComponentCheck(
120+
component_type="RCC",
121+
artifact_id=rcc_artifact_id,
122+
present=bool(rcc_artifact_id),
123+
),
124+
]
125+
n_present = sum(1 for c in checks if c.present)
126+
verdict = _compute_verdict(n_present)
127+
xlg = PhaseXLearningGate(
128+
xlg_id=xlg_id,
129+
pipeline_version=pipeline_version,
130+
component_checks=checks,
131+
n_components_present=n_present,
132+
xlg_verdict=verdict,
133+
dry_lab_only=True,
134+
limitations=limitations,
135+
created_at=created_at,
136+
)
137+
validate_phase_x_learning_gate(xlg)
138+
return xlg
139+
140+
141+
def format_phase_x_learning_gate(xlg: PhaseXLearningGate) -> str:
142+
lines = [
143+
f"Phase X Learning Gate — {xlg.xlg_id}",
144+
f"Pipeline: {xlg.pipeline_version}",
145+
f"Verdict: {xlg.xlg_verdict}",
146+
f"Components present: {xlg.n_components_present}/{len(REQUIRED_X_COMPONENTS)}",
147+
]
148+
lines.append("Component checks:")
149+
for check in xlg.component_checks:
150+
status = "PRESENT" if check.present else "ABSENT"
151+
artifact = f" [{check.artifact_id}]" if check.artifact_id else ""
152+
lines.append(f" {check.component_type}: {status}{artifact}")
153+
lines.append(f"Created: {xlg.created_at}")
154+
lines.append(f"Limitations: {'; '.join(xlg.limitations)}")
155+
lines.append(f"dry_lab_only: {xlg.dry_lab_only}")
156+
return "\n".join(lines)

0 commit comments

Comments
 (0)