Skip to content

Commit b3d7ecf

Browse files
committed
ci(bench-tests): gate the probe-verdict choke point and the manifest memory parity
Both defects fixed in the previous three commits were invisible in every existing gate: the site tests read published JSON, and no test ever looked at whether two verdicts for the same observation could differ, or whether a manifest's two memory readers covered the same processes. * new step runs lib/probe_verdict_test.sh — the verdict and the retry budget must be functions of the observation alone, there must be exactly one budget, and matrix/run.sh's transient block must contain no cap() call. * the existing RSS step now also parse-checks all thirteen manifests and runs mem_rss_test.sh's new manifest-parity section (gw_rss/gw_hwm matched pair over one target, no hand-rolled /proc walk). * gateways/*/gateway.sh added to the trigger paths, so a manifest that reintroduces a hand-rolled reader fails CI on the push that does it.
1 parent d50b5c2 commit b3d7ecf

1 file changed

Lines changed: 20 additions & 2 deletions

File tree

‎.github/workflows/bench-tests.yml‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@ on:
1414
- "lib/stream_measure_test.sh"
1515
- "lib/harness.sh"
1616
- "lib/mem_rss_test.sh"
17+
- "lib/probe_verdict.sh"
18+
- "lib/probe_verdict_test.sh"
19+
- "gateways/*/gateway.sh"
1720
- "lib/mock_restart_test.sh"
1821
- "lib/ingress.sh"
1922
- "lib/ingress_path_test.sh"
@@ -37,6 +40,9 @@ on:
3740
- "lib/stream_measure_test.sh"
3841
- "lib/harness.sh"
3942
- "lib/mem_rss_test.sh"
43+
- "lib/probe_verdict.sh"
44+
- "lib/probe_verdict_test.sh"
45+
- "gateways/*/gateway.sh"
4046
- "lib/mock_restart_test.sh"
4147
- "lib/ingress.sh"
4248
- "lib/ingress_path_test.sh"
@@ -71,8 +77,20 @@ jobs:
7177
# did not treat 0 as unmeasured — so a rig-side MEASUREMENT FAILURE was published as a certified
7278
# 0.0 MiB and won first place on the ascending memory ranking (audit P0-2). Gate the
7379
# unmeasurable-prints-nothing contract and the single mem_num_or_null guard directly.
74-
- name: RSS measurement-failure test (unreadable /proc + dead container -> null, never 0)
75-
run: bash -n lib/harness.sh && bash -n lib/mem_rss_test.sh && bash lib/mem_rss_test.sh
80+
# Same file also gates MANIFEST PARITY: gw_rss and gw_hwm must read the SAME process set for
81+
# every one of the 13 gateways (three native manifests read a single pid for rss and the whole
82+
# tree for hwm, publishing two different populations of the same gateway side by side).
83+
- name: RSS measurement-failure + manifest-parity test (null-never-0; gw_rss/gw_hwm same process set)
84+
run: bash -n lib/harness.sh && bash -n lib/mem_rss_test.sh && bash lib/mem_rss_test.sh && for m in gateways/*/gateway.sh; do bash -n "$m" || exit 1; done
85+
# lib/probe_verdict.sh is the ONE choke point deciding how long a persistently-000/5xx cell is
86+
# retried and what verdict is published for it. run_cell used to branch BOTH on the gateway's own
87+
# advisory capability declaration: the same observation (persistent 5xx, mock verifiably healthy)
88+
# was published not_configured when undeclared and not_verified when declared, and the declared
89+
# cell also got 3x120s of retries against the undeclared cell's 2x10s. A gateway's unverified
90+
# claim therefore moved both the published verdict and the effort spent measuring it. Gate the
91+
# observation-only contract and the single uniform budget so the branch cannot come back.
92+
- name: Probe-verdict choke-point test (verdict + retry budget from observables only, one budget for all cells)
93+
run: bash -n lib/probe_verdict.sh && bash -n lib/probe_verdict_test.sh && bash lib/probe_verdict_test.sh
7694
# lib/stream_measure.sh produces EVERY published streams_sustained / cpu_fps / added-TTFT number but
7795
# had ZERO CI coverage (MEDIUM-R2-4): the sustained/cpu-fps bisect+peak searches, the sustained
7896
# SM_MOCK_BOUND decision (the MEDIUM-R2-2 gate's source signal), stream_c1's added-latency honesty

0 commit comments

Comments
 (0)