Skip to content

Commit be59d15

Browse files
committed
fix: two more guards that could not fail, in the shapes today keeps producing
FRESHNESS, FAIL-OPEN. The 180-day wholesale-stale floor ran only `if (boardNewest > 0)`, so the one board it could not judge - none carrying a resolvable displayed stamp - was the one it waved through. Publishing generated_at=now over a board whose age cannot be established at all is exactly what that floor exists to refuse, so it is now a hard failure with its own reason. Test confirmed RED against the old branch. ONE-API, VERIFIED ONE MODEL OF THREE. The setup creates three channels (openai, anthropic, gemini) and then proved only that gpt-4o-mini relays. A channel that inserted but never became routable was invisible there and surfaced later as the GATEWAY failing: "no available channel for model gemini-1.5-pro", two red cells, in all eight runs of 2026-07-27/28 including the oldest engine. Every model is now relayed before measuring, and a model that does not aborts the run naming itself. A setup this harness got wrong must stop the run, not become a verdict about the gateway. The channel POSTs already aborted on a non-zero exit; this closes the gap where the POST succeeds and the routing still does not work. Both are the same shape as the retry budget nothing called, the box qualification that always seeded, the history appender scanning directories the engine stopped writing, and the site fixtures asserting against an empty board. Silent rather than wrong, which is why none of them were noticed.
1 parent 95aafd2 commit be59d15

3 files changed

Lines changed: 45 additions & 1 deletion

File tree

‎gateways/one-api/commands‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,4 +35,12 @@ curl -sf -b /tmp/oa.jar -X POST http://127.0.0.1:3000/api/channel/ -H 'content-t
3535
# (manifest.rs's loader), so the retry has to be a single line - `;` joined, not real newlines. The
3636
# retry covers the channel rows this file just inserted not yet being visible to the router. No -f on
3737
# the curl: the 401 body is exactly what this loop inspects, and -f discards it on any non-2xx.
38-
curl -s -m 10 -X POST http://127.0.0.1:3000/v1/chat/completions -H 'content-type: application/json' -H 'Authorization: Bearer dummy' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' -o /tmp/oa-auth-check; i=1; while [ $i -le 10 ] && ! grep -q chat.completion /tmp/oa-auth-check; do sleep 0.5; i=$((i+1)); curl -s -m 10 -X POST http://127.0.0.1:3000/v1/chat/completions -H 'content-type: application/json' -H 'Authorization: Bearer dummy' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' -o /tmp/oa-auth-check; done; grep -q chat.completion /tmp/oa-auth-check || { echo "one-api: the declared auth never relayed after 10 checks - INITIAL_ROOT_TOKEN may no longer be honoured by this image: $(cat /tmp/oa-auth-check)" >&2; exit 1; }
38+
#
39+
# EVERY MODEL, NOT JUST THE FIRST. This checked gpt-4o-mini alone while creating three channels, so a
40+
# channel that inserted but never became routable was invisible here and surfaced later as the
41+
# GATEWAY failing: one-api published "no available channel for model gemini-1.5-pro" as two red cells
42+
# in all eight runs of 2026-07-27/28, including the oldest engine. A setup this harness got wrong must
43+
# stop the run and say so, not become a verdict about the gateway. The channel POSTs above already
44+
# abort on a non-zero exit (bin/otb.rs runs each line and stops the run on failure); this closes the
45+
# gap where the POST succeeds and the routing still does not work.
46+
for m in gpt-4o-mini claude-3-5-sonnet-20240620 gemini-1.5-pro; do i=1; ok=0; while [ $i -le 20 ]; do curl -s -m 10 -X POST http://127.0.0.1:3000/v1/chat/completions -H 'content-type: application/json' -H 'Authorization: Bearer dummy' -d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}],\"max_tokens\":1}" -o /tmp/oa-auth-check; if grep -q chat.completion /tmp/oa-auth-check; then ok=1; break; fi; sleep 0.5; i=$((i+1)); done; [ "$ok" = 1 ] || { echo "one-api: model $m never relayed after 20 checks - its channel is registered but not routable, so every cell for that egress would publish a red this harness caused: $(cat /tmp/oa-auth-check)" >&2; exit 1; }; done

‎site/gen-data.mjs‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -616,6 +616,22 @@ const MAX_BOARD_AGE_DAYS = 180; // wholesale-stale floor (soft anchor): the w
616616
// guard the rewrite KEPT) never fires and a wholesale-stale matrix board publishes generated_at=now.
617617
// The floor now ages exactly what the board shows (same basis as MED-1's footer + the per-row badge).
618618
const boardNewest = Math.max(...gateways.map(displayedMeasuredMs), 0);
619+
// A BOARD WHOSE AGE CANNOT BE ESTABLISHED IS NOT A FRESH BOARD.
620+
//
621+
// This guard only ran `if (boardNewest > 0)`, so the one situation it could not judge - no gateway
622+
// carrying a resolvable displayed stamp - was the one it let straight through. That is the same
623+
// shape as every other guard that turned out to be doing nothing today: the retry budget nothing
624+
// called, the box qualification that always seeded, the history appender scanning directories the
625+
// engine stopped writing. Each was silent rather than wrong, which is why none of them were noticed.
626+
//
627+
// Publishing generated_at=now over a board we cannot date is exactly what the 180-day floor exists
628+
// to refuse, so an unresolvable board is a hard failure with its own reason rather than a pass.
629+
if (boardNewest <= 0 && gateways.length > 0) {
630+
throw new Error(
631+
`gen-data: FRESHNESS FAILURE (undatable board): ${gateways.length} gateway(s) but not one carries a resolvable ` +
632+
`displayed measured_at, so the board's age cannot be established at all. Refusing to publish ` +
633+
`generated_at=${generatedAt} over data that cannot be dated.`);
634+
}
619635
if (boardNewest > 0) {
620636
const boardAgeDays = (Date.parse(generatedAt) - boardNewest) / 86400000;
621637
if (boardAgeDays > MAX_BOARD_AGE_DAYS) {

‎site/test.mjs‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3170,3 +3170,23 @@ test("C8: the repo's own instrument-equivalence.json meets the evidence rule it
31703170
}
31713171
}
31723172
});
3173+
// ---- a board whose age cannot be established is not a fresh board -------------------------------
3174+
// The 180-day floor ran only `if (boardNewest > 0)`, so the one case it could not judge was the one
3175+
// it waved through. That is the shape of every guard that turned out to be inert today: the retry
3176+
// budget nothing called, the box qualification that always seeded, the history appender scanning
3177+
// directories nothing writes. Silent, not wrong, which is why none were noticed.
3178+
test("freshness guard REFUSES a board it cannot date, rather than passing it", () => {
3179+
// Gateways present, but no resolvable displayed stamp anywhere.
3180+
const root = mkdtempSync(join(tmpdir(), "site-undatable-"));
3181+
mkdirSync(join(root, "gateways", "alpha"), { recursive: true });
3182+
writeFileSync(join(root, "gateways", "alpha", "definition.json"), JSON.stringify({
3183+
name: "alpha", display: "alpha", lang: "Rust", class: "Gateway", model: "m", port: 1,
3184+
path: "/v1/chat/completions", auth: "dummy", egress: ["openai"],
3185+
matrix: ["100000", "000000", "000000", "000000", "000000", "000000"],
3186+
}));
3187+
mkdirSync(join(root, "results", "snapshots"), { recursive: true });
3188+
const msg = genThrows(root);
3189+
assert.ok(msg, "expected gen-data to THROW on a board with no datable measurement, but it succeeded");
3190+
assert.match(msg, /FRESHNESS FAILURE \(undatable board\)/, `expected the undatable-board failure, got: ${msg}`);
3191+
});
3192+

0 commit comments

Comments
 (0)