Skip to content

Commit 9023c53

Browse files
committed
feat(site): derive what the rig can RESOLVE, so a tie is not published as a ranking
The board can rank two gateways 1.2% apart. It could not say whether that 1.2% was a finding or which box each landed on. This derives the answer from the board itself. MEASURED, NOT CHOSEN. Every box runs the same qualification before it measures anything, and `box_qualify.drift_pct` records how far it landed from the shared baseline. Those boxes are identical by construction - same instance type, same image, same mock - so the spread between the luckiest and unluckiest box IS what the rig cannot tell apart. On the 2026-07-30 field that is 8.27% (-6.22% to +2.05% across 13 boxes). A hard-coded 1% or 2% would have been an undeclared rule deciding which published comparisons count, and this project's position is that no undeclared rule may author a number. The figure also moves with the fleet: a tighter fleet resolves finer, and a noisier one honestly admits it resolves less. WHAT IT SAYS ABOUT TONIGHT'S BOARD, and it is uncomfortable: busbar 46,031 vs litellm-rust 48,394 is 4.9% apart on the one cell both serve, and the rig resolved 8.27%. That comparison did not demonstrate a difference. The wider gaps still stand - 25,101 vs 48,394 is far outside it. Relative to the LARGER value, so the rule means the same at the bottom of the board as at the top: 19 vs 20 is 5% and 49,000 vs 50,000 is 2%, both correctly tied, while their ABSOLUTE gaps differ by three orders of magnitude - which is precisely why an absolute threshold could not serve this board. Null with fewer than two boxes: with one sample there is no spread to observe, and inventing a floor from it would be the magic number this exists to avoid. Nothing may be called indistinguishable without a resolution, since that would assert a tie on a figure we just said we could not derive. One test bug caught writing these: I asserted 19 vs 20 was resolvable, reasoning 1/19 = 5.3%. It is 5% of the larger value, inside the rig's 8.27%, so the code was right and the expectation was wrong. 206 site tests, 0 failures.
1 parent 26c7436 commit 9023c53

2 files changed

Lines changed: 81 additions & 1 deletion

File tree

‎site/app.js‎

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1071,6 +1071,44 @@ function hasCost(data = (typeof state !== "undefined" ? state.data : null)) {
10711071
COST_CACHE.set(data, yes);
10721072
return yes;
10731073
}
1074+
/* rigResolutionPct(data): the smallest DIFFERENCE this rig can actually resolve, derived from the
1075+
board itself rather than chosen.
1076+
1077+
Every box runs the SAME qualification before it measures anything, and `box_qualify.drift_pct` is
1078+
how far that box landed from the shared baseline. Those boxes are identical by construction - same
1079+
instance type, same image, same mock - so the spread between the luckiest and unluckiest box is a
1080+
direct measurement of what the rig cannot tell apart. Two gateways closer than that did not
1081+
demonstrate a difference; they demonstrated which box they happened to land on.
1082+
1083+
DERIVED, NOT PICKED. A hard-coded "1%" or "2%" would be a rule nobody measured deciding which
1084+
published comparisons count - and this project's whole position is that no undeclared rule may
1085+
author a number. The figure moves with the fleet: a tighter fleet resolves finer, and a noisier one
1086+
honestly admits it resolves less.
1087+
1088+
Null when fewer than two boxes report a drift: with one box there is no spread to observe, and
1089+
inventing a floor from a single sample would be the magic number this exists to avoid. */
1090+
const RESOLUTION_CACHE = new WeakMap();
1091+
function rigResolutionPct(data = (typeof state !== "undefined" ? state.data : null)) {
1092+
if (!data || typeof data !== "object") return null;
1093+
if (RESOLUTION_CACHE.has(data)) return RESOLUTION_CACHE.get(data);
1094+
const drifts = (data.gateways || [])
1095+
.map((g) => g && g.rig && g.rig.box_qualify && g.rig.box_qualify.drift_pct)
1096+
.filter((v) => typeof v === "number" && Number.isFinite(v));
1097+
const out = drifts.length >= 2 ? Math.max(...drifts) - Math.min(...drifts) : null;
1098+
RESOLUTION_CACHE.set(data, out);
1099+
return out;
1100+
}
1101+
1102+
/* indistinguishable(a, b, pct): are two published values closer than the rig can resolve?
1103+
Relative to the LARGER value, so the comparison means the same thing at 19 rps and at 49,000. */
1104+
function indistinguishable(a, b, pct) {
1105+
if (typeof pct !== "number" || !(pct > 0)) return false;
1106+
if (typeof a !== "number" || typeof b !== "number") return false;
1107+
const hi = Math.max(Math.abs(a), Math.abs(b));
1108+
if (hi === 0) return true; // two measured zeros are the same measurement
1109+
return (Math.abs(a - b) / hi) * 100 < pct;
1110+
}
1111+
10741112
// The data bundle a (possibly synthetic) state refers to; falls back to the live state's.
10751113
function stateData(st) {
10761114
return (st && st.data) || (typeof state !== "undefined" ? state.data : null);
@@ -4887,7 +4925,7 @@ if (NODE) {
48874925
perCellMemory, memoryCells, hasPerCellMemory, widestDialect, chosenMemory, memoryFor,
48884926
idleAcrossCells, neverPlateaued, worstGrowth, memCellTip, neverPlateauedPill,
48894927
idleStatic, memShape, memGrowing, memShaped,
4890-
hasMatrixGrid, matrixFailureReason, matrixRoster, hasCost,
4928+
hasMatrixGrid, matrixFailureReason, matrixRoster, hasCost, rigResolutionPct, indistinguishable,
48914929
laneRecord, lanePathNote, perfSweepSeries, concAt,
48924930
// THE FRONTIER: the constants (mirrored from seal.mjs and checked against it), the readers every
48934931
// surface goes through, and the two renderers that make the curve's SHAPE legible. Exported because the

‎site/test.mjs‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2246,6 +2246,48 @@ test("stripRigPaths scrubs absolute bench-box paths from diagnostic notes", () =
22462246
assert.ok(!na.note.includes("/home/"));
22472247
});
22482248

2249+
test("NOISE: the rig's resolution is DERIVED from box qualification, never a chosen constant", () => {
2250+
// Every box runs the same qualification before measuring, and identical boxes still land apart.
2251+
// That spread IS what the rig cannot resolve - a measurement, not a policy. A hard-coded 1% or 2%
2252+
// would be an undeclared rule deciding which published comparisons count.
2253+
const board = (drifts) => ({
2254+
gateways: drifts.map((d, i) => ({ key: `g${i}`, rig: { box_qualify: { drift_pct: d } } })),
2255+
});
2256+
// The real 2026-07-30 field: 13 identical boxes spanning -6.22%..+2.05%.
2257+
assert.ok(Math.abs(app.rigResolutionPct(board([-6.22, -1.04, 1.86, 2.05, -0.38])) - 8.27) < 1e-9);
2258+
2259+
// ONE box has no spread to observe. Inventing a floor from a single sample is exactly the magic
2260+
// number this exists to avoid, so it reports that it cannot say.
2261+
assert.equal(app.rigResolutionPct(board([1.5])), null);
2262+
assert.equal(app.rigResolutionPct({ gateways: [] }), null);
2263+
2264+
// A gateway whose qualification is absent contributes nothing rather than counting as 0 drift -
2265+
// a missing measurement is not a perfectly-calibrated box.
2266+
const mixed = { gateways: [{ key: "a", rig: { box_qualify: { drift_pct: -3 } } }, { key: "b" }, { key: "c", rig: { box_qualify: { drift_pct: 2 } } }] };
2267+
assert.equal(app.rigResolutionPct(mixed), 5);
2268+
});
2269+
2270+
test("NOISE: two values closer than the rig can resolve are NOT a ranking", () => {
2271+
// busbar 46,031 vs litellm-rust 48,394 is 4.9% apart, and the 2026-07-30 fleet could only resolve
2272+
// 8.27%. Presenting that as a ranking claims a difference the rig never demonstrated.
2273+
assert.equal(app.indistinguishable(46031, 48394, 8.27), true);
2274+
// A gap wider than the rig's own spread IS a finding.
2275+
assert.equal(app.indistinguishable(25101, 48394, 8.27), false);
2276+
// RELATIVE TO THE LARGER VALUE, so the rule means the same thing at the bottom of the board as at
2277+
// the top. 19 vs 20 is 5% apart and 49,000 vs 50,000 is 2% - both inside a rig that resolves 8.27%,
2278+
// and both correctly tied. The absolute gaps (1 and 1,000) differ by three orders of magnitude,
2279+
// which is exactly why an absolute threshold could not serve this board.
2280+
assert.equal(app.indistinguishable(19, 20, 8.27), true);
2281+
assert.equal(app.indistinguishable(49000, 50000, 8.27), true);
2282+
// And a small-value pair that IS resolvable stays resolvable: 19 vs 25 is 24% apart.
2283+
assert.equal(app.indistinguishable(19, 25, 8.27), false);
2284+
// Two MEASURED zeros are the same measurement, not an undefined ratio.
2285+
assert.equal(app.indistinguishable(0, 0, 8.27), true);
2286+
// With no resolution known (a single box), nothing may be declared indistinguishable: that would
2287+
// be asserting a tie on the strength of a figure we just said we could not derive.
2288+
assert.equal(app.indistinguishable(100, 101, null), false);
2289+
});
2290+
22492291
test("COST: the cost columns appear only on a board that can answer them, and read through the envelope", () => {
22502292
// ADDITIVE, NOT A REPLACEMENT. The existing columns are untouched; these follow the per-cell-memory
22512293
// precedent and appear only when the board carries the field.

0 commit comments

Comments
 (0)