Description
Both layer kinds now execute and match their probes (#875, #877), but nothing runs the tower end to end: the scaled embedding, the 48-layer local/global dispatch, the final norm, the tied LM head and the final-logit softcap have never been composed, and the constants that live only at the ends are still unpinned.
The gate cannot be an elementwise tolerance at every width. At one and nine tokens the gap to the reference sits at the bf16 rounding floor and top-1 is exact. At 1024 tokens — exactly the sliding window, the widest prefill that evicts nothing — even two HF attention backends diverge chaotically: sdpa vs eager on the same weights and the same sequence agree on top-1 at only 802/1024 with max |dlogprob| 16.4, spread across all four quarters. A useful window-edge gate is therefore distributional: a global top-1 floor near that baseline, a chaos ceiling on the logprob gap, and a per-quarter top-1 floor bounding localized damage (a whole-quarter failure cannot hide inside a passing global rate) — with our own determinism pinned separately.
Proposed Plan
- One
full_forward: scaled embedding, layer dispatch driven by the parsed layer map, final norm, tied head, softcap from the typed config; geometry derived inside from the weights' config (one source of truth for the buffer sizes handed to CUDA), token ids bounds-checked against the embedding before upload.
- Oracle over the fixture's three cases (one token, nine tokens, 1024 tokens), comparing log-probabilities at the reference's top-64 ids per position; tight elementwise gates plus exact top-1 on the short cases, the distributional gate at the edge, and a bitwise replay assert for our own determinism.
- Derive the embedding scale and assert it against the fixture's recorded bf16 value before it multiplies anything.
- Re-open on the weights only what the forward reads (
config, embed_tokens, norm); carry final_logit_softcapping on the typed config, validated positive at parse.
Acceptance Criteria
- The three cases pass their declared gates — global and per-quarter — in one checkpoint load, and non-finite logprobs fail outright.
- The one- and nine-token cases hold tight elementwise gates with exact top-1; the window-edge case holds the distributional gate (global and per-quarter) at the HF-backend-comparison noise floor; a second edge run is bitwise identical (
to_bits comparison).
- The embedding scale equals the fixture's recorded bf16-rounded value.
Description
Both layer kinds now execute and match their probes (#875, #877), but nothing runs the tower end to end: the scaled embedding, the 48-layer local/global dispatch, the final norm, the tied LM head and the final-logit softcap have never been composed, and the constants that live only at the ends are still unpinned.
The gate cannot be an elementwise tolerance at every width. At one and nine tokens the gap to the reference sits at the bf16 rounding floor and top-1 is exact. At 1024 tokens — exactly the sliding window, the widest prefill that evicts nothing — even two HF attention backends diverge chaotically: sdpa vs eager on the same weights and the same sequence agree on top-1 at only 802/1024 with max |dlogprob| 16.4, spread across all four quarters. A useful window-edge gate is therefore distributional: a global top-1 floor near that baseline, a chaos ceiling on the logprob gap, and a per-quarter top-1 floor bounding localized damage (a whole-quarter failure cannot hide inside a passing global rate) — with our own determinism pinned separately.
Proposed Plan
full_forward: scaled embedding, layer dispatch driven by the parsed layer map, final norm, tied head, softcap from the typed config; geometry derived inside from the weights' config (one source of truth for the buffer sizes handed to CUDA), token ids bounds-checked against the embedding before upload.config,embed_tokens,norm); carryfinal_logit_softcappingon the typed config, validated positive at parse.Acceptance Criteria
to_bitscomparison).