Releases: RalphLabsAI/recipe
Release list
recipe-v0.3.10 — maxmfu-canonical
King: star-dust9023 (hotkey 5HTERMEJ…, uid 124) — recipe "maxmfu-canonical"
| metric | value |
|---|---|
| val_bpb | 1.023364 |
| previous king (5DkeUCZN / HadesHappy) | 1.059665 |
| improvement | −0.036301 |
| benchmark | 0.197333 |
| crowned block | 8,567,947 |
Recipe: canonical 254M (dim 1024 · 16L · 16H · vocab 50257) trained with Muon + WSD (0.55/0.45, 1-sqrt) + split embedding LR (0.009) + torch.compile(max-autotune), determinism off. 5050 steps · batch 1024 · seq 512 · micro 128. Full config in configs/beat/max_mfu.json.
Compute: ~225k tok/s on attested H200 (real TDX + NVIDIA CC), 2.65B tokens, MFU 27.7%, normalized 4.84 H100h (under the 5.0 cap).
Provenance: bundle 6ec294cb…, checkpoint 67b3076a…. Recipe = frozen op4 base c813831 + the validated king delta (recipe/train.py). Independently op4-scored, memorization-gate clean (dd −0.013). Landed from PR #1561.
recipe-v0.3.9 — danielortega-dev
Metrics
- val_bpb:
1.2679 - quality_gain vs previous king:
+0.0334 - compute_cost (H100-hours):
4.5459 - benchmark_accuracy:
0.189
Attribution
- GitHub: @danielortega-dev
- hotkey:
5CXEMm6u6onoMpFPbpSqFLdeDzkG68nyTirSg3SVQfiMVhJa - bundle_hash:
7e5f8241dfed969cd23a85aed6819a6c2145a5a67b8807d9e4066d21757cfecf
Links
Contributors
Assets 2
recipe-v0.3.8 — HadesHappy
Hypothesis
Miner's claim (self-reported, unverified by validator):
Same model as v14 (untied zero-init head, z-loss, qk-norm, U-Net skips, in-model torch.compile/TF32). Two additions: (1) data/dataset.py samples non-overlapping windows via a (seed, epoch)-keyed permutation instead of independent uniform draws — full corpus coverage per epoch wi…
Metrics
- val_bpb:
1.2652 - quality_gain vs previous king:
+0.0361 - compute_cost (H100-hours):
4.8296 - benchmark_accuracy:
0.197
Attribution
- GitHub: @HadesHappy
- hotkey:
5DkeUCZNYrcGneX7wJDUqYxENX7CicSbzFhnbwhpyyDqYok6 - bundle_hash:
902843ca94719b11d8ea8f39335a7c97893888ff56ec96fdcac8a5c181a592de
Reasoning
Miner's claim (self-reported, unverified by validator):
# v15: v14 model + epoch-permutation data sampler + full compute budget
Same model as v14 (untied zero-init head, z-loss, qk-norm, U-Net skips, in-model torch.compile/TF32). Two additions: (1) data/dataset.py samples non-overlapping windows via a (seed, epoch)-keyed permutation instead of independent uniform draws — full corpus coverage per epoch with uniform repeat counts (uniform draws leave ~32% of the 2.2B-token corpus unseen at this budget); determinism/audit contract unchanged. (2) total_steps 5150 = 2.70B tokens, using the full 5.0 H100h budget.Links
Contributors
Assets 2
recipe-v0.3.7 — Kaizen0304
Metrics
- val_bpb:
1.2549 - quality_gain vs previous king:
+0.0172 - compute_cost (H100-hours):
4.8481 - benchmark_accuracy:
0.200
Attribution
- GitHub: @Kaizen0304
- hotkey:
5H3xirPkNrwRRedZYZfTCvUaVaJ9tN945zcEeaAmBNvWa9Dv - bundle_hash:
51c5847f68414f3c7f546242ecf8f4df67b7219a5088819129690ad16a93d68f
Links
Contributors
Assets 2
recipe-v0.3.6 — Kaizen0304
Metrics
- val_bpb:
1.2584 - quality_gain vs previous king:
+0.0137 - compute_cost (H100-hours):
2.0966 - benchmark_accuracy:
0.173
Attribution
- GitHub: @Kaizen0304
- hotkey:
5H3xirPkNrwRRedZYZfTCvUaVaJ9tN945zcEeaAmBNvWa9Dv - bundle_hash:
b1930c27b127c1f0ebaed2485499b7a0eb574cd8b627452ff8955befd3914125
Links
Contributors
Assets 2
recipe-v0.3.5 — danielortega-dev
Metrics
- val_bpb:
1.2721 - quality_gain vs previous king:
+0.0292 - compute_cost (H100-hours):
4.5103 - benchmark_accuracy:
0.195
Attribution
- GitHub: @danielortega-dev
- hotkey:
5CXEMm6u6onoMpFPbpSqFLdeDzkG68nyTirSg3SVQfiMVhJa - bundle_hash:
2725a3cab62346ec20022f79e752cd332fd5d8f2a550d06a1e64b2cc35b10033
Links
Contributors
Assets 2
recipe-v0.3.4 — andreastanm-bot
Metrics
- val_bpb:
1.2804 - quality_gain vs previous king:
+0.0209 - compute_cost (H100-hours):
3.6082 - benchmark_accuracy:
0.177
Attribution
- GitHub: @andreastanm-bot
- hotkey:
5HT3ARMxRtt4dJEVbQU4StgQGsatc7PD48rVCzDLmyP9fytq - bundle_hash:
33da09d02de7d901de4d8128f46087e9d5d2f6ce6cff00773ab501dae5a29d6e
Links
Contributors
Assets 2
recipe-v0.3.3 — nailcutter-mirror
Hypothesis
Miner's claim (self-reported, unverified by validator):
Muon + z-loss regularizer (254M h100_proxy scale)
Metrics
- val_bpb:
1.2845 - quality_gain vs previous king:
+0.0168 - compute_cost (H100-hours):
3.3300 - benchmark_accuracy:
0.187
Attribution
- GitHub: @nailcutter-mirror
- hotkey:
5FTfrwU3NSG5rvBGaGZLGK1H4eEaQ6FP6KZZtrGzjYTuYtay - bundle_hash:
3b29945e87473aae5eb39d7679d65228d086157f605baaa70af51898f3dfcec4
Links
Contributors
Assets 2
recipe-v0.3.2 — everettelages
Hypothesis
Miner's claim (self-reported, unverified by validator):
v0.2.21 readout-calibration arch (logit_scale + per-vocab readout_gain/readout_bias) on the 254M config — a genuinely trained 3800-step WSD run. Held-out val_bpb 1.3013, benchmark 0.198.
Metrics
- val_bpb:
1.3013 - quality_gain vs previous king:
+0.0182 - compute_cost (H100-hours):
4.0284 - benchmark_accuracy:
0.198
Attribution
- GitHub: @everettelages
- hotkey:
5CqhtHE7BE8HgZTnCkhc7rWFSx5jzCQSsRYwhZtLfBBtLHkS - bundle_hash:
09caba0d5964ccc021acfa7561b5eba6b654c01bff0d9dd0068ee32cfa871c43
Reasoning
Miner's claim (self-reported, unverified by validator):
# Readout-calibration arch on the 254M config (genuinely trained)
Real 3800-step WSD run of the 254M config with the v0.2.21 readout-calibration arch (logit_scale + per-vocab readout_gain/bias). Held-out val_bpb 1.3013, benchmark 0.198. 4.03 H100h (more efficient than the king's 4.34).Links
Contributors
Assets 2
recipe-v0.3.1 — martyniukr
Metrics
- val_bpb:
1.3195 - quality_gain vs previous king:
+0.0560 - compute_cost (H100-hours):
4.3371 - benchmark_accuracy:
0.184
Attribution
- GitHub: @martyniukr
- hotkey:
5HBQzPF5oEQ4nLXsYR3iTQ6mZintZp4xdtQLxgKCJgtEUbUt - bundle_hash:
30971aef56998ffe09560ad73bd5208043c2d21c7f114f5b1803d24455ae727e
