ttt_hash_embed and ttt_hash_buckets present in seed 42 log but not in other seeds or train_gpt.py
Record: records/track_10min_16mb/2026-04-09_SP8192_3LayerRecur_ParResid_QK525_LegalTTT
While running experiments based on this record, I noticed a discrepancy across the three seed logs. Seed 42 includes two hyperparameters — ttt_hash_embed: True and ttt_hash_buckets: 16384 — that don't appear in seeds 314 or 999, and don't appear anywhere in the committed train_gpt.py:
| Field |
Seed 42 |
Seed 314 |
Seed 999 |
ttt_hash_embed |
True |
not present |
not present |
ttt_hash_buckets |
16384 |
not present |
not present |
Code size |
16630 bytes |
16594 bytes |
16594 bytes |
quantized_ttt val_bpb |
1.08079 |
1.08103 |
1.08118 |
Seeds 314 and 999 both report Code size: 16594 bytes, consistent with the submitted code. Seed 42 reports 16630 bytes, suggesting it was run with a different version of train_gpt.py that includes the ttt_hash_embed / ttt_hash_buckets feature.
A question:
Could the version of train_gpt.py used for seed 42 be shared? I'm also curious about ttt_hash_embed and ttt_hash_buckets as techniques.
ttt_hash_embedandttt_hash_bucketspresent in seed 42 log but not in other seeds ortrain_gpt.pyRecord:
records/track_10min_16mb/2026-04-09_SP8192_3LayerRecur_ParResid_QK525_LegalTTTWhile running experiments based on this record, I noticed a discrepancy across the three seed logs. Seed 42 includes two hyperparameters —
ttt_hash_embed: Trueandttt_hash_buckets: 16384— that don't appear in seeds 314 or 999, and don't appear anywhere in the committedtrain_gpt.py:ttt_hash_embedTruettt_hash_buckets16384Code sizequantized_ttt val_bpbSeeds 314 and 999 both report
Code size: 16594 bytes, consistent with the submitted code. Seed 42 reports 16630 bytes, suggesting it was run with a different version oftrain_gpt.pythat includes thettt_hash_embed/ttt_hash_bucketsfeature.A question:
Could the version of
train_gpt.pyused for seed 42 be shared? I'm also curious aboutttt_hash_embedandttt_hash_bucketsas techniques.