Activity

1. vladimirjovanovicTT

   added a parent issue on Jul 29, 2026

   • Training Q3 Goals #672
2. vladimirjovanovicTT

   changed the title [-]Single-chip dense LLMs to 20% MFU[/-] [+][PERF] Single-chip dense LLMs to 20% MFU[/+] on Aug 1, 2026
3. vladimirjovanovicTT

   changed the title [-][PERF] Single-chip dense LLMs to 20% MFU[/-] [+][Perf] Single-chip dense LLMs to 20% MFU[/+] on Aug 1, 2026
4. vladimirjovanovicTT

   assigned

   agobeljicTT

   and

   pglusacTT

   on Aug 24, 2026
5. vladimirjovanovicTT

   added sub-issues on Aug 31, 2026

   • Optimizer level 2 hangs on fused Llama 3.2 1B training step tt-mlir#9189
   • [ttml] Add cross_entropy_fw op tt-mlir#9098
   • [ttml] Add cross_entropy_bw op tt-mlir#9099
   • [ttml] Add layernorm_fw op tt-mlir#9100
   • [ttml] Add layernorm_bw op tt-mlir#9101
   • [ttml] Add rmsnorm_fw op tt-mlir#9102
   • [ttml] Add rmsnorm_bw op tt-mlir#9103
   • [ttml] Integrate TTML ops in EmitPy tt-mlir#9118
   • Add support for ttnn.transformer.chunk_gated_delta_rule to TTIR and TTNN dialects tt-mlir#9243
   • [EmitC] Float attributes formatted with std::to_string; small values silently truncated to 0 tt-mlir#9198
6. 1 remaining item

   Load more
7. vladimirjovanovicTT

   changed the title [-][Perf] Single-chip dense LLMs to 20% MFU[/-] [+][Bringup] Single-chip dense LLMs to 20% MFU[/+] on Sep 7, 2026
8. umalesTT

   assigned

   umalesTT

   and unassigned

   pglusacTT

   on Sep 7, 2026
9. agobeljicTT commented on Sep 8, 2026

   

   agobeljicTT

   on Sep 8, 2026

   Contributor

   More actions

   Update 2026-09-08

   Status

   • tt-kurbla rebased onto tt-mlir main (fork-only commits kept, everything already upstream dropped); the ttml AdamW patch (tensor lr / beta_pow) is now on tt-mlir main.
   • Both ttml lowerings are in kurbla: AdamW (ttir.adamw, eager and torch.compile path) and SDPA (sdpa_fw / sdpa_bw composites promoted to the ttml kernels), plus GQA head-expansion peeling so K/V reach the kernel unexpanded.
   • Multichip (2x2 mesh, data parallel) runs with the same lowerings.

   Results

   Model	opt level	step time	tok/s	TFLOPS/s	MFU	HFU
   Llama 3.2 1B (bs=1, sl=1024, 1 chip)	1	0.825s	1240.7 tok/s	9.05 TFLOP/s	MFU 5.95%	HFU 5.62%
   Llama 3.2 1B (bs=1, sl=1024, 1 chip)	2	0.767s	1335.0 tok/s	9.74 TFLOP/s	MFU 6.40%	HFU 6.05%
   Llama 3.1 8B LoRA r=4 (bs=4, sl=256, 4 chips)	1	3.099s	330.4 tok/s	10.06 TFLOP/s	MFU 1.65%	HFU 1.63%
10. vladimirjovanovicTT

    unassigned

    umalesTT

    on Sep 15, 2026
11. agobeljicTT commented on Sep 15, 2026

    

    agobeljicTT

    on Sep 15, 2026

    Last edited by agobeljicTT

    Contributor

    More actions

    Update 2026-09-15

    Status

    • tt-crank is officially a part of tt-mlir
    • for the sake of thorough tracking of progress, we measure from scratch (main tt-mlir = 646c2d41d0)
    • we have done more offline testing and are now in progress of materializing changes on tt-crank

    Results

    Model	opt level	step time	tok/s	TFLOPS/s	MFU	HFU
    Llama 3.2 1B (bs=1, sl=1024, 1 chip) main	1	3.478s	294.4 tok/s	2.15 TFLOP/s	MFU 1.41%	HFU 1.41%
    Llama 3.2 1B (bs=1, sl=1024, 1 chip) main + ttml.AdamW	1	0.789s	1298.3 tok/s	9.47 TFLOP/s	MFU 6.23%	HFU 6.23%
12. agobeljicTT

    added 4 commits that reference this issue on Sep 17, 2026

    [tt-crank] lower fused AdamW to ttir.adamw (#9302)

    ...

    a03f722

    [tt-crank] lower fused AdamW to ttir.adamw (#9302)

    ...

    1ac96f8

    [tt-crank] max reduction: build_max, aten.amax lowering, max.dim valu…

    ...

    33e83a8

    [tt-crank] lower aten _log_softmax and nll_loss (cross entropy fallba…

    ...

    6e9e3c7
13. agobeljicTT commented on Sep 22, 2026

    

    agobeljicTT

    on Sep 22, 2026

    Last edited by agobeljicTT

    Contributor

    More actions

    Update 2026-09-22

    Status

    • main = b5f1f345ee

    Results

    Model	opt level	step time	tok/s	TFLOPS/s	MFU	HFU
    Llama 3.2 1B (bs=1, sl=1024, 1 chip) main	1	0.783s	1308.6 tok/s	9.54 TFLOP/s	MFU 6.28%	HFU 6.27%
    Llama 3.2 1B (bs=1, sl=1024, 1 chip) main + slice_scatter	1	0.577s	1776.2 tok/s	12.95 TFLOP/s	MFU 8.52%	HFU 8.52%
14. agobeljicTT commented on Sep 25, 2026

    

    agobeljicTT

    on Sep 25, 2026

    Last edited by agobeljicTT

    Contributor

    More actions

    Update 2026-09-25

    Status

    • main = aad5ad863c

    Results

    Model	opt level	step time	tok/s	TFLOPS/s	MFU	HFU
    Llama 3.2 1B (bs=1, sl=1024, 1 chip) main	1	0.574s	1784.7 tok/s	13.01 TFLOP/s	MFU 8.56%	HFU 8.56%
    Llama 3.2 1B (bs=1, sl=1024, 1 chip) main + SDPA (tenstorrent/tt-mlir#9349)	1	0.428s loss 0.4121	2393.3 tok/s	17.45 TFLOP/s	MFU 11.48%	HFU 10.84%
    Llama 3.2 1B (bs=1, sl=2048, 1 chip) main + SDPA (tenstorrent/tt-mlir#9349)	1	0.787s	2602.3 tok/s	20.02 TFLOP/s	MFU 13.17%	HFU 11.79%
15. agobeljicTT commented on Sep 29, 2026

    

    agobeljicTT

    on Sep 29, 2026

    Last edited by agobeljicTT

    Contributor

    More actions

    Update 2026-09-29

    Status

    • main = 14c76446d7

    Results

    Model	opt level	step time	tok/s	TFLOPS/s	MFU	HFU
    Llama 3.2 1B (bs=1, sl=2048, 1 chip) main	1	0.521s loss 2.8750	3928.3 tok/s	30.23 TFLOP/s	MFU 19.88%	HFU 17.80%
    Llama 3.2 1B (bs=1, sl=2048, 1 chip) main + cross entropy tenstorrent/tt-mlir#9324	1	0.496s loss 0.0884	4133.1 tok/s	31.80 TFLOP/s	MFU 20.91%	HFU 18.72%
16. vladimirjovanovicTT

    closed this as completedon Oct 9, 2026
17. agobeljicTT

    added a commit that references this issue on Oct 13, 2026

    [tt-crank] fuse cross entropy onto the ttml cross_entropy_fw/bw compo…

    ...

    240e14d