Perf - Training

[pglusacTT]

The discussion tracks the performance of training through tt-blacksmith.

In each entry of the discussion, the following information must be provided:

• tt-blacksmith commit / PR
• tt-xla commit / PR
• measured DEVICE training performance of all of the target models through tt-blacksmith
• measured E2E training performance of all of the target models through tt-blacksmith

Performance tables must be formatted as following:

Model	Target	Current	Percentage
Name	ms	ms	%

Performance percentage is calculated as following:

$$ perf = \frac{target}{current} * 100% $$

[abogdanovicTT]

• tt-blacksmith: 52f78e5
• tt-xla: main

Llama 3.1 70B - 1L (bs=1, seq_len=1024) - Blackhole qb
Llama 3.1 70B (bs=1, seq_len=1024) - Blackhole galaxy
Llama 3.1 8B (bs=1, seq_len=1024) - P150
Qwen 3 8B (bs=1, seq_len=1024) - P150
Llama 3.2 1B (bs=1, seq_len=1024) - P150

E2E:

Model	Target	Current	Percentage
Llama 3.1 70B - 1L	469.1	1430	32,8 %
Llama 3.1 70B	1745.38	29044	6.01 %
Llama 3.1 8B	969.852	9758	9,94 %
Qwen 3 8B	477.324	11266	4,24 %
Llama 3.2 1B	3539.196	13413	26,4%

Device time:

Model	Target	Current	Percentage
Llama 3.1 70B - 1L	82.72	256.79	32,2 %
Llama 3.1 8B	808.21	3095.97	26.11 %
Qwen 3 8B	397.77	5170.70	7.69 %
Llama 3.2 1B	385.77	---	%

[abogdanovicTT]

• tt-blacksmith: cfbd72e- [PERF] Join FWD + BWD and Sync Tensors Precisely
• tt-xla: main

Llama 3.1 70B (bs=1, seq_len=1024) - Blackhole galaxy
Llama 3.1 8B (bs=1, seq_len=512) - P150 - issue #625
Qwen 3 8B (bs=1, seq_len=1024) - P150
Llama 3.2 1B (bs=1, seq_len=1024) - P150

Device time:

Model	Target	Current	Percentage
Qwen 3 8B	397.77	5170.70	7.69 %
Llama 3.1 8B	468.51	3095.97	15.13 %
Llama 3.2 1B	385.77	---	%
Llama 3.1 70B	1454.48	-	- %

E2E:

Model	Target	Current	Percentage
Qwen 3 8B	477.324	11266	4.24 %
Llama 3.1 8B	562.212	4959	11.34 %
Llama 3.2 1B	462.924	13413	3.45%
Llama 3.1 70B	1745.38	9033	19.32 %

[abogdanovicTT]

• tt-blacksmith: 4b4b130 - [PERF] Disable const eval for Llama 3.1 8B - single chip
• tt-xla: main

Llama 3.1 70B (bs=1, seq_len=1024) - Blackhole galaxy
Llama 3.1 8B (bs=1, seq_len=512) - P150 - issue #625
Qwen 3 8B (bs=1, seq_len=1024) - P150
Llama 3.2 1B (bs=1, seq_len=1024) - P150

Device time:

Model	Target	Current	Percentage
Qwen 3 8B	397.77	5170.70	7.69 %
Llama 3.1 8B	468.51	3095.97	15.13 %
Llama 3.2 1B	385.77	---	%
Llama 3.1 70B	1454.48	-	- %

E2E:

Model	Target	Current	Percentage
Qwen 3 8B	477.324	11266	4.24 %
Llama 3.1 8B	562.212	1448	38.8 %
Llama 3.2 1B	462.924	-	- %
Llama 3.1 70B	1745.38	9033	19.32 %

[abogdanovicTT]

• tt-blacksmith: d68ba33 - Join FWD+BWD+OPT for single chip
• tt-xla: main

Llama 3.1 70B (bs=1, seq_len=1024) - Blackhole galaxy
Llama 3.1 8B (bs=1, seq_len=512) - P150 - issue #625
Qwen 3 8B (bs=1, seq_len=1024) - P150
Llama 3.2 1B (bs=1, seq_len=1024) - P150

Device time:

Model	Target	Current	Percentage
Qwen 3 8B	397.77	5170.70	7.69 %
Llama 3.1 8B	468.51	3095.97	15.13 %
Llama 3.2 1B	385.77	---	%
Llama 3.1 70B	1454.48	-	- %

E2E:

Model	Target	Current	Percentage
Qwen 3 8B	477.324	11266	4.24 %
Llama 3.1 8B	562.212	1377	40.83 %
Llama 3.2 1B	462.924	-	- %
Llama 3.1 70B	1745.38	9033	19.32 %

[agobeljicTT]

Model	Theoretical MFU	e2e/step
Llama 3.2 1B (bs=1 sl=1024)	3.43%	1.844s
Llama 3.1 70B (bs=1 sl=256) (lora) (galaxy)	0.20%	10.084s
Llama 3.1 8B (bs=1 sl=512) (lora) (qb2)	8.80% / 14.00%	1.178s
WAN (lora) (galaxy)	0.13%	11.758s