Skip to content

Commit f8148e4

Browse files
huydhnfacebook-github-bot
authored andcommitted
Add atol to test_within_transformer output comparison (#24723)
Summary: `test_within_transformer` compares the static-attention transformer against the MHA transformer with `torch.isclose(rtol=1e-3)` and the default `atol=1e-8`, which is effectively a pure relative tolerance. Some output logits land near zero (~7e-4), where float noise of ~3e-7 is already a large fraction of the relative budget. The margin depends on which GEMM kernel MKL picks: on master the worst relative error is 3.2e-4 to 6.3e-4 with the default path and 9.5e-4 with `MKL_CBWR=COMPATIBLE`, just under the 1e-3 limit. D129776559 (pytorch/pytorch#197785) makes `Vectorized<float>::exp_u20()`, used by the CPU SDPA softmax, more accurate, which shifts the MHA output enough to cross the limit on CI hosts. Add `atol=1e-5`, matching `test_sliding_window_chunked_prefill` in the same file. The largest absolute error observed is 3.6e-7, while a real attention regression moves outputs far more than 1e-5. This is independent of the PyTorch change and can land before the DiffTrain import. Authored with assistance from Claude. Reviewed By: georgehong Differential Revision: D130128860 Pulled By: huydhn
1 parent ebdb936 commit f8148e4

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

‎examples/models/llama/tests/test_static_attention.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -250,7 +250,9 @@ def test(style, attention_type):
250250
ys.append(y_i)
251251

252252
self.assertTrue(
253-
torch.isclose(ys[-1].flatten(), expected.flatten(), rtol=1e-3).all()
253+
torch.isclose(
254+
ys[-1].flatten(), expected.flatten(), rtol=1e-3, atol=1e-5
255+
).all()
254256
)
255257

256258
for args in itertools.product(

0 commit comments

Comments
 (0)