Repository navigation
Commit b11f902
fix(xtoken): mark alignment pairs over tokenizer padding invalid
TokenAligner aligns the padded student and teacher sequences, so it
also pairs padding with padding. _drop_padding reset the padded
positions' chunk ids to -1 but left pair_valid True for every pair.
CSD's per-step chunk count, sum(pair_valid * sample_mask), therefore
counted padding pairs: on a padded text batch of ten documents it
counted 7515 pairs for 2330 chunks holding real tokens, which shrank
the KD term (and its logged value) by a batch-dependent factor.
When both attention masks are given, _drop_padding now marks a pair
invalid if it has tokens on both sides and all of its student or all
of its teacher tokens are padding. Pairs that are one-sided by
alignment, such as the student-only start-of-text pair, keep their
flag. The CSD chunk count then equals the number of chunks that hold
real tokens.
X-Token losses are unchanged: their chunk mask already requires real
tokens on both sides, so every pair this marks invalid was already
excluded. CSD's per-chunk terms are unchanged too; only its chunk
count, and so the KD scale, moves. Under dynamic_loss_scaling the
training loss is unchanged, while the logged KD values grow.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Adithya Hanasoge <avenkateshha@nvidia.com>1 parent 9bdc5d3 commit b11f902
3 files changed
Lines changed: 89 additions & 8 deletions
File tree
- nemo_rl/algorithms/x_token
- tests/unit
- algorithms/x_token
- data
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
252 | 252 | | |
253 | 253 | | |
254 | 254 | | |
255 | | - | |
| 255 | + | |
| 256 | + | |
256 | 257 | | |
257 | 258 | | |
258 | 259 | | |
| |||
547 | 548 | | |
548 | 549 | | |
549 | 550 | | |
550 | | - | |
| 551 | + | |
551 | 552 | | |
552 | 553 | | |
553 | 554 | | |
554 | 555 | | |
555 | | - | |
556 | | - | |
557 | | - | |
558 | | - | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
559 | 561 | | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
560 | 566 | | |
561 | 567 | | |
562 | 568 | | |
563 | 569 | | |
564 | 570 | | |
565 | 571 | | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
566 | 586 | | |
567 | 587 | | |
568 | 588 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
288 | 288 | | |
289 | 289 | | |
290 | 290 | | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
291 | 328 | | |
292 | 329 | | |
293 | 330 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
404 | 404 | | |
405 | 405 | | |
406 | 406 | | |
407 | | - | |
| 407 | + | |
| 408 | + | |
408 | 409 | | |
409 | 410 | | |
410 | 411 | | |
| |||
414 | 415 | | |
415 | 416 | | |
416 | 417 | | |
417 | | - | |
| 418 | + | |
418 | 419 | | |
419 | 420 | | |
420 | 421 | | |
| |||
797 | 798 | | |
798 | 799 | | |
799 | 800 | | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
800 | 824 | | |
801 | 825 | | |
802 | 826 | | |
| |||
0 commit comments