Solves #2598

[nouranali]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/2599

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ebsmothers]

Thanks for the PR! Before giving a more detailed review, one high-level comment: I wonder if we really want to handle this for people in general. E.g. with pad_logits it is possible for me to run KD on two totally different tokenizers (and vocab spaces) without seeing any errors, right? This goes beyond just the case of "vocab of teacher model is subset of vocab of student model" (or vice versa).

Anyways I agree that mismatched vocab is an challenge (especially for Qwen models) -- it's come up in a couple issues on the repo before. But I also want to make sure we don't do something behind the scenes that allows someone to accidentally shoot themselves in the foot. Would be interested to hear your thoughts on this though.

[nouranali]

Thanks for the PR! Before giving a more detailed review, one high-level comment: I wonder if we really want to handle this for people in general. E.g. with pad_logits it is possible for me to run KD on two totally different tokenizers (and vocab spaces) without seeing any errors, right? This goes beyond just the case of "vocab of teacher model is subset of vocab of student model" (or vice versa).

Anyways I agree that mismatched vocab is an challenge (especially for Qwen models) -- it's come up in a couple issues on the repo before. But I also want to make sure we don't do something behind the scenes that allows someone to accidentally shoot themselves in the foot. Would be interested to hear your thoughts on this though.

When I run KD for Qwen models that don't have the same tokenizer I get size mismatch exception, Qwen14B as teacher and Qwen3B as student, that's why I added the padding solution to my local code and after #2589 is accepted other people who run the recipe will face the same error.
I also agree with you that it's not necessary for this exception to happen but in my case, using a chat_dataset in openai format it does happen as not all entries have the same size