llama3.1-8B微调后推理没有结果

[Telogen]

微调完毕，按照教程重启模型后，在model.generate部产生警报

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.
The attention mask is not set and cannot be inferred from input because pad token is same as eos token.As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.

但代码仍然可以跑，最终的response为空。

如果添加参数pad_token_id = tokenizer.eos_token_id，警报消失，代码仍然可以跑，但response仍然为空，请问这是怎么回事，谢谢！

[zzdandyy]

+1

[xturan]

我也遇到了同样的问题，我把生成的结果打印出来得到输出如下：
“Generated IDs: [tensor([128009], device='cuda:0')]
<|eot_id|>”
就只有一个输出，并且译码完后是<|eot_id|>。

[Uncle-Yuanl]

1、使用huggingface models里面的config，不要使用hf提供的meta转hf的脚本
2、transformers版本不要低于readme中的版本
3、避免手动指定chat_tempalte。如果保证前两点，应该是不需要手动指定