Fix Hunyuan I2V for transformers>4.47.1

[DN6]

What does this PR do?

We need to manually expand the text_input_ids to include the token ids for the image. The function responsible for it in transformers has been deprecated, and the logic in the current LLaVAProcessor isn't compatible with the way max_sequence_length should be applied in this model.

This PR

1. Adds an _expand_input_ids_with_image_tokens function to replicate some of the behaviour in the LLava model in transformers<4.47.1
   https://github.com/huggingface/transformers/blob/241c04d36867259cdf11dbb4e9d9a60f9cb65ebc/src/transformers/models/llava/modeling_llava.py#L302

This should keep backwards compatibility and work with the newer transformers versions, provided the prompt template and token indices for the image token do not change (seems unlikely?)

Fixes # (issue)
#11118

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[a-r-r-o-w]

LGTM if the outputs match before/after these changes! Prompt template and token indices are very unlikely to change by end users, but even if they do, they need to provide the correct indices to work, so I would assume it should be okay on our end

[a-r-r-o-w]

Maybe could point to a specific commit by adding a comment about where this was adapted from (assuming if it was)

[DN6]

LGTM if the outputs match before/after these changes!

Hmm so prompt embeds do change, because the way the input embeds are created in LLava <=4.47.1 is different from the current version. Old version, the input embeds ended up being zero padded because of this line
https://github.com/huggingface/transformers/blob/241c04d36867259cdf11dbb4e9d9a60f9cb65ebc/src/transformers/models/llava/modeling_llava.py#L325

With the current changes, I think the padding token embedding is used to pad to the end of the sequence. Since the attention mask is passed in both cases, all the differences in the embeddings appear to be where the pad tokens are supposed to be, visually the pipeline outputs look pretty much the same, but numerically the prompt embeds are different.

Old:
https://github.com/user-attachments/assets/77917cc9-a52e-41bf-9ec5-4831fd2f61a4

Current Fix:
https://github.com/user-attachments/assets/2af19aa0-23c4-410e-9638-40d100cd6700

[DN6]

@vladmandic Do you have some time to check if this fixes the issue you were facing?

[vladmandic]

quick test HunyuanVideoImageToVideoPipeline with

• diffusers==hunyuan-i2v-update
• transformers==4.51.1

lgtm!