Visionzip with Qwen2.5-VL

[aurora-xin]

Hello, when I change the code as README using Qwen2.5-VL fine-tuning code (https://github.com/QwenLM/Qwen2.5-VL/tree/main/qwen-vl-finetune), I encountered this problem:

img_mask[first:last+1] = ~select_mask
[rank5]: RuntimeError: The expanded size of the tensor (657) must match the existing size (655) at non-singleton dimension 0.
Target sizes: [657]. Tensor sizes: [655]

After I fix this, I encountered another problem as:
attention_mask = attention_mask[:, img_mask]
[rank0]: IndexError: too many indices for tensor of dimension 1

Could you please tell me how to fix these bugs or how to avoid? thank u so much!!

[zry13]

I encountered the first problem too, and I changed these two lines of code:
img_mask[first:last+1] = ~select_mask
to
img_mask[st_idx] = ~select_mask,
and
hidden_states_filtered = inputs_embeds[:, first:last+1][:,contextual_mask]
to
hidden_states_filtered = inputs_embeds[:, st_idx][:,contextual_mask].
Then it works

[aurora-xin]

I encountered the first problem too, and I changed these two lines of code: img_mask[first:last+1] = ~select_mask to img_mask[st_idx] = ~select_mask, and hidden_states_filtered = inputs_embeds[:, first:last+1][:,contextual_mask] to hidden_states_filtered = inputs_embeds[:, st_idx][:,contextual_mask]. Then it works

thanks it works~ and did you meet the second problem as:
attention_mask = attention_mask[:, img_mask]
[rank7]: IndexError: too many indices for tensor of dimension 1

Could you please tell me which fine-tuning framework did you use? (e.g. llamafactory, or Qwen2.5-VL original code: https://github.com/QwenLM/Qwen2.5-VL/tree/main/qwen-vl-finetune) thx:))

[zry13]

I encountered the first problem too, and I changed these two lines of code: img_mask[first:last+1] = ~select_mask to img_mask[st_idx] = ~select_mask, and hidden_states_filtered = inputs_embeds[:, first:last+1][:,contextual_mask] to hidden_states_filtered = inputs_embeds[:, st_idx][:,contextual_mask]. Then it works

thanks it works~ and did you meet the second problem as: attention_mask = attention_mask[:, img_mask] [rank7]: IndexError: too many indices for tensor of dimension 1

Could you please tell me which fine-tuning framework did you use? (e.g. llamafactory, or Qwen2.5-VL original code: https://github.com/QwenLM/Qwen2.5-VL/tree/main/qwen-vl-finetune) thx:))

Sorry, I didn't meet the second problem. I only used the original Qwen2.5-VL code for inference, not training. This might be a training-related bug that I haven't encountered.