Skip to content

Commit 5885037

Browse files
quic-dhirajkuSwati Allabadi
authored andcommitted
InternVL dual QPC model patch (#318)
BUGFIX: added patch for InternVL to have vit_embeds 0th dim as dynamic based on num_patches. Signed-off-by: quic-dhirajku <[email protected]>
1 parent a55f3b8 commit 5885037

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

QEfficient/transformers/models/internvl/modeling_internvl.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,7 @@ def get_onnx_dynamic_axes(self, kv_offload: bool = False):
124124
lang_dynamic_axes = {}
125125
lang_dynamic_axes["input_ids"] = {0: "batch_size", 1: "seq_len"}
126126
lang_dynamic_axes["position_ids"] = {0: "batch_size", 1: "seq_len"}
127+
lang_dynamic_axes["vit_embeds"] = {0: "num_patches"}
127128
vision_dynamic_axes["pixel_values"] = {0: "num_patches", 2: "img_size", 3: "img_size"}
128129

129130
pkv_dynamic_axes = {0: "batch_size", 2: "ctx_len"}

0 commit comments

Comments
 (0)