需要冻结Image Encoder的参数吗，我看代码里是全量微调？

[LaFeuilleMorte]

No description provided.

[snakeztc]

currently we tune the entire model, and it also works.

[LaFeuilleMorte]

currently we tune the entire model, and it also works.

Thank you for your reply. Is there any difference between full-tuning and freeze the image encoder. Which option would be better for downstream tasks (visual grounding, classification, detection)?

[SZhanZ]

Hello, you could set freeze_vision_modules as true to freeze the image encoder. In our preliminary experiment, compared with full-finetuning, freezing image encoder seems to be a little better on OOD dataset while comparable on in-domain dataset.