Vllm raise error when token_num exceed max_model_len #403
Description
[rank1]: raise ValueError(
[rank1]:ValueError: The decoder prompt (length 16476) is longer than the maximum model length of 16384. Make sure that max_model_len is no smaller than the number of text tokens plus multimodal tokens. For image inputs, the number of image tokens depends on the number of images, and possibly their aspect ratios as well.
Should filter out those samples instead directly raise error. This error can be easily triggered for vlm when image is large.