Content of message is "!!!!!!!!!!!!!!!"

[nhha1602]

Hi,

I loaded Kimi vl by vLLM - version 0.9.2:

vllm serve ./Kimi-VL-A3B-Instruct --port 8007 --trust-remote-code --served-model-name Kimi-VL-A3B-Instruct --max-model-len 65536 --max-num-seqs 1024 --max-num-batched-tokens 8192 --swap-space 32 --tensor-parallel-size 1 --enforce-eager

Then I use below code for client and and message to vLLM:

import base64
from PIL import Image
from io import BytesIO
from openai import OpenAI
import os

client = OpenAI(base_url="http://abc.com/v1:8007", api_key="not-needed")

image_path = "test.jpg"
image = Image.open(image_path).convert("RGB")

buffered = BytesIO()
image.save(buffered, format="JPEG")
img_b64_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
base64_image_url = f"data:image/jpeg;base64,{img_b64_str}"

SYSTEM_PROMPT = "You are an AI specialized in recognizing and extracting text from images."
USER_PROMPT = """

• OCR input document.
  """

messages = [
{"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]},
{"role": "user", "content": [{"type": "image_url", "image_url": {"url": base64_image_url}}, {"type": "text", "text": USER_PROMPT}]}
]

completion = client.chat.completions.create(
model="Kimi-VL-A3B-Instruct",
messages=messages,
temperature=0.2,
max_tokens=8192,
)

print(completion.choices[0].message)

But the content is: content='!!!!!!!!!!!!!!!!!!!!!! ...'

Could you, please help to advise this.