How to evaluate the performance of kimi-vl on GUI tasks？

[bill4689]

Thank you for your outstanding work and for open-sourcing such a great model!

I noticed that the technical report included the OSWorld benchmark. Would it be possible to release the prompts that were used for these evaluations?

Additionally, I would like to know whether the model supports deployment on Android platforms (such as the AndroidWorld benchmark) and on the web (such as the WebVoyager benchmark). If so, could you please recommend some prompts for these scenarios?

Thank you very much!

[andersonbcdefg]

Hello, I would also like to see the information needed to reproduce the benchmarks. In my testing with vLLM deployment, I am getting screenspot scores of around 70%, which is much lower than reported in the technical report.