Different evaluation results for UI-TARS-1.5-7B

[ZHWWWWW]

We locally evaluated UI-TARS-1.5-7B, and there was a significant difference between the evaluation results and the data in the table. Our evaluated average accuracy is 73.16 for grounding. Basic and advanced accuracies for fine-grained categories are different as well.

uitars-1.5-7b-local_GUIElementGrounding_score.json

[huiserwang]

Could you please tell me which inference manner you tried? Local or Api? We actually observed obvious difference on the same model between two manners. In our paper, we report the api-style results.

[ZHWWWWW]

Local. Could you provide the evaluation results of local manner？I just want to verify the real performance of UI-TARS, because I got different result in Screenspot-pro compared to the leaderboard. 发自我的iPhone

…

------------------ Original ------------------ From: Huiser WANG ***@***.***> Date: Tue,Aug 26,2025 6:50 PM To: open-compass/MMBench-GUI ***@***.***> Cc: Haiwen Zhang ***@***.***>, Author ***@***.***> Subject: Re: [open-compass/MMBench-GUI] Different evaluation results for UI-TARS-1.5-7B (Issue #5) huiserwang left a comment (open-compass/MMBench-GUI#5) Could you please tell me which inference manner you tried? Local or Api? We actually observed obvious difference on the same model between two manners. In our paper, we report the api-style results. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[coco515alt]

@ZHWWWWW What‘s your result about uitars1.5 in Screenspot-pro? I tried api and also got different result.