Where can I save the agent's trajectory? I

[WEYAI]

Thank you very much for the author's hard work！Where can I save the agent's trajectory? I ran the model I want to test, but I can't find where the trajectory is. I want to analyze what went wrong and how it compares to other models.

[ZhentingWang]

Hi, thank you very much for your question. The agent's trajectory is recorded in execution_summary in line 539 in ./benchmark/evaluator.py. You can save it by dumping this string to a file.

[gganapavarapu]

@ZhentingWang as you suggested execution_summary resulted in attached trajectory for task openapi_explorer_000.

Both the rounds seem to have executed successfully. What does this summary at the top of the file mean?

Total rounds: (2,); Tools executed: 1; Successful: 0; Failed: 1; Failed tools: unknown

mcpbench_tasks_single_runner_format__openapi_explorer_000.txt

[Sehlani042]

I opened PR #40 to add an optional --trajectory-output-dir flag for saving per-task trajectory text files from the same execution summary used for judge evaluation:

#40

The default behavior stays unchanged when the flag is not provided.