harness(openai): recover tool-call args corrupted by leaked chat-template delimiters#45
Conversation
…late delimiters
Some OpenAI-compatible gateways fail to strip a model's chat-template
tool-call delimiters (e.g. a trailing `<tool_call|>`) before placing the
call in `function.arguments`. The otherwise-valid JSON then fails to parse,
so the tool call is unusable and the sub-agent produces no valid call —
orphaning the parent's join/reduce (empty reply, no terminal event).
`parse_tool_arguments` (shared by the streaming `OpenAiStreamAcc::into_response`
reduce and the unary `parse_response`) now attempts one conservative repair
on a parse failure before erroring:
1. strip known chat-template tool-call markers and re-parse; else
2. recover the first complete leading JSON object (handles a valid call
followed by trailing marker noise or a second concatenated fragment).
The repair runs only after the raw string has already failed to parse, so
well-formed arguments are never rewritten. If the input is still unparseable
it fails fast with the existing clear, non-retryable Model error — never a
never-resolving tool call that stalls the agent loop.
Adds unit tests for marker-stripping recovery, tag-wrapped args, concatenated
fragments, fail-fast on genuinely malformed args, a valid-args-with-marker
no-op, and the SSE streaming reduce path.
Closes tinyhumansai/openhuman#4766
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 2 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
|
CI guardian: ran |
Sha256 no longer implements std::io::Write, so serde_json::to_writer into the hasher fails to compile (E0277) under `clippy --all-targets -D warnings`. Serialize to a Vec and feed the bytes via Digest::update instead. Pre-existing break on tinyagents main; folded in here to unblock this PR's Rust SDK check.
|
CI guardian: Rust SDK also hit a pre-existing compile error on |
|
CI guardian: fixed |
2 participants
Summary
Fixes the concrete mechanism behind the dominant sub-agent reduce-hang found in the openhuman agent-efficiency eval: a sub-agent produced a
composio_executetool call whosefunction.argumentswere corrupted by a leaked chat-template delimiter (a trailing<tool_call|>marker), so the JSON never parsed, the tool call never resolved, and the parent's join/reduce hung — ending in an empty reply with no terminal event.Root cause (in this crate): Some OpenAI-compatible gateways fail to strip a model's chat-template tool-call delimiters before placing the call in
function.arguments.parse_tool_arguments(insrc/harness/providers/openai/convert.rs, shared by both the streamingOpenAiStreamAcc::into_responsereduce and the unaryparse_response) then fails to parse the otherwise-valid JSON.Change
On a parse failure,
parse_tool_argumentsnow attempts one conservative repair before erroring:<tool_call|>,</tool_call>,<|tool_call|>, and Kimi/Qwen/Hermes-style variants) and re-parse — recovers a valid call whose only corruption is a leaked delimiter.Guarantees:
serde_json::from_str, so well-formed arguments are never rewritten (covered by a test where the marker text sits inside a legitimate string value).Modelerror — never a never-resolving tool call that stalls the agent loop.Tests
Added to
src/harness/providers/openai/test.rs:recovers_tool_args_with_leaked_trailing_template_markerrecovers_tool_args_wrapped_in_tool_call_tagsrecovers_first_call_when_fragments_are_concatenatedstill_fails_fast_on_genuinely_malformed_tool_args(the exact stray-]blob from the eval)valid_tool_args_containing_marker_substring_are_left_intactsse_stream_recovers_tool_args_with_leaked_template_marker(streaming reduce path)Behavior change
Tool calls whose arguments are corrupted only by a leaked chat-template delimiter now execute instead of failing the turn; genuinely malformed arguments still fail closed (unchanged). No public API change.
Closes tinyhumansai/openhuman#4766