Skip to content

Harness: fatal tool-arg schema validation aborts the whole turn (legacy loop self-corrected) #4451

Description

@senamakel

Part of the v0.58.7 → HEAD TinyAgents-migration audit (see parent issue).

Problem

The tinyagents 1.3 harness runs fatal JSON-schema validation on every tool call before execution (tinyagents-1.3.0/src/harness/agent_loop/mod.rs:618validate_call, enforcing required, types, and enum). Any violation returns TinyAgentsError::Validation, which propagates out of run_loop and fails the entire turn with a generic "tinyagents harness run failed: Validation(...)".

The legacy engine (v0.58.7 engine/core.rs / engine/tools.rs) had no pre-execution schema gate: args went straight to tool.execute(), and argument errors came back as recoverable tool results the model could read and self-correct on the next iteration.

ArgRecoveryMiddleware (src/openhuman/tinyagents/middleware.rs:1449) makes it worse: it coerces non-object arguments to {} — which then guarantees a "<field> is required" validation abort for any tool with required params (most: e.g. src/openhuman/threads/tools.rs:121, src/openhuman/agent_memory/tools.rs:58, billing tools). Its own comment shows the failure mode was understood, but the fix only covers the non-object case.

Failure scenario

Model emits query_memory with {"q": "…"} instead of {"query": "…"} (or truncated/streaming-mangled JSON args) → whole chat turn errors → user sees chat_error. On the subagent path the whole subagent run dies the same way. This is a routine model behavior that the old loop absorbed; it will be hit in normal operation.

Fix plan

  1. Convert schema-validation failures into model-visible tool errors instead of run aborts. Options, in preference order:
    a. If the crate exposes a policy for validation failures (analogous to UnknownToolPolicy::ReturnToolError), enable it in assemble_turn_harness (src/openhuman/tinyagents/mod.rs).
    b. Otherwise, pre-validate in our adapter: in ArgRecoveryMiddleware::before_tool (or a new SchemaGuardMiddleware), run the same validate_call check ourselves; on failure, short-circuit the call and inject a synthetic failed ToolResult (error: Some("invalid arguments for <tool>: <detail>. Expected schema: …")) so the loop continues and the model self-corrects — mirroring the legacy behavior.
    c. If neither is possible, wrap SharedToolAdapter::execute tools with permissive schemas (required: []) at registration and let each tool's own argument handling produce the recoverable error (tools already do this — they predate the schema gate).
  2. Fix ArgRecoveryMiddleware to stop coercing to {} when the schema has required fields — recover what it can (e.g. parse JSON-encoded-string args, strip markdown fences), otherwise fall through to the tool-error path from (1).
  3. Upstream (tinyagents repo): file for a ValidationPolicy::ReturnToolError run-policy option.

Acceptance criteria

  • A turn where the model sends a missing-required-field / wrong-type / bad-enum tool call continues, the model receives a descriptive tool error, and typically self-corrects in the next iteration.
  • No path exists where a single malformed tool call converts to a turn-level chat_error.
  • Regression tests: chat-path and subagent-path e2e where a scripted provider emits one malformed call then a corrected one (extend tests/agent_harness_e2e.rs).

Metadata

Metadata

Assignees

Labels

agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.bugpriority: criticalBlocks boot or core functionalityrust-coreCore Rust runtime in src/: CLI, core_server, shared infrastructure.subtaskSubtask of a larger tracked effort.

Type

No type

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions