Skip to content

perf: bound tool I/O and trim tool surface - #172

Closed
huytg2610 wants to merge 4 commits into
SAIL-Research-Lab:mainfrom
huytg2610:perf/bounded-io-hardening
Closed

perf: bound tool I/O and trim tool surface#172
huytg2610 wants to merge 4 commits into
SAIL-Research-Lab:mainfrom
huytg2610:perf/bounded-io-hardening

Conversation

@huytg2610

@huytg2610 huytg2610 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Bounded I/O and a smaller default tool surface

Motivation

Several high-volume tools could do substantial work before their output reached the normal tool-result truncation layer. In practice, this meant a large file, PDF, or slow HTTP response could consume unnecessary memory, latency, and context budget even when the final result was eventually shortened.

At the same time, every agent turn exposed the full tool catalog by default. This increased schema and prompt size for ordinary coding tasks and made it easier for the model to reason about tools that were not relevant to the current task.

This PR moves those limits and decisions closer to the source of the data.

What changed

Bounded input handling

  • Read now processes files incrementally and enforces byte limits while preserving logical line boundaries, including CRLF handling.
  • WebFetch and WebSearch now enforce:
    • response byte limits;
    • a bounded redirect chain;
    • identity-only content encoding;
    • one cancellable deadline covering connection, redirects, and response-body streaming.
  • PDF extraction and large-file summarization now have explicit limits for source size, extracted text, page count, chunk output, and reduce-stage input.
  • Large-file redirects no longer suggest SummarizeLargeFile when that tool is not available in the active profile.

Tool profiles

The default standard profile contains the common coding tools only. Research, orchestration, and full tool surfaces are explicit opt-ins.

The selected profile is now applied consistently across:

  • tool schemas sent to the provider;
  • runtime dispatch;
  • prompt instructions;
  • web settings;
  • /config validation; and
  • cache keys for profile-sensitive tool results.

Existing configuration files without tool_profile now use the compact standard default. Users who need the complete legacy surface can explicitly select full.

Prompt and cache correctness

  • The system prompt lists the active tool surface instead of maintaining a broad static catalog.
  • Planning guidance is included only when EnterPlanMode and ExitPlanMode are active.
  • Tmux guidance is included only when the corresponding registered tool is active.
  • Read-only tool cache entries are invalidated before and after file mutations to avoid returning stale data during an in-flight write.
  • Output truncation is bounded even when a tool result contains an unusually long path.

Design notes

The goal is not to make every tool artificially restrictive. The limits are applied at the point where input is read or streamed, before that work can become an avoidable memory, latency, or token cost.

For web tools, the request deadline is enforced over the entire operation rather than reset at each redirect. The regression suite includes a local slow-drip HTTP server to verify that a response which stalls after sending initial bytes is cancelled near the configured deadline.

Validation

  • GitHub Actions CI passes on Python 3.10, 3.11, 3.12, and 3.13.
  • Package smoke tests pass on Python 3.10 and 3.13.
  • Focused regression coverage includes:
    • bounded file reads and CRLF boundaries;
    • oversized and compressed HTTP responses;
    • redirect and slow-drip deadline handling;
    • PDF and summarization limits;
    • tool-profile filtering and cache isolation;
    • prompt assembly and planning guidance;
    • Web API profile configuration.
  • compileall and git diff --check pass.

@huytg2610 huytg2610 closed this Jul 17, 2026
huytg2610 added a commit to huytg2610/cheetahclaws that referenced this pull request Jul 19, 2026
Four defects found by independent review of the bounded-I/O work:

- files.py (SummarizeLargeFile): the PDF branch of _read_file_for_summary
  returned an Error whenever the bounded extractor appended its truncation
  marker, so summarizing any large PDF (papers/books — the tool's main use
  case) hard-failed and dead-ended ReadPDF's own redirect to it. Strip only
  the trailing marker and summarize the extracted text; genuine Error returns
  still propagate.

- fs.py (_read_logical_line): a complete CRLF whose LF landed on the read-chunk
  boundary re-entered the bare-CR probe and swallowed a following blank line,
  shifting every later line number (Read line numbers drive Edit targeting).
  Probe only when the last selected byte is a bare CR.

- files.py (_summarize_large_file reduce): a failed (None) chunk hit break and
  dropped all later chunk summaries; the reduce cap silently truncated the tail
  while the header still claimed all N chunks. Skip None chunks, report the
  chunks actually merged, and warn on incomplete coverage — including when the
  cap clips the last included chunk.

- web.py (_DuckDuckGoResultParser): HTML void elements (<img>, <br>) inside a
  result title/snippet left the depth counters permanently offset, bleeding
  text between fields. Skip void tags in both start/end handlers.

Adds no new deps. Verified with targeted repros and the bounded-I/O,
summarize, tool-registry and prompt-assembly test suites.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant