Bug: Oversized image in tool result causes infinite error loop

[jccrump]

Bug: Oversized image in tool result causes infinite error loop

Summary

When the read tool returns an image that exceeds Anthropic's 5MB base64 limit, the API returns a 400 invalid_request_error. The oversized image stays in message history, so every subsequent API call fails with the same error — creating an infinite loop that locks up the chat.

Error

Error: 400
{"type":"error","error":{"type":"invalid_request_error","message":"messages.31.content.0.tool_result.content.1.image.source.base64: image exceeds 5 MB maximum: 5529508 bytes > 5242880 bytes"},"request_id":"req_011CYwW5ktEeTQ8L5jKqt4G4"}

Root Cause

Three gaps in the image size handling:

1. image-resize.js — Fallback paths return oversized originals

When Photon fails to load (loadPhoton() returns null) or image processing throws in the catch block, resizeImage() returns the original unchanged image — which may be >5MB:

// Line ~33: Photon not available
if (!photon) {
    return { data: img.data, ... wasResized: false }; // ← original, could be >5MB
}

// Line ~152: catch block
catch {
    return { data: img.data, ... wasResized: false }; // ← original, could be >5MB
}

The "last resort" return (line ~141) also doesn't guarantee the result is under the limit.

2. read.js — No post-resize size validation

The read tool calls resizeImage() and blindly sends whatever comes back — no final check that the result is actually under 5MB:

const resized = await resizeImage({ type: "image", data: base64, mimeType });
// No size check here
content = [
    { type: "text", text: textNote },
    { type: "image", data: resized.data, mimeType: resized.mimeType }, // ← could be >5MB
];

3. agent-session.js — No recovery from image-too-large 400 errors

_isRetryableError() only matches 429/500/502/503/504/overloaded/rate-limit errors. The 400 invalid_request_error for oversized images is NOT matched, so:

• The error gets stored as an assistant message with stopReason: "error"
• The oversized image tool result stays in message history
• Every subsequent agent.continue() or new user message sends the same oversized image → same 400 → infinite loop

Proposed Fix

Layer 1: image-resize.js

• When Photon is unavailable or processing fails AND the original exceeds maxBytes, return null instead of the original
• Add a comment that callers must handle null return

Layer 2: read.js

• Handle null return from resizeImage() — return a text-only error explaining the image is too large
• Add a final safety check: if the resized result is still >5MB, return text error instead

Layer 3: agent-session.js

• In the agent_end handler, detect image exceeds 5 MB errors before checking retryable errors
• Strip oversized image blocks from message history (replace with text explanations)
• Remove the error assistant message so the agent can continue cleanly

Environment

• pi-coding-agent (npm, macOS ARM64)
• Anthropic Claude API (5MB per-image base64 limit)
• Photon loads successfully — issue occurs in edge cases where resize still exceeds limit, or in the fallback/catch paths

[Br1an67]

I'd like to take a stab at this one.

I traced the code and the issue description nails it. Three gaps:

1. image-resize.ts (lines ~58, ~206): both the "Photon unavailable" and catch fallbacks return the original image blindly — no size check
2. read.ts (line ~109): uses resized.data directly, no post-resize validation. Same in file-processor.ts (line ~59)
3. agent-session.ts _isRetryableError() (line ~2311): the regex only matches 429/5xx/overloaded — a 400 image exceeds 5 MB falls through, gets stored in history, and every subsequent call replays it → infinite loop

My plan:

• Layer 1: resizeImage() returns null when fallback would exceed maxBytes
• Layer 2: read.ts and file-processor.ts handle null → return text error instead of image
• Layer 3: agent-session.ts detects image-too-large 400s, strips the offending image blocks from history (replace with text), removes the error message so the agent can continue

Will add tests for each layer. Happy to discuss before I start coding.

[deltoid9818]

I'd like to fix this. Three-layer approach matching the root cause analysis in the issue:

Layer 1 — image-resize.ts: Changed return type to Promise<ResizedImage | null>. Three paths now return null instead of the oversized original: (a) Photon unavailable + input exceeds maxBytes, (b) all quality/scale steps exhausted but still over maxBytes, (c) processing throws + input exceeds maxBytes. Images under maxBytes are returned unchanged.

Layer 2 — read.ts and file-processor.ts: Both handle null from resizeImage() by returning a text-only error/warning instead of an image block. Safety net: also checks if the base64 string exceeds 5MB even when resizeImage returns non-null.

Layer 3 — agent-session.ts: Detects image-too-large 400 errors via pattern match on the error message. Walks message history, replaces oversized image blocks with text placeholders, removes the error assistant message, and allows the agent to continue. Breaks the infinite loop.

16 tests across 3 test files (image-resize, read-tool-image, image-processing). npm run check clean.

[badlogic]

Implemented a minimal fix focused on auto-resized images.

Changes:

• image-resize.ts now returns null when Photon cannot load, processing fails, or no resized candidate can get under the configured limit
• size checks now use the base64 payload size, not raw binary size
• read.ts and file-processor.ts now treat null as text-only fallback and never fall back to the original image in auto-resize mode

Validation:

• added caller coverage for read and @file
• updated image resize tests
• npm run check passes

[BloodyTalonsHuisman]

Very cool. Good fix.