fix(agent): prune excess images from history to prevent session deadlock#2613
fix(agent): prune excess images from history to prevent session deadlock#2613iceymoss wants to merge 3 commits into
Conversation
When a conversation accumulates more images than a provider's per-request limit (e.g. Gemini's 10-image cap), every subsequent request — including text-only follow-ups — fails with "too many images" because the full message history (with all historical images) is re-sent each time. This makes the session permanently unusable. Root cause: preparePrompt() converts all historical messages including their image attachments into the outgoing request, and no layer between the database and the provider checks or caps the image count. Once the limit is exceeded, the error persists in a loop because the offending images remain in the stored history. Fix: add pruneExcessImages() which runs in PrepareStep right after workaroundProviderMediaLimitations(). It counts image FileParts across the outgoing messages and, when the total exceeds the provider's known limit, replaces the oldest images with a text placeholder. Only the sent history is modified; persisted messages in the database are untouched. The limit is determined by: 1. User-configured `max_images` in crush.json (takes priority), or 2. A built-in provider default (Gemini/VertexAI: 10, others: unlimited) This lets users override the default when a model supports more (or fewer) images than the hard-coded fallback. Fixes charmbracelet#2604
iceymoss
requested review from
aymanbagabas and
meowgorithm
and removed request for
a team
…ialect The CI runs tests with -race. Our DB pipeline and E2E tests call testEnv() which invokes goose.SetDialect(), a write to a package-level global. Running these tests in parallel triggers a data race. Also fix gofumpt formatting: add required blank lines between method declarations on geminiMockModel.
|
A note on the impact of image pruning Only the raw image data of the oldest images is removed from the sent history — the assistant's prior text responses (which describe what was in those images) are fully preserved. Example: Suppose a conversation has 12 images and we need to prune the 2 oldest: Before pruning (12 images — exceeds Gemini's 10-image limit)User: [image-1] Can you check this error? After pruning (10 images — within limit)User: [placeholder] Can you check this error? The model loses the raw pixels of images 1–2, but retains its own earlier analysis of them in the assistant turns. For the vast majority of workflows this is sufficient — users The one edge case where this matters is if the user asks the model to re-inspect a pruned image (e.g. "Go back to screenshot 1 — what color is the button in the top-right Worth noting: nothing is lost on disk — the database retains all original images. Only the copy sent to the provider is trimmed. |
|
Hi @meowgorithm @aymanbagabas @andreynering — gentle ping on this one when you get a chance 🙏 Flagging this mostly because the underlying bug makes sessions permanently Happy to rebase, split into smaller commits, reduce scope, or adjust the |
|
@andreynering Good point, I agree that Here's how I see the transition: Current state (this PR):
Target state (after Catwalk adds the field):
What I'd like to do:
Does this approach work for you? I can update this PR now to remove the Crush-side |
…ve MaxImages config field Remove MaxImages from SelectedModel and the defaultMaxImages provider map. Replace with a simpler maxImagesForModel switch that hard-codes known limits (Gemini/VertexAI: 10) with a TODO referencing the Catwalk issue for per-model MaxAttachments. Also enhance the prune placeholder to include the original filename so the model retains awareness of what was removed
|
@iceymoss Yep, that sounds like a good plan. Thank you! |
|
@andreynering The changes we discussed are now pushed — I've removed the |
|
Hi @andreynering, gentle ping on this PR! 🏓 |
3 participants
Summary
max_imagesfield incrush.jsonto override provider defaultsProblem
When using image-capable models with per-request image limits (like Google Gemini which caps at 10 images), the chat session becomes completely unusable once the conversation
accumulates more images than the limit allows.
Root cause:
preparePrompt()inagent.goconverts all historical messages — including every image ever sent — into the outgoing API request. No layer between the databaseand the provider checks or limits the image count. Once exceeded, the provider returns an error like:
too many images: maximum allowed for model gemini-3.1-pro-preview is 10, got 11
The error is caught and displayed, but because the images are permanently stored in the database, every subsequent request (even text-only follow-ups) re-sends the same
oversized history and hits the same error — creating an unrecoverable loop.
Bug flow:
User sends message → →→ Load history (11 images) →→ → Send to Gemini →→ → "too many images" error
↑ ↓
└──────────── User sends another message ←───────────────────────────────┘
Solution
Add
pruneExcessImages()which runs inPrepareStep(right afterworkaroundProviderMediaLimitations). It:FileParts across the outgoing message history[Earlier image removed to stay within model limits]The image limit is resolved with a two-tier priority:
max_imagesin crush.json)"max_images": 20This means users can override the default if their model supports more images:
{ "models": { "large": { "model": "gemini-2.5-pro", "provider": "gemini", "max_images": 20 } } }Changes
Test plan
Unit tests (12) — core logic
Integration tests (3) — DB → preparePrompt → prune pipeline
E2E test (1) — full sessionAgent.Run() with mock Gemini model
Regression
Fixes #2604