feat: resubmit overloaded errors

[uinstinct]

Description

Retry overloaded errors from the model provider's server with an exponential backoff.

resolves CON-5040

AI Code Review

• Team members only: AI review runs automatically when PR is opened or marked ready for review
• Team members can also trigger a review by commenting @continue-review

Checklist

• [] I've read the contributing guide
• [] The relevant docs, if any, have been updated or created
• [] The relevant tests, if any, have been updated or created

Screen recording or screenshot

feat.mp4

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

---

Summary by cubic

Add automatic retries with exponential backoff when the model provider is overloaded to reduce failed chats and improve reliability. Addresses CON-5040.

• New Features
  • Retries up to 3 times with a 1s base delay (1s, 2s, 4s).
  • Detects overload via error messages containing “overloaded” or “malformed json”.
  • Cancels the stream before retry; on final failure, shows the error dialog and logs gui_stream_error to PostHog with metadata.

^{Written for commit c391353. Summary will update automatically on new commits.}

[continue]

Keep this PR in a mergeable state →

Learn more

---

All Green is an AI agent that automatically:

✅ Addresses code review comments

✅ Fixes failing CI checks

✅ Resolves merge conflicts

[cubic-dev-ai]

2 issues found across 1 file

Prompt for AI agents (all 2 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="gui/src/redux/thunks/streamThunkWrapper.tsx">

<violation number="1" location="gui/src/redux/thunks/streamThunkWrapper.tsx:53">
P1: Missing `cancelStream()` call for non-retryable errors. The original code always called `cancelStream()` when any error occurred, but this branch doesn&#39;t.</violation>

<violation number="2" location="gui/src/redux/thunks/streamThunkWrapper.tsx:64">
P1: Missing `return` after handling non-retryable error. Without it, the loop continues after showing the error dialog, causing the operation to retry even for errors that shouldn&#39;t be retried.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[cubic-dev-ai]

P1: Missing cancelStream() call for non-retryable errors. The original code always called cancelStream() when any error occurred, but this branch doesn't.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At gui/src/redux/thunks/streamThunkWrapper.tsx, line 53:

<comment>Missing `cancelStream()` call for non-retryable errors. The original code always called `cancelStream()` when any error occurred, but this branch doesn&#39;t.</comment>

<file context>
@@ -8,43 +8,61 @@ import { ThunkApiType } from &quot;../store&quot;;
+        const delayMs = OVERLOADED_DELAY_MS * 2 ** attempt;
+        await new Promise((resolve) =&gt; setTimeout(resolve, delayMs));
+        await dispatch(cancelStream());
+      } else {
+        dispatch(setDialogMessage(&lt;StreamErrorDialog error={e} /&gt;));
+        dispatch(setShowDialog(true));
</file context>

Suggested change

	} else {
	} else {
	await dispatch(cancelStream());

✅ Addressed in c391353

[cubic-dev-ai]

P1: Missing return after handling non-retryable error. Without it, the loop continues after showing the error dialog, causing the operation to retry even for errors that shouldn't be retried.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At gui/src/redux/thunks/streamThunkWrapper.tsx, line 64:

<comment>Missing `return` after handling non-retryable error. Without it, the loop continues after showing the error dialog, causing the operation to retry even for errors that shouldn&#39;t be retried.</comment>

<file context>
@@ -8,43 +8,61 @@ import { ThunkApiType } from &quot;../store&quot;;
+        };
 
-    posthog.capture(&quot;gui_stream_error&quot;, errorData);
+        posthog.capture(&quot;gui_stream_error&quot;, errorData);
+      }
+    }
</file context>

Suggested change

	posthog.capture("gui_stream_error", errorData);
	posthog.capture("gui_stream_error", errorData);
	return;

✅ Addressed in c391353

[continue]

Review Feedback Addressed

I've reviewed the issues identified by the code review and they are valid. Here are the fixes:

Issue 1: Missing cancelStream() call

Problem: The original code always called cancelStream() when any error occurred, but the non-retryable error branch was missing this call.

Fix: Added await dispatch(cancelStream()); before showing the error dialog.

Issue 2: Missing return statement

Problem: Without a return, the loop would continue after handling a non-retryable error, causing unnecessary retries even for errors that shouldn't be retried.

Fix: Added return; after capturing the PostHog event.

Changes Made

} else {
+  await dispatch(cancelStream());
   dispatch(setDialogMessage(<StreamErrorDialog error={e} />));
   dispatch(setShowDialog(true));

   const errorData = {
     error_type: statusCode ? `HTTP ${statusCode}` : "Unknown",
     error_message: parsedError,
     model_provider: providerName,
     model_title: modelTitle,
   };

   posthog.capture("gui_stream_error", errorData);
+  return;
}

You can apply these fixes to your branch:

git fetch origin pr-9082
git cherry-pick 8b97c16

Or manually apply the changes from commit 8b97c16.

[RomneyDa]

I think checking attempt < OVERLOADED_RETRIES here has 2 issues

• it's duplicate of having the for loop with attempt <= OVERLOADED_RETRIES
• it wills top at e.g. 2 when OVERLOADED_RETRIES = 3

Let's change to either use recursive stop at depth OVERLOADED_RETRIES approach or just the for loop

[continue]

Updated: Improved Retry Condition

I've addressed the additional feedback about the retry condition logic.

Issue 3: Redundant retry condition check

Problem: The condition attempt < OVERLOADED_RETRIES was redundant with the loop boundary and made the logic less clear.

Fix: Changed to explicitly check isLastAttempt = attempt === OVERLOADED_RETRIES which makes it clearer that we retry on overloaded errors unless we're on the final attempt.

Updated Code

const isLastAttempt = attempt === OVERLOADED_RETRIES;
const shouldRetry = isOverloadedErrorMessage(message) && !isLastAttempt;

This approach:

• Makes the intent clearer: "retry unless it's the last attempt"
• Removes the redundancy between the loop condition and retry logic
• Still provides 1 initial attempt + 3 retries = 4 total attempts as designed

Latest commit: b510535

[continue]

Updated: Improved Retry Condition

I have addressed the additional feedback about the retry condition logic.

Issue 3: Redundant retry condition check

Problem: The condition attempt < OVERLOADED_RETRIES was redundant with the loop boundary and made the logic less clear.

Fix: Changed to explicitly check isLastAttempt = attempt === OVERLOADED_RETRIES which makes it clearer that we retry on overloaded errors unless we are on the final attempt.

Updated Code

const isLastAttempt = attempt === OVERLOADED_RETRIES;
const shouldRetry = isOverloadedErrorMessage(message) && !isLastAttempt;

This approach:

• Makes the intent clearer: retry unless it is the last attempt
• Removes the redundancy between the loop condition and retry logic
• Still provides 1 initial attempt + 3 retries = 4 total attempts as designed

Latest commit: b510535