Skip to content

Commit 1035b79

Browse files
committed
fix: record cooldown for last model in cascade before throwing
Previously the isLast check threw before setModelCooldown ran, so the last model in GEMINI_FALLBACK_MODELS never got its 429 recorded. A resume/retry would skip the earlier (cooling-down) models but go straight back to the last one and re-hit the same exhausted quota immediately instead of backing off.
1 parent 878ec14 commit 1035b79

1 file changed

Lines changed: 11 additions & 4 deletions

File tree

connectors/gemini/client.js

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -93,15 +93,22 @@ async function callGenerateContent(body, requestedModel) {
9393
const isRateLimited = err.status === 429;
9494
const isOverloaded = err.status === 503;
9595
const isNetworkTransient = err.transient === true; // timeout/dropped connection, see callGenerateContentOnce
96-
if ((!isRateLimited && !isOverloaded && !isNetworkTransient) || isLast) throw err;
96+
if (!isRateLimited && !isOverloaded && !isNetworkTransient) throw err;
9797
if (isRateLimited) {
9898
// Rate-limited on this model -- record a cooldown (best-effort; never
99-
// blocks or throws on its own) so future calls can skip straight past
100-
// it. No equivalent recording for 503: there's no per-model quota
101-
// hint to parse, and an overload isn't reliably tied to this model
99+
// blocks or throws on its own) so future calls -- including a
100+
// resumed/retried one -- can skip straight past it. Recorded even
101+
// when this is the LAST model in the chain (isLast below): a 429 on
102+
// the last model still means it's exhausted for the window, and
103+
// skipping the setModelCooldown call in that case (as this used to)
104+
// meant a resume would skip the earlier cooling-down models but walk
105+
// straight back into this same exhausted one and fail identically.
106+
// No equivalent recording for 503: there's no per-model quota hint to
107+
// parse, and an overload isn't reliably tied to this model
102108
// specifically the way a 429 is.
103109
await setModelCooldown(model, parseRetryDelaySeconds(err.message));
104110
}
111+
if (isLast) throw err;
105112
// Fall through to try the next model either way.
106113
}
107114
}

0 commit comments

Comments
 (0)