Skip to content

Retry waits as long as the provider's Retry-After asks - #1703

Merged
Muizzkolapo merged 1 commit into
mainfrom
retry-waits-as-the-provider-asks
Nov 8, 2026
Merged

Muizzkolapo merged 1 commit into
mainfrom
retry-waits-as-the-provider-asks

Conversation

@Muizzkolapo

@Muizzkolapo Muizzkolapo commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • When a provider rate-limited a request and said how long to wait, agac parsed Retry-After into the error and then waited a random backoff of up to 1, 2, 4 seconds anyway.
  • RetryService now waits the Retry-After the error carries, capped at max_delay with a warning that names both numbers. Without one, it keeps the jittered backoff.
  • The header is read as retry-after-ms, then as seconds or as an HTTP date. A past date is 0; a value that is not a number is unreadable and gets the backoff.
  • RetryConfig declares base_delay (above 0, at most 300, default 1.0) and max_delay (above 0, at most 3600, default 60.0, at least base_delay). Before, the code read them but the schema refused them.
  • Batch retry (BatchRetryService) is unchanged and reads neither delay.

Verification

Gate Result
RED (tests/unit/processing/test_retry_waits_as_the_provider_asks.py on main) 6 assertion failures
Review (risk HIGH, 3 blind lenses) Lens A and B: NO, a Retry-After: nan made time.sleep(nan) raise. Lens C: YES, but the cap test used the default 60, so a hard-coded 60 passed
Fix and re-review NaN is unreadable in both places; cap test uses max_delay=5. Lens A re-review: YES
Full suite, ruff, mypy 17261 passed, 37 skipped (before and after the commit)
Smoke ring Skipped: the smoke ring was scoped to the bottom layer of the first stack. The ring runs on agac-provider, which never rate-limits, so it cannot reach this path
  • Not changed here, recorded on the coordination board: RateLimitEvent still shows the provider's uncapped value ("retrying in 500.0s" when the wait is 60 s); the OpenAI, Anthropic and Groq SDKs keep their own two retries.

@Muizzkolapo
Muizzkolapo added this pull request to stack #1705 November 8, 2026 11:41
@Muizzkolapo
Muizzkolapo marked this pull request as ready for review November 8, 2026 12:40
When a provider rate-limited a request and said how long to wait,
agac parsed the header into the error and then waited a random
backoff of up to 1, 2, 4 seconds anyway. The backoff's base_delay
and max_delay were read from the retry block, but the schema
refused those keys, so they were always 1 and 60 seconds.

RetryService now waits the Retry-After the error carries, capped at
max_delay with a warning that names both numbers; without one it
keeps the jittered backoff. The header is read as retry-after-ms,
then as seconds or an HTTP date; a past date is 0 and a value that
is not a number is unreadable. RetryConfig declares base_delay
(above 0, at most 300) and max_delay (above 0, at most 3600, at
least base_delay). Batch retry is unchanged and reads neither.
@Muizzkolapo
Muizzkolapo force-pushed the retry-waits-as-the-provider-asks branch from 7ea902f to d42a10d Compare November 8, 2026 16:01
@Muizzkolapo
Muizzkolapo merged commit dcae1f9 into main Nov 8, 2026
5 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 8, 2026
@Muizzkolapo
Muizzkolapo deleted the retry-waits-as-the-provider-asks branch November 8, 2026 17:08
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant