Skip to content

Add async single-flight coalescing (dogpile prevention)#25

Merged
toloco merged 1 commit into
masterfrom
async-single-flight
Apr 8, 2026
Merged

Add async single-flight coalescing (dogpile prevention)#25
toloco merged 1 commit into
masterfrom
async-single-flight

Conversation

@toloco
Copy link
Copy Markdown
Owner

@toloco toloco commented Apr 8, 2026

Summary

  • When multiple async coroutines miss the cache for the same key concurrently, only one computes the result — the rest await an asyncio.Event and reuse it
  • Works for both memory and shared backends, default on for all async cached functions
  • Handles error recovery (waiters retry via leader re-election loop) and cancellation cleanly

Closes #16 (single-flight portion; non-blocking waits deferred as low-priority).

Test plan

  • test_async_single_flight — 10 concurrent coroutines, assert exactly 1 computation
  • test_async_single_flight_different_keys — no false coalescing across keys
  • test_async_single_flight_error_recovery — leader fails, waiters recover
  • test_async_single_flight_cancellation — leader cancelled, waiters recover
  • test_async_single_flight_shared — works with shared backend
  • test_async_concurrent assertion tightened from >= 3 to == 3
  • Full test suite (77 tests) passes
  • Benchmarks show no regression

🤖 Generated with Claude Code

When multiple async coroutines miss the cache for the same key concurrently,
only one computes the result — the rest await an asyncio.Event and reuse it.
Handles error recovery (waiters retry) and cancellation cleanly.

Closes #16 (single-flight portion; non-blocking waits deferred).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@toloco toloco merged commit 17969f1 into master Apr 8, 2026
15 checks passed
@toloco toloco deleted the async-single-flight branch April 8, 2026 11:07
@toloco toloco mentioned this pull request Apr 13, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Async shared backend support improvements

1 participant