feat: idempotency TTL sweeper with pressure metrics#478
Merged
Conversation
Adds a cooperative IdempotencySweeper that periodically evicts
TTL-expired keys from the in-memory store and exposes storage-
pressure metrics so SREs can tune retention.
- IdempotencyStore gains optional sweepExpired() and count() hooks.
- RedisIdempotencyStore.count() walks the keyspace via SCAN with the
configurable MATCH pattern; sweepExpired() is a no-op since Redis
self-evicts via PEXPIRE.
- inMemoryIdempotencyStore.sweepExpired() prunes expired entries and
emits idempotency_evictions_total{reason="expired"}; set() overflow
pruning now emits reason="overflow"; delete() emits reason="manual".
- New metrics: idempotency_keys_count (gauge, backend),
idempotency_evictions_total (counter, backend+reason),
idempotency_sweep_runs_total (counter, backend+outcome).
- IdempotencySweeper: unref'd interval, single-flight runOnce, error
swallowing with structured logging and metric, awaits in-flight
cycle on stop(). Wired into startServer() and the shutdown
onCleanup hook. Interval is IDEMPOTENCY_SWEEP_INTERVAL_MS
(default 60s, hard floor 1s).
- README documents TTL semantics and the new metrics.
- Adds tests/unit/middleware/idempotency.sweeper.test.ts (35 tests).
|
@Mkalbani Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
Contributor
Author
|
@thlpkee20-wq hello please review and merge!! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a cooperative TTL sweeper for the idempotency key store and exposes
storage-pressure metrics so SREs can tune retention.
The in-memory store previously only pruned during
set()whenMAX_MEMORY_STORE_SIZEwas reached, leaving idle entries to accumulateotherwise. Redis relied on
PEXPIREfor self-eviction but had no gaugeto plot pressure. This change closes both gaps.
Changes
IdempotencyStoreinterface gains optionalsweepExpired()andcount().inMemoryIdempotencyStoreimplements both; overflow prune, manual deleteand TTL sweep now emit
idempotency_evictions_total{reason=…}.RedisIdempotencyStoreadds a SCAN-basedcount()(configurable MATCH);sweepExpired()is a no-op since PEXPIRE handles it.IdempotencySweeper: unref'd interval, single-flightrunOnce(),swallows store errors with structured logging and a metric, awaits
in-flight cycle on
stop().idempotency_keys_count(gauge,backend)idempotency_evictions_total(counter,backend+reason)idempotency_sweep_runs_total(counter,backend+outcome)startIdempotencySweeper()/stopIdempotencySweeper()wired intostartServer()and the shutdownonCleanuphook.IDEMPOTENCY_SWEEP_INTERVAL_MSenv var (default 60s, hard floor 1s).Tests
tests/unit/middleware/idempotency.sweeper.test.ts— 35 new testscovering lifecycle, single-flight, error swallowing, interval floor,
env parsing, backend selection, and in-memory / Redis store
integration.
Security & Correctness Notes
unref()'d: it cannot keep the event loop aliveon its own, so process shutdown is never gated on the sweeper.
runOnce()swallows store errors and emitsidempotency_sweep_runs_total{outcome="error"}; a transient Redisblip cannot wedge the loop. The next cycle resumes once ioredis
reconnects.
stop()awaits the in-flight cycle, so shutdown handlers can relyon a quiescent state.
— the only addition is the eviction metric.
RedisIdempotencyStoreconstructor is backwards compatible (thenew
optionsarg is optional).Closes #448