Repository navigation
fix(storage): only remove keeper entry when its own PieceRef drops - #16
Closed
detail-app[bot] wants to merge 1 commit into
Closed
detail-app[bot] wants to merge 1 commit into
detail-app[bot] wants to merge 1 commit into
Conversation
Author
|
Closing this pull request because it has had no human activity in 30 days (since 2026-09-11) and was opened 30 days ago. Detail closes its pull requests once they are more than 30 days old and have had no human activity for 14 days. If this change is still useful, reopen the pull request and Detail won't close it again. To keep a Detail pull request open, add the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Detail bug report: View on Detail
Bug
The disk-cache
Keeperholds in-flightPieces soStore::loadcan serve the newest not-yet-flushed value from memory instead of a stale value from disk.PieceRef::dropremoved the keeper entry by key only. SinceKeeper::insertoverwrites an existing same-key entry in place, dropping an older (superseded)PieceRefwhile a newer one for the same key was still alive deleted the newer entry.Store::loadthen fell through to disk and returned the already-flushed older value — a stale read that broke read-your-writes for the window between the two flushes. Reachable in production whenever a key is re-enqueued before its prior flush completes (e.g. memory churn /evict_all).Fix
Piece.recordis private tofoyer-memory, soPiece::ptr_eq(&self, other: &Self) -> boolwas added tofoyer-memory(compares the record pointer). Clones of oneinsertshare the pointer; distinctinserts differ.PieceRef::dropnow removes the shard entry only when the in-table piece is the same record thisPieceRefinserted (self.piece.ptr_eq(o.get())), so a superseded ref's drop is a no-op and the current entry survives.Piece::cloneshares the record pointer; eachmemory::insertallocates a freshArc<Record>— so a superseded insert's pointer never matches the newer in-table piece).Testing
Versioned unit tests:
Piece::ptr_eqcontract test (foyer-memory): clones of one insert compare equal; distinct inserts compare unequal.foyer-storage): two same-key pieces inserted, then the superseded ref is dropped — the keeper must keep returning the newer value; dropping the current ref then removes the entry. Fails without the fix with "keeper lost entry after dropping superseded piece ref".End-to-end (verified locally, not committed): a
Store-level test forced two same-key pieces into separate flush batches — batch 1 (v1) flushes and drops itsPieceRefwhile batch 2 (v2) is held in flight, thenStore::loadis issued mid-window. With the fix it served the in-flightv2from the keeper instead of the stale on-diskv1; after releasing the batch the final load returnedv2from disk. Deterministic across 20 runs; reverting only the keeper guard reproduced the stale on-disk read. That test additionally requires a one-line fix toPsyncIoEngineConfig::build, which currently discards the configured write/read IO latency (hardcodesNone) — a separate test-infra defect, so neither that plumbing fix nor the store test is included here; the committed keeper unit tests fully cover the invariant.Routine checks:
cargo check(foyer-memory, foyer-storage, foyer),cargo fmt --check, andcargo clippyare clean (only a pre-existing workspace-level unknown-lint notice remains). Fullnextestsuites pass withstrict_assertions— foyer-memory 32/32, foyer-storage 26/26 — the foyer hybrid end-to-end fuzzy test passes, and doctests (including the nightly-featured build) pass. No checks were left unverified.Automatic Fixes PRs can be configured here.