Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Moving a shard currently changes ownership derived from the host list. Explicit placement groups let one group move while preserving every other name's destination. During a move, live writes must reach both the donor and destination while immutable history is copied. Reapplying a point is safe under the storage contract that
(name, slot)writes are idempotent and rollups are recomputed from slots.Architecture
An opt-in
placementcluster loads the canonical virtual-shard map and binds its store incarnations to the existing configured host endpoints. Frozen bootstrap selectors preserve existing jump owners. The scalar and batch paths select the replica'swriterand optionalmoving_tofrom one immutable map snapshot, then use ordinary bounded host queues and existing transports.SIGHUP validates a complete successor and publishes it with one atomic pointer swap per cluster. Validation checks the predecessor digest, revision, frozen identity, monotone write epochs and fixed transport inventory. A writer change must promote the prior
moving_toand clear it. Failed reloads retain the active map and increment a rejection counter. A process restart reads the configured current map at any revision.Relay output carries ordinary points through the existing queues; placement epochs remain control metadata. Store-side snapshot pinning, checksum copies, copied-part precedence, read switching, relay drain and final incremental copying belong to the move controller. A final copy can fill missing slots but cannot repair a dropped same-slot overwrite when B already has an older live-written value, because copied parts must remain older than B's live parts. Already staged output can finish under the preceding map, so adoption alone does not certify drain.
moving_tois an optional map extension. Omitting it preserves existing canonical bytes and golden digests. Older strict decoders reject documents using it; store/controller schema support must be available before moving maps are enabled. The copied contract's historical receipt helpers remain unused by relay routing.Before / after
Usage
Configure
type = "placement",placement_map,placement_replica, and explicit hosts as shown indocs/placement-routing.md. The shared bootstrap example is exercised by scalar/batch legacy equality tests.For one replica, publish a successor containing this fragment:
{"writer":"a","moving_to":"d","write_epoch":1}After snapshot/incremental copying and the read switch, publish its successor:
{"writer":"d","write_epoch":2}Wait for all relays to adopt and drain, perform the final incremental copy, then retain the donor until collection is safe. A group carrying byte share
fadds one extra logical copy of its records, increasing logical egress fromBtoB*(1+f); actual compressed NIC traffic also depends on framing and dictionaries.Validation
Warmed routing/dispatch on a shared Apple M4 Max laptop, GOMAXPROCS=1, five 300ms repetitions with counting targets:
These figures exclude parsing, output queue locks, encoding, sockets and stores; they do not establish NIC throughput.
The final paired legacy comparison uses 31 processes, each with 2,048 alternating windows of 4,096 records per layout: hosts=3/batch=False: -4.96% (95% process-bootstrap -6.94% to -4.05%); hosts=3/batch=True: +0.17% (95% process-bootstrap +0.16% to +0.34%); hosts=40/batch=False: -2.21% (95% process-bootstrap -2.44% to -1.48%); hosts=40/batch=True: +1.58% (95% process-bootstrap +0.90% to +3.45%). A 40-host batch control comparing two copies of the baseline reports +1.58% (95% process-bootstrap +0.09% to +3.15%). This demonstrates sensitivity to the benchmark/code layout on the shared laptop. The requested zero-regression qualification remains open; neither source equality nor this control is treated as proof of zero CPU cost.
Rollout and rollback
Keep this PR Draft until the matching store/controller move implementation and schema support are ready. Provision all map endpoints before a move; map reloads cannot edit transport inventory. Preserve the current map through restarts. Reverse an ownership move through successive maps and epochs with store reconciliation rather than restoring an old file.