Skip to content

Commit 24ef18f

Browse files
committed
feat(io): a segment-keyed IO protocol for planner-driven scans
Replaces the byte-range planning protocol (`vortex_io::request`) with `vortex_io::segments`, keyed by segment source and source request. - A request names `IoKey { source: SegmentSourceId, request: SourceRequest }`, where `SourceRequest ::= Size | Segment(id)`, with an `IoIntent`: `Fetch`, `Prefetch` or `Announce`. One service serves many segment sources, and reads resolve to a `SourceResult` (`Size(u64)` or `Bytes(BufferHandle)`). - `SegmentIoSession::submit` registers a whole batch before any read starts and returns one `IoHandle` per request. The session holds fetches and prefetches until they finish; an announcement is withdrawn by dropping its handles. - `poll_ready` reports every fetch's owner once it finishes and drives the session's prefetches, through the shared `FetchTracker` helper. `SegmentIoService::session` opens one session per scope, and `RegisteredSource` is a source's registration with a service. - vortex-scan: `Planner` and `Morsel` drop the `IoConsumer` supertrait, return `NeedsIO(Vec<IoRequest>)`, and receive `deliver(key, result)` once per successful fetch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N4PmceDHg8rFe5bgkhGFU9 Signed-off-by: Claude <noreply@anthropic.com>
1 parent bc9e0e6 commit 24ef18f

9 files changed

Lines changed: 404 additions & 578 deletions

File tree

‎vortex-io/src/lib.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@ mod limit;
2323
#[cfg(feature = "object_store")]
2424
pub mod object_store;
2525
mod read_at;
26-
pub mod request;
2726
pub mod runtime;
27+
pub mod segments;
2828
pub mod session;
2929
#[cfg(not(target_arch = "wasm32"))]
3030
pub mod std_file;

‎vortex-io/src/request/mod.rs‎

Lines changed: 0 additions & 223 deletions
This file was deleted.

‎vortex-io/src/request/source.rs‎

Lines changed: 0 additions & 108 deletions
This file was deleted.

0 commit comments

Comments
 (0)