Repository navigation
Commit 24ef18f
committed
feat(io): a segment-keyed IO protocol for planner-driven scans
Replaces the byte-range planning protocol (`vortex_io::request`) with
`vortex_io::segments`, keyed by segment source and source request.
- A request names `IoKey { source: SegmentSourceId, request: SourceRequest }`,
where `SourceRequest ::= Size | Segment(id)`, with an `IoIntent`: `Fetch`,
`Prefetch` or `Announce`. One service serves many segment sources, and reads
resolve to a `SourceResult` (`Size(u64)` or `Bytes(BufferHandle)`).
- `SegmentIoSession::submit` registers a whole batch before any read starts and
returns one `IoHandle` per request. The session holds fetches and prefetches
until they finish; an announcement is withdrawn by dropping its handles.
- `poll_ready` reports every fetch's owner once it finishes and drives the
session's prefetches, through the shared `FetchTracker` helper.
`SegmentIoService::session` opens one session per scope, and
`RegisteredSource` is a source's registration with a service.
- vortex-scan: `Planner` and `Morsel` drop the `IoConsumer` supertrait, return
`NeedsIO(Vec<IoRequest>)`, and receive `deliver(key, result)` once per
successful fetch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4PmceDHg8rFe5bgkhGFU9
Signed-off-by: Claude <noreply@anthropic.com>1 parent bc9e0e6 commit 24ef18f
9 files changed
Lines changed: 404 additions & 578 deletions
File tree
- vortex-io/src
- request
- source
- vortex-scan/src/planning
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
27 | 26 | | |
| 27 | + | |
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
| |||
This file was deleted.
This file was deleted.
0 commit comments