Skip to content

PHASE-THOUGHTS.md

Latest commit

 

History

History
20 lines (16 loc) · 2.34 KB

PHASE-THOUGHTS.md

File metadata and controls

20 lines (16 loc) · 2.34 KB
Raw

Phase 4 thoughts and observations

Context: Phases 1–3 are complete with jq-parity semantics and our single-output commit policy. Phase 4 aims to add jq-like streaming (--stream/NDJSON) while keeping the state engine deterministic and ref-aware.

What’s defined and implemented so far

  • Plan calls for a read-only default stream mode via CLI flag, emitting jq-style [path, leaf] and [path] end markers, depth-first. If writes are ever enabled, they must honor the single-output commit rule and atomic multi-file writes.
  • Runtime already has tostream(val) producing jq-like events, fromstream reconstruction, and state_chunks(array, size) with validation. CLI stream mode (--stream [--stream-input <path>|-] --filter ...) now feeds tostream-compatible events (JSON/NDJSON or current state) into filters and blocks mutations.
  • Docs updated for streaming flags/helpers; tests cover tostream/fromstream, chunking, event emission, and the mutation guard.

Key considerations

  • Default read-only stream mode: We enforce a hard error on mutation ops when stream mode is enabled. If a write gate is ever added, keep it explicit and deterministic.
  • Commit policy: Even if we allow writes in stream mode, keep single-output + atomic across shards. Do not add global “commit strategies”; at most, an opt-in jq-compat first-value for |= only, scoped via CLI.
  • API surface: Reuse jq names only when semantics match (e.g., tostream/fromstream). For state-specific helpers (chunks, possible slurp/inputs analogs that don’t match jq), prefix (state_*) to avoid collisions.
  • Stream source: Stream mode reads JSON or NDJSON from --stream-input (or defaults to the expanded state) and feeds tostream events into eval_stream.
  • Tests: Differential tests vs jq’s tostream/fromstream for ordering and shape remain useful for future coverage; current suite covers event emission, chunk errors, and the mutation guard.
  • Docs: Clearly separate language semantics (jq) from execution policy (state apply mode, commit rule, stream mode read-only). Document event shape/order, flags, helper names, and limitations.

Open decisions

  • Whether to add inputs/slurp analogs: only if we can match jq’s semantics; otherwise prefix and mark as extensions.
  • Whether to expose a jq-compat |= first-value mode via CLI flag (without changing defaults).