Skip to content

Add opt-in NDJSON streaming for bulk prediction - #1026

Open
pavsoss wants to merge 1 commit into
Userunknown84:mainfrom
pavsoss:feat/1021-bulk-predict-streaming
Open

Add opt-in NDJSON streaming for bulk prediction#1026
pavsoss wants to merge 1 commit into
Userunknown84:mainfrom
pavsoss:feat/1021-bulk-predict-streaming

Conversation

@pavsoss

@pavsoss pavsoss commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Part 2 of 2 for #1021. Adds an opt-in streaming mode to bulk prediction so
large uploads can be consumed incrementally instead of waiting for the entire
result set to be buffered in memory.

Depends on Part 1 (feat/1021-bulk-predict-robust). This branch is stacked
on top of it; once Part 1 merges into the default branch, this diff shrinks
to just the streaming additions.

Behaviour

  • Clients opt in with ?stream=ndjson or Accept: application/x-ndjson and
    receive newline-delimited JSON: an opening meta line (model version and row
    count), one result line per scored row, any skipped lines, and a closing
    summary.
  • The same per-row isolation and skip reasons apply, so the stream never aborts
    on a single bad row.
  • File-type, size, and total-row-cap checks run before the stream opens, so
    those failures still return the normal typed error rather than a half-written
    stream.
  • The default buffered JSON response is the fallback and is unchanged; a request
    without the opt-in behaves exactly as before.

Parts

Test plan

Coverage for the streaming shape (meta/result/skipped/summary framing and the
model-version header) via both the query param and the Accept header, that the
row cap is enforced before streaming begins, and that the non-streaming JSON
path remains the default.

Closes #1021

@vercel

vercel Bot commented Jul 29, 2026

Copy link
Copy Markdown

@pavsoss is attempting to deploy a commit to the Aditya Sharma's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Resilient bulk prediction — per-row error isolation, input caps, and reload-consistent version stamping

1 participant