Skip to content

feat(cli): stream model uploads with progress - #429

Merged
dylanuys merged 1 commit into
mainfrom
feat/stream-model-uploads
Aug 24, 2026
Merged

dylanuys merged 1 commit into
mainfrom
feat/stream-model-uploads

Conversation

@dylanuys

Copy link
Copy Markdown
Contributor

Summary

  • hash discriminator model archives incrementally instead of reading the entire file into memory
  • stream model archives from disk during the presigned R2 upload
  • show upload percentage, transferred size, throughput, and ETA while the PUT is active
  • preserve the existing API flow, explicit content length, and five-minute timeout

Why

Large model uploads previously appeared idle during a single blocking PUT and temporarily consumed memory roughly equal to the archive size. Streaming keeps memory usage bounded and makes slow or stalled network paths visible without relaxing the existing timeout.

Validation

  • pytest tests/test_miner_upload_streaming.py -q — 2 passed
  • verifies incremental hashing
  • verifies the request consumes a file-backed reader, retains Content-Length, reconstructs the exact payload, and reports completion progress

@dylanuys
dylanuys merged commit 0ce11d1 into main Aug 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant