Skip to content

Conversation

@rohanshah
Copy link
Collaborator

@rohanshah rohanshah commented Dec 17, 2025

Description

Low-hanging fruit for performance improvements:

  • Calculate timestamps lazily instead of loading all into memory
  • Reading data more appropriate for HDF5 layout (cross-channel chunks vs columnar single-channel chunks)
  • Channel parallelism for chunked writes
  • Remove any compression from gzip since chunk size is small anyway , keep gzip wrapper for downstream compatibility

@rohanshah rohanshah merged commit 450107b into main Dec 17, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants