Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
hi!
currently the package can only open a stream from the opened position -> end of file
i store indexed blobs on s3 and perform small random reads on them (static database), so for me, it is useful to open a stream and read only say, 1kb out of a 30gb file
i made this change which makes Read() only request up to the size of the byte buffer passed in. I also added tests related to this, and the current tests pass (at least on my computer)
this does mean that current applications which use readAll on a large file, the request will be broken up into many smaller requests. Whether or not this is preferable, it is not for me to say. The previous behavior can be replicated through passing in a byte buffer with a length larger than or equal to the size of the file, something which is known to the user before they have a chance to call read.
i'm currently using a fork of this library for this purpose, would love to be able to switch back. let me know if there is anything else i would need to do to get this in!