-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(_streaming_download
): handle retry-level JSON error responses
#2329
base: main
Are you sure you want to change the base?
Conversation
01201a7
to
3b27867
Compare
if contents[0:1] == b"{": | ||
logger.error(f"Retry {retry}, received JSON error response.") | ||
return self._handle_json_response(contents) | ||
# ...and then at the retry level: | ||
partial = contents[bytes_written_prev:bytes_written] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should/can we skip this check if bytes_written_prev == 0
(a no-retry request)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wondered about this. In terms of control flow:
- In a retry request:
bytes_written_prev < len(contents)
andbytes_written = len(contents)
, solen(partial) < len(contents)
, sopartial[0:1] != contents[0:1]
and is worth checking,- unless we've already returned on line 220.
- In a non-retry request:
bytes_written_prev = 0
andbytes_written = len(contents)
, solen(partial) == len(contents)
, sopartial[0:1] == contents[0:1]
and is indeed redundant,- unless we've already returned on line 220.
So the "should" here for me comes down to: Is this check too expensive to do twice for a successful request? Probably not. But it is confusing, so let me make it explicit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in c5d0b87. I've kept the conditionals parallel between contents
and partial
, but let me know if you'd prefer a nested one for partial
.
3b27867
to
7448448
Compare
…uest halfway through actual file-size Now that we can respond correctly to retry-level errors, this change should allow running "make regenerate-sdk-cassettes" on any valid "make dev" instance, not only one with submissions above a hard-coded file-size.
7448448
to
d233eb3
Compare
Status
Ready for review.
Description
Fixes #2232 by checking for JSON responses the proxy may have returned midstream in a
Range
retry, not only at the very beginning of the expected response.#2330 is a limitation of the test suite I've discovered in the course of this investigation. It's out of the scope of this fix.
Test Plan
make regenerate-sdk-cassettes
succeeds against unparameterizedmake dev
..py
files for easier review!Checklist
If these changes modify code paths involving cryptography, the opening of files in VMs or network (via the RPC service) traffic, Qubes testing in the staging environment is required. For fine tuning of the graphical user interface, testing in any environment in Qubes is required. Please check as applicable:
If these changes add or remove files other than client code, the AppArmor profile may need to be updated. Please check as applicable:
If these changes modify the database schema, you should include a database migration. Please check as applicable:
main
and confirmed that the migration is self-contained and applies cleanlymain
and would like the reviewer to do so