Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error processing #1550

Open
sdarwin opened this issue Dec 12, 2024 · 1 comment
Open

Improve error processing #1550

sdarwin opened this issue Dec 12, 2024 · 1 comment
Assignees
Labels
Bug Something isn't working

Comments

@sdarwin
Copy link
Collaborator

sdarwin commented Dec 12, 2024

  1. In import_release_notes

When importing release notes, the code tries to get many S3 objects that don't exist, causing these errors:

NoSuchKey</Code><Message>The specified key does not exist.

And accompanying long stack traces.

It's known during import_release_notes that the files might not exist. Could the error be handled more gracefully? For example

  • Using a try block, catch the NoSuchKey exceptions and print "File not available in S3, which is expected in this case", instead of dumping errors.

or even better,

  • List the contents of the directory in S3. Determine which files exist beforehand.

Another similar issue, about avoiding an error: when importing releases, the script contacts https://archives.boost.io:443 to import archives. It will search in a directory and find files with names such as https://archives.boost.io/release/1.87.0/source/boost_1_87_0.tar.gz

The "releases" named 'master' and 'develop' should be skipped on that particular step, as it tries to import
archive files from /release/develop/source/ and /release/master/source which are nonexistent directories, causing errors.

Check the name of the release. Don't try to import archives for the specific releases that are named 'master' and 'develop'.

[2024-12-12 11:26:22,473: DEBUG/ForkPoolWorker-2] https://archives.boost.io:443 "GET /release/master/source/ HTTP/1.1" 404 123
[2024-12-12 11:26:22,477: ERROR/ForkPoolWorker-1] get_archives_releases_list_error
(stack trace)
@daveoconnor daveoconnor self-assigned this Dec 12, 2024
@rbbeeston rbbeeston added the Bug Something isn't working label Dec 12, 2024
@rbbeeston rbbeeston moved this to Accepted in website-v2 Dec 12, 2024
@sdarwin
Copy link
Collaborator Author

sdarwin commented Dec 16, 2024

Another issue, that could be part of this or create a separate bug.

When running any django script that requires GITHUB_TOKEN, and that hasn't been set, it may result in raised unexpected: OSError() , and a stack dump, which is difficult to debug.

How about something like this. In the code, any place, everyplace, where GITHUB_TOKEN is used.

  • check that it's not undefined
  • check that it's not literally GITHUB_TOKEN=="top-secret" (from env.template), or GITHUB_TOKEN==""

In any of those cases, print a message "The environment variable GITHUB_TOKEN needs to be set as a GitHub api token. The current value is _, which is not correct. Please set this variable. Exiting." And then gracefully exit with a non-zero status.

Should be a targeted fix. If some scripts don't use GITHUB_TOKEN, they are permitted to run without the check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
Status: Done
Development

No branches or pull requests

3 participants