Ongoing DevOps/Prod. support tasks. [Jan 15 - Jan 28]

[landreev]

An automatically-opened new issue for the miscellaneous systems work for the next 2 week period. (the automated script for creating/deleting these issues via GitHub API and scripts is still somewhat experimental)

[landreev]

Harvesting work for Bertarelli - 2-3 hours of my time this week.

[landreev]

Globus is a gift that keeps on giving. It appears that the processing queue keeps getting stuck on app-2; while not fatal (whatever is sitting in the queue will be processed after the next restart), it makes life difficult for the users. Especially for those uploading data, since their completed transfers are not finalized on the Dataverse end, files are not added to the datasets, etc.
I don't think this would be a result of any newly deployed globus-related patches (but will need to review); so this is likely something new - maybe on the globus end? I've enabled verbose logging on the globus classes, will keep investigating.
Will have to spend at least a couple of hours of my time on it this week.

[landreev]

Rachel from Bertarelli keeps sending more DOIs to harvest/collection mapping changes for their complex harvesting setup. I have the entire process largely automated, with a script that takes a list of DOIS-subcollections and configures/deletes/recreates their harvesting clients as needed (10 clients total at the moment).
But I need to document this somewhere.

[landreev]

There are quite a few other things that need to be documented - for ex., the whitelisting procedure for the spam filter; the AWS document needs to be reorganized/rewritten/split into multiple pieces, and more. ... I'll try to have a "documentation day" and address as much of this as I can.

[landreev]

Need to go through the data accumulated over the years of using the automated spam filter and put together a corpus of identified spam metadata, for the ai anti-spam initiative.

[landreev]

Still need to adjust the firewall rules on demo and qa to make GitHub actions work again.
GitHub publishes the list of IPs/subnets used by the actions, so it is possible to create iptables rules excepting these.
In practice, the list is 4K+ long. Even if I round all their subnet definitions down to X.Y.0.0/16, .../24 and .../32, it looks like there will still be a 1K+ rules to add. I'll try to script/automate this; I don't expect iptables to have a problem with this setup - but will need to be confirmed.