Auto-generate OTC PGP ids

[lukebakken]

I did some poking around how the openssl.org website is generated, and found my way to this document:

https://github.com/openssl/omc-tools/blob/master/QueryApp/RESTAPI.txt

Turns out with a couple API queries, we can get the OTC team member's PGP key IDs.

@tianon - in the interest of politeness, I'm assuming that we wouldn't want to fetch this data frequently. Do you imagine a manual process, monthly cron job, something like that? I'm assuming it would then end up as an input file to apply-templates.sh?

$ for otc_member_email in $(curl -s https://api.openssl.org/0/Group/otc/Members | jq -r '.[] | .[0]'); do echo $otc_member_email; curl -s "https://api.openssl.org/0/Person/$otc_member_email" | jq '.tags'; done
matt@openssl.org
{
  "country": "UK",
  "rev": "Matt Caswell <matt@openssl.org>",
  "pgp": "8657 ABB2 60F0 56B1 E519 0839 D9C4 D26D 0E60 4491"
}
...
...
...

[tianon]

Oh, really nice find!

We run the automated invocations of versions.sh ~4 times per day, which I think is pretty reasonable for a public / publicly-accessible API like this (especially one that provides data that's likely pretty highly cacheable given how frequently it changes).

We could do something clever there to have it only update the list when the version of OpenSSL changes, but I don't know that it's actually worth the complexity (we typically just don't push effectively no-op automated changes like that downstream until there's a bigger change like a version bump to go with them).

Given that this doesn't give us a list per version or even major.minor, we probably want to fetch the whole list once at the beginning of the script, and then just embed the result in the "openssl": { "version": "..." } object in each loop? Then if we had some simple way later to cross-reference the list with "who signed this specific release" we'd already be all set and it'd be just a matter of filtering the list.

For our sake, it might be worth keeping that whole object so that automated changes to it are easier to review/verify/understand (instead of simplifying it to just a list of IDs), but I suppose we could do something like { "8657 ABB2 60F0 56B1 E519 0839 D9C4 D26D 0E60 4491": "Matt Caswell <matt@openssl.org>", ... } if we wanted to compress it somewhat (I don't have a strong preference either way here), and then just use something like for key in {{ .openssl.keys | keys | unique | map(@sh) | join(" ") }}; do ... inside the template (or more complex if we wanted to embed the signer names as comments, which is also pretty reasonable and would extend the transparency of "what is this key" all the way to the generated Dockerfile which would be kind of neat).

(See https://github.com/docker-library/golang/blob/326acd5eed36954174ba8b3b6d0efda96087e18a/Dockerfile-linux.template#L75-L102 for an example of another template that generates something kind of similar with a set of complex/list-form data if this is something you want to tackle. I'm also totally OK if you want to make me do the integration work here now that you've found all the details of where we can get a solid source of data! 😄)

[lukebakken]

I'm happy to do this, or review it if you beat me to it. Just leave a comment if you start work so we don't duplicate it.

Obviously this isn't high-priority as manually updating the PGP information has worked fine so far.