CROM-6917 Add optional ref disk auth

[kshakir]

No description provided.

[kshakir]

Cromwell communicates with various gcloud endpoints. Instead of having N creds for N endpoints, sometimes Cromwell reuses creds for something else. For example: using the GCS creds for docker hash lookups here.

This PR adds an optional config for reference disk validation auth falling back to the genomics auth that was used previously.

One can then use USA authentication for individual genomics calls and a separate system SA for verifying which of the reference disks are valid (at startup).

[kshakir]

If the direction is ok, I can add unit/integration tests.

[aednichols]

I think we're looking to freeze GCP changes for now due the imminent migration to GCP Batch. We're also not sure if reference disks are staying around, they are maintenance-intensive in Terra.

[kshakir]

Makes sense. This can wait for the official announcement of which way the feature is going.

In the meantime, our users are gradually migrating from on-prem to Terra. Our Cromwell instance allows users to run workflows on GCP or GridEngine. We want to ensure our instance has feature parity with launching workflows in Terra, so we needed something like this commit.

After the announcement, I'll update the PR to copy these auth config lines over to the Batch backend. Otherwise, Terra will have removed the checkbox for reference disks and we can close the PR.