Inviting other admins to access dashboard

[JGreenlee]

The pool of program administrators who have access to the dashboard is currently controlled by a Cognito user pool, which is propagated based on admin_dashboard.admin_access of the dynamic config

Admins frequently want to add other admins to the pool. To take the burden off of us, we'd like to provide the ability to invite other admins from within the dashboard.

I believe we can do this in Python with boto3

https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_AdminCreateUser.html
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html

Something along the lines of:

import boto3

client = boto3.client('cognito-idp', region_name='us-west-2')

def invite_admin(email):
    return client.admin_create_user(
        UserPoolId=f"nrelopenpath-prod-{os.getenv('STUDY_CONFIG')}",
        Username=email,
        UserAttributes=[
            {'Name': 'email', 'Value': email},
            {'Name': 'email_verified', 'Value': 'true'},
        ],
        DesiredDeliveryMediums=['EMAIL'],
    )

---

However, it is hard to know what needs to happen to configure the client correctly on production.
According to boto3 docs:

Boto3 will look in several locations when searching for credentials. The mechanism in which Boto3 looks for credentials is to search through a list of possible locations and stop as soon as it finds credentials. The order in which Boto3 searches for credentials is:

Passing credentials as parameters in the boto3.client() method
Passing credentials as parameters when creating a Session object
Environment variables
Assume role provider
Assume role with web identity provider
AWS IAM Identity Center credential provider
Shared credential file (~/.aws/credentials)
AWS config file (~/.aws/config)
Boto2 config file (/etc/boto.cfg and ~/.boto)
Container credential provider
Instance metadata service on an Amazon EC2 instance that has an IAM role configured.

What config files and/or environment variables are available in the environment where the dashboard runs? Will they be picked up or do we need to manually handle aws_access_key_id, aws_secret_access_key, and aws_session_token?

[shankari]

However, it is hard to know what needs to happen to configure the client correctly on production.

We currently use environment variables to pass in information to the admin dashboard. The admin dashboard already supports multiple environment variables for cognito, but they are primarily to log in with cognito, not to modify the cognito configuration.

op-admin-dashboard/docker-compose-prod.yml

Line 22 in e223ba9

COGNITO_CLIENT_ID: ''

I think that the best option is to use the approach that the github action in the deploy config uses - it uses an AWS_ACCT_ID and ROLE_NAME to retrieve the keys every time.
https://github.com/e-mission/nrel-openpath-deploy-configs/blob/8bfc75dee6cebca32192f6e175bde72166c60561/.github/workflows/AWS-auth.yml#L10

We can configure the same account and role as environment variables for the admin dashboard (instead of secrets as in the github action), and use it to retrieve the credentials, and then use boto to create the accounts.

Here's the history of the email automation issue
e-mission/e-mission-docs#1008

An alternate approach would be to not add the entry directly, but to commit a change to https://github.com/e-mission/nrel-openpath-deploy-configs that adds the entry (essentially automate the manual step that we currently do. This has the advantages that:

• there is a single source of truth (the github repo) instead of having a potential inconsistency between the repo and reality
• a single set of scripts to add users
• a single place where the AWS role is configured

Thoughts?

[JGreenlee]

An alternate approach would be to not add the entry directly, but to commit a change to https://github.com/e-mission/nrel-openpath-deploy-configs that adds the entry (essentially automate the manual step that we currently do.

If I understand what you're saying, when a new admin is invited from the dashboard, we would programmatically commit to the main branch of nrel-openpath-deploy-configs, adding the new email to the admin_access of that program's config

I see the advantages, but is that safe to do? Triggering Git commits programmatically from a live web app seems like it could carry some risks (or just generally be considered bad practice / unorthodox)

[JGreenlee]

A third option would be to make a new Actions workflow on nrel-openpath-deploy-configs that updates the email list, and have the dashboard trigger that instead of committing directly. The dashboard would still need an access token but not one with repo write access – only enough permissions to trigger a workflow

The new workflow itself would accept program and email, sanitize those inputs and commit the changes to the appropriate config file

This seems better to me because:

• better separation of concerns
  • nrel-openpath-deploy-configs owns single source of truth (admin_access list)
  • it also controls the changes to that list
• no need to make a token with repo write access
  • the actions:write permission allows triggering workflows but not writing to the repo (contents:write) https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/controlling-permissions-for-github_token
• more easily extensible
  • easy to support removing admins as well as adding them
  • we could define a subset of fields, not limited to admin_access, that admins can edit without a manual step from us

[JGreenlee]

I have been experimenting with this on JGreenlee/nrel-openpath-deploy-configs and was able to add an email to the list through a workflow triggered from the dashboard (running locally, provided certain environment variables are set)

https://github.com/JGreenlee/nrel-openpath-deploy-configs/commits/main/

[shankari]

A third option would be to make a new Actions workflow on nrel-openpath-deploy-configs that updates the email list, and have the dashboard trigger that instead of committing directly. The dashboard would still need an access token but not one with repo write access – only enough permissions to trigger a workflow

This sounds fine to me.

For the record, I didn't write this out in detail, but I had envisioned a flow where the dashboard would generate a PR, and PRs from the "dashboard" user would trigger a workflow which would automate our sanity checks and merge if they passed. The existing "on merge" workflow would then send the email.

In addition to all the separation of concerns considerations, this would preserve the ability for us to make manual changes if needed because we wouldn't inadvertently override the users added by the existing admins. PRs from non "dashboard" users would not be automatically merged, and would require approval from one of us.

It would also ensure transparency, so that program participants can see who has access to the data collected about them.

[JGreenlee]

I had envisioned a flow where the dashboard would generate a PR, and PRs from the "dashboard" user would trigger a workflow which would automate our sanity checks and merge if they passed. The existing "on merge" workflow would then send the email.

This is a much nicer solution, and since it’s not far off from what I had been experimenting with, I went ahead and here’s what I have landed on:
https://github.com/JGreenlee/nrel-openpath-deploy-configs/blob/main/.github/workflows/config-update.yml

I kept this generic so we can support other config updates in the future. Currently there’s just update_admin_access.py but we can add more scripts in bin/config_update_scripts and the workflow will invoke them the same way. If there are changes after running the script, it commits and PRs. If that PR passes checks, it will be auto-merged
This can be triggered from the admin dash given that a few environment variables are made available.
We can also trigger it ourselves in the Github UI, which is neat





---

If we are going to auto-merge with confidence, we need a second workflow to validate the configs. So what constitutes “valid”?
Mainly, I think we need to: (i) assert the presence of required fields; (ii) refute the presence of unknown fields (to help us catch typos/ mistakes); (iii) ensure all defined fields are the right type

I could think of 2 options for this:

• Write a custom .py script with a bunch of functions that check each field and make sure it is in order
• Write a JSON schema and validate against that

Neither of these options sound fun to maintain; JSON schema seems more principled but – at least to me – is notoriously ugly; either way we’d have to dig through something messy to add/extend config options

I found https://github.com/YousefED/typescript-json-schema and got the idea to use the existing (partial) TypeScript definitions in e-mission-phone as the source of truth and just generate JSON schema for the purpose of validating configs in the workflow. That’s what I have done here:
https://github.com/JGreenlee/nrel-openpath-deploy-configs/blob/main/.github/workflows/validate-configs.yml

It works really well and helped me find what was missing from the partial type def, as well as a bunch of inconsistencies, such as:

• start_month and start_year are strings, not numbers
• why_we_collect_es exists and is not needed
• raw_data_use only exists on denver-casr
• several configs had “name” instead of “url_abbreviation” (some have neither)
• a few configs had “es” “deployment_partner_name” but nothing else in “es”, which would be invalid
• dfc_fermata had unsupported metrics options for planned features we postponed/canceled

[JGreenlee]

• 🦾 Automated workflows for config updates and config validation nrel-openpath-deploy-configs#154
• ✨ add "Manage Admin Access" features to "Settings" page #168