Discussion: Best practice to restrict permissions

[schwichti]

What is best practice to manage the access of Pods to Secrets?
There are different approaches how you can restrict access.
I think that when you do not have a self-obligated policy how you implement access restriction, managing the permission will quickly become uncontrollable.
In my setting I have ~70 pods and ~10 secrets.
In the following, I discuss four approaches.

In AWS Secrets Manager you can store JSON in a Secret.
What I mean with Key in the following is an attribute of a JSON object stored in a Secret.

Approach 1

Every Pod has a Volume for managing its Secrets.
This Volume has a SecretProviderClass.
Every SecretProviderClass references many Secrets.
Every Secret contains exactly one Key.
Every Pod has its own ServiceAccount.
Every ServiceAccount has its own IAM Role.
For every Secret an IAM Role needs access to, it references an IAM Policy.
An IAM Policy allows access to exactly one Secret.

New Secrets can be added to a Pod by doing the following:

1. Add the Secret to the Pod's SecretProviderClass
2. Add an IAM Policy to the IAM Role that the Pod's ServiceAccount points to.

Pros

• ✔️ Pods have only access to the Secrets they need (least priviledge)
• ✔️ Allows to restrict Secret access on AWS level
• ✔️ Allows to use different types of Secrets (i.e. Credentials for Amazon RDS database, Credentials for Amazon DocumentDB database, Credentials for Amazon Redshift cluster, or Credentials for other database, or Other)
• ✔️ Allows rotation logic per secret

Cons

• ❌ SecretProviderClass specification may become cluttered with a lot of Secrets

Approach 2

Description

Every Pod has a Volume per Secret it needs access to.
Every SecretProviderClass references exactly one Secret.
Every Secret contains exactly one key.
Every Pod has its own ServiceAccount.
Every ServiceAccount has its own IAM Role.
For every secret an IAM Role needs access to, it references the respective IAM Policy.
An IAM Policy allows access to exactly one Secret.

Usage

New Secrets can be added to a Pod by doing the following:

1. Add a new Volume to the Pod pointing to the SecretProviderClass that points to the Secret
2. Add a IAM Policy to the IAM Role that the Pod's ServiceAccount points to.

Pros

• ✔️ Pods have only access to the Secrets they need (least priviledge)
• ✔️ Allows to restrict Secret access on AWS level
• ✔️ Allows to use different types of Secrets
• ✔️ Allows rotation logic per secret

Cons

• ❌ Pod specification may become cluttered with a lot of Volumes

Approach 3

Every Pod has one Volume that references the only SecretProviderClass.
The SecretProviderClass references exactly the only Secret.
There is one Secret that contains all Keys.
All Pods use the same ServiceAccount.
There is only one ServiceAccount, one IAM Role, one IAM Policy.
This IAM Role references the IAM Policy.
This IAM Policy allows access to exactly one Secret.

New Secrets can be added to a Pod by doing the following:

1. Add a new Key to the Secret

Pros

• ✔️ Easy to manage

Cons

• ❌ Insecure: Pods have access to all Secrets
• ❌ Does not allow to restrict Secret access on AWS level
• ❌ Does not allow to use different types of Secrets
• ❌ Does not allow rotation logic per secret
• ❌ Cannot use different Secret types

Approach 4

Allow all combinations of approaches 1-3 at once.

Pros

• ✔️ Flexible

Cons

• ❌ If all approaches are mixed, it is hard to trace permissions. Error-prone.

My personal preference currently is Approach 1.
I would like to hear experiences from others.
Are there even better approaches?
Are there any pros/cons that I have missed?