Helm unable to see secrets

[acutillas-gs]

Hello,

I'm using ARC to control my self-hosted runners. My jobs always run in a docker image I have in my private repo. I compile my project fine, I build the docker image and push it to my repo fine, I can pull the image fine.

Everything is working except when I try to use helm to deploy my microservice using my own helm chart (it's a basic server where I run my docker image). The issue is that helm in my runner can't see the secrets in kubernetes.

I have tried a bunch of different things but nothing seems to work and I'm getting frustrated. In the ARC chart I set RBAC to true and gave it way too many permissions:

rbac:
  create: true
  rules:
    - apiGroups: [""]
      resources: ["pods"]
      verbs: ["list", "get", "watch", "create", "delete"]
    - apiGroups: [""]
      resources: ["pods/exec"]
      verbs: ["create"]
    - apiGroups: [""]
      resources: ["pods/log"]
      verbs: ["get"]
    - apiGroups: [""]
      resources: ["pods/attach"]
      verbs: ["list", "get", "create", "delete", "update"]
    - apiGroups: [""]
      resources: ["secrets"]
      verbs: ["list", "get", "create", "delete", "update"]

I also set this to true, but I'm not 100% sure what it does:

allowGrantingKubernetesContainerModePermissions: true

I thought the RBAC + rules would be giving my service account all these permissions so running helm would be able to see secrets (as specified in the rules) but it isn't working. I see that my pod is made of a runner and a "docker". I wonder if my workflow is being ran on the docker part and maybe that doesn't have the right permissions?

The actual error is:

Error: UPGRADE FAILED: query: failed to query with labels: secrets is forbidden: User "system:node:ip****.compute.internal" cannot list resource "secrets" in API group "" in the namespace "common": No Object name found

Can anybody shed some light with any of this? Thank you!

[acutillas-gs]

Hi again, I'm still stuck with this. I'm now trying using containerMode: kubernetes but my image is in a private repository so it can't get it. Is there a way around this or are private repositories just not allowed?

Before changing the containerMode I've tried creating roles with so many permissions and assigning them to my service account but it still can't see the secrets. I guess the process that is running in my (private) docker image, inside the actions-runner docker doesn't have the right permissions to read secrets in the cluster. Is there a way around this?

@mumoshu sorry to be tagging you directly but I'm really frustrated. Any hints will be appreciated, thanks!!

[acutillas-gs]

In case this happens to anyone, I ended up having to give all those permissions to system:nodes. I don't know why the service account is not handling this (expected behaviour, IMO) but at least that did the trick.