how to run docker build on this runner without privileged container

[roshvin]

Describe the bug
we are running a docker build in this runner and its failing because no docker engine is installed in the runner pod.
do you have any sample runnerdeployment which will work with docker build workflow.
do you have any sample workflow which we can use to run docker build in this pod with some actions.

[mumoshu]

@roshvin It should work out-of-box as usually it has a sidecar container that runs dind.

[mumoshu]

@roshvin BTW, please don't try to use issues as a place for Q&A! We use Discussions for that.

[roshvin]

@mumoshu Thanks for the update. i am using below runnerdeployment and when i start a docker build inside this runner , i get below error.

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: dot-runner
spec:
  template:
    spec:
      enterprise: daimler
      labels:
      - dot-enterprise-runner
      dockerdWithinRunnerContainer: false
      image: rxxxxxxx/actions-runner
      dockerEnabled: false
      securityContext:
        seLinuxOptions:
          level: "s0"
          role: "system_r"
          type: "super_t"
          user: "system_u"
        runAsUser: 1000
      env:
      - name: RUNNER_FEATURE_FLAG_EPHEMERAL
        value: "true"

[roshvin]

@mumoshu
i have changed the runner deployment as per the doc and when the pod is starting , we are getting below error.

# dindrunnerdeployment.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: example-dindrunnerdeploy
spec:
  replicas: 1
  template:
    spec:
      image: xxxxx/actions-runner-dind
      dockerdWithinRunnerContainer: true
      repository: ROK-ACTION-TEST/docker
      env: []

      securityContext:
        runAsUser: 1000
        seLinuxOptions:
          level: "s0"
          role: "system_r"
          type: "super_t"
          user: "system_u"

[root@devops-runner ent]# k logs -f example-dindrunnerdeploy-jcct4-92nvz
sudo: effective uid is not 0, is /usr/bin/sudo on a file system with the 'nosuid' option set or an NFS file system without root privileges?
Using /etc/docker/daemon.json with the following content:
---
cat: /etc/docker/daemon.json: No such file or directory
---
Using /etc/supervisor/conf.d/dockerd.conf with the following content:
---
[program:dockerd]
command=/usr/local/bin/dockerd
autostart=true
autorestart=true
stderr_logfile=/var/log/dockerd.err.log
stdout_logfile=/var/log/dockerd.out.log
---
2022-04-18 11:39:44.657  DEBUG --- Starting supervisor daemon
2022-04-18 11:39:44.661  DEBUG --- Waiting for processes to be running...
2022-04-18 11:39:44.666  DEBUG --- Process dockerd is not running yet. Retrying in 1 seconds
2022-04-18 11:39:44.668  DEBUG --- Waited 0 seconds of 30 seconds
2022-04-18 11:39:45.674  DEBUG --- Process dockerd is not running yet. Retrying in 1 seconds
2022-04-18 11:39:45.677  DEBUG --- Waited 1 seconds of 30 seconds
2022-04-18 11:39:46.683  DEBUG --- Process dockerd is not running yet. Retrying in 1 seconds
2022-04-18 11:40:14.96  DEBUG --- Waited 29 seconds of 30 seconds
2022-04-18 11:40:15.102  ERROR --- dockerd is not running after max time
Dumping /var/log/dockerd.err.log to aid investigation
---
cat: /var/log/dockerd.err.log: No such file or directory
---
You have mail in /var/spool/mail/root

[mumoshu]

@roshvin What if you run it without runAsUser: 1000 and without the seLinuxOptions options? It seems like it's failing because you're blocking it with those "security" features.

[roshvin]

Hi @mumoshu as per our Kubernetes policy , it wont allow to run container as root.
in this scenario could you pls suggest any alternate way of using this feature running the container as non-root user.

[mumoshu]

I've changed the title of this question to better reflect your goal.

[mumoshu]

@roshvin It should really be the responsibility of your security team who configured the policy, to provide a feasible way to achieve what they ask!! Security and usability needs to achieved at the same time, otherwise folks start looking for ways to bypass the security mechanism at all.

Well anyway, it really depends on your requirement. You can try other container runtimes for runing pods, like sysbox and kata containers. The former should be able to run a runner pod with no privileged containers to run dind. The latter should be able to run privileged pods in a microvm which can be said safer than standard privileged pods.

You can use another way to build container images, like kaniko, which doesn't depend on dind.

However, in case you run self-hosted runners in a regulated environment and you are quite sure that self-hosted runners are used with private github repositories only, chances are you don't necessarily forbid privileged containers and instead make an exception. Furthermore, if you need to avoid privileged containers to prevent chances of runner pods for one repository to access credentials associated with another repository, it might be enough to use podAntiAffinity in combination with taints and tolerations to avoid allocating pods across repositories to one node.

That said, I'd recommend you discuss with your security team what the most viable option is for your organization.

[roshvin]

hi @mumoshu thanks for the detailed explanation. i went through all you links provide and couldn't find any working solution to run as non-privileged user.
do you remember any user with a working solution for this issue. and if yes could you please suggest .