Runners Without Embedded Docker Daemon

[matthenry87]

We are in the process of setting up kpack and want to delegate image creation+pushing to that. Security doesn't want anyone to be able to do a docker run and spin up arbitrary containers in the context of a CI/CD pipeline, or run privileged containers for that matter.

What is the best way to remove docker from the runners? If I set dockerdWithinRunnerContainer to true, and just never try to invoke it, is that the way to go?

As I type this - I do realize that a lot of GitHub Actions are implemented as containers - so it looks like I'll likely be switching to using the 'kubernetes' container mode, so that K8s will be the one to launch the individual actions (I'll have to see how it behaves if I'm trying to execute shell commands like mvn compile etc - maybe it will stay in the original runner pod unless it's a docker-based action).

Is switching to this new container mode all I need?

Thanks!

[matthenry87]

I figured this out - but the kubernetes containerMode seems to be broken:

Errors I get:

Error: Missing file at path: /runner/_work/_temp/_runner_hook_responses/<redacted>.json

Access to the path '/runner/_work/_temp/_github_workflow' is denied.

And when I shell into the pod, I cannot cd into /runner/_work or list it's contents despite being the owner:

runner@k8s-container-mode:/$ cd /runner
runner@k8s-container-mode:/runner$ ls -la
total 104
drwxrwxrwx. 8 runner docker  4096 Apr 15 20:25 .
drwxr-xr-x. 1 root   root    4096 Apr 15 20:25 ..
-rw-r--r--. 1 runner runner   266 Apr 15 20:25 .credentials
-rw-------. 1 runner runner  1667 Apr 15 20:25 .credentials_rsaparams
-rw-r--r--. 1 runner runner    58 Apr 15 20:25 .env
-rw-r--r--. 1 runner runner   134 Apr 15 20:25 .path
-rw-r--r--. 1 runner runner   338 Apr 15 20:25 .runner
drwxr-xr-x. 4 runner runner  4096 Apr 15 20:25 _diag
drwxr-xr-x. 8 runner docker  4096 Apr 15 20:26 _work
drwxr-xr-x. 4 runner runner 16384 Apr 15 20:25 bin
-rwxr-xr-x. 1 runner runner  2458 Apr 15 20:25 config.sh
-rwxr-xr-x. 1 runner runner   646 Apr 15 20:25 env.sh
drwxr-xr-x. 6 runner runner  4096 Apr 15 20:25 externals
drwxr-xr-x. 2 runner runner  4096 Apr 15 20:25 externalstmp
drwxr-xr-x. 2 runner runner  4096 Apr 15 20:25 k8s
-rw-r--r--. 1 runner runner  1487 Apr 15 20:25 run-helper.cmd.template
-rwxr-xr-x. 1 runner runner  2522 Apr 15 20:25 run-helper.sh
-rwxr-xr-x. 1 runner runner  2522 Apr 15 20:25 run-helper.sh.template
-rwxr-xr-x. 1 runner runner  2537 Apr 15 20:25 run.sh
-rwxr-xr-x. 1 runner runner    65 Apr 15 20:25 safe_sleep.sh
-rwxr-xr-x. 1 runner runner  5222 Apr 15 20:25 svc.sh
runner@k8s-container-mode:/runner$ cd _work
bash: cd: _work: Permission denied
runner@k8s-container-mode:/runner$

I also just re-built my customized runner image on top of docker.io/summerwind/actions-runner:latest and made sure my runner's ImagePullPolicy is set to Always.

The volume is successfully being attached (or else runner pod wouldn't have come up).

[matthenry87]

actions/runner#981 (comment)

[vishu42]

@matthenry87 if I understand your concern right and you want to run the Runner with a docker sidecar then there is an option to do so. You can disable docker like so

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: example-runnerdeploy
spec:
  replicas: 1
  template:
    spec:
      repository: gdo-globaldevops-devops/hello-world-app-me
      dockerEnabled: false

[matthenry87]

Turns out I was needing to use the kubernetes containerMode.

However, seems to currently be broken:
#2438

[matthenry87]

Also fyi: The self hosted runner already runs a docker sidecar container. We have made customizations to the dind default image and set it as the docker image all created runners use by default.

[afernandez-01]

Hi @vishu42 I wish to run only the runners without the docker sidecar.
I've added this to my runner deployment spec but the runners keep getting terminated
dockerEnabled: false