Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error using buildx create using the default docker context with TLS #2520

Open
3 tasks done
chrisbecke opened this issue Jun 13, 2024 · 3 comments
Open
3 tasks done

Comments

@chrisbecke
Copy link

Contributing guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

ERROR when calling docker buildx create using the default, docker context with a TLS protected docker socket.

Expected behaviour

  • docker buildx create to succeed with a non TLS default docker context
  • docker buildx create to succeed with a TLS default docker context

Actual behaviour

  • docker buildx create with a non TLS docker context creates a new buildx builder
  • docker buildx create with a TLS default docker context emits the following error:
ERROR: could not create a builder instance with TLS data loaded from environment. Please use `docker context create <context-name>` to create a context for current environment and then create a builder instance with context set to <context-name>

Buildx version

github.com/docker/buildx v0.14.1 59582a8

Docker info

Client:
 Version:    24.0.6
 Context:    jack
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.5
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.22.0-desktop.2
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.8
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-scan
  scout: Docker Scout (Docker Inc.)
    Version:  v1.0.7
    Path:     /Users/chris.becke/.docker/cli-plugins/docker-scout

Server:
 Containers: 49
  Running: 28
  Paused: 0
  Stopped: 21
 Images: 40
 Server Version: 25.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: runc custom io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-94-generic
 Operating System: Ubuntu 22.04.1 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 64
 Total Memory: 62.75GiB
 Name: jack
 ID: 16905e46-a40d-4722-bd49-bfa4253291bf
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Builders list

NAME/NODE     DRIVER/ENDPOINT   STATUS    BUILDKIT   PLATFORMS
default*      docker                                 
 \_ default    \_ default       running   v0.13.2    linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

Configuration

compose.yaml

volumes:
  certs:
services:
  docker:
    image: docker:dind
    privileged: true
    volumes:
    - certs:/certs

  cli:
    image: docker:cli
    depends_on:
      - docker
    volumes:
    - certs:/certs
# verify the cli can use docker using the tls connection
docker compose run cli info
# Try run docker buildx create
docker compose run cli buildx create --use

compose.yaml

  docker:
    image: docker:dind
    privileged: true
    environment:
      DOCKER_TLS_CERTDIR: ""
    command: --tls=false

  cli:
    image: docker:cli
    depends_on:
      - docker
    environment:
      DOCKER_TLS_CERTDIR: ""
# verify docker is no-tls
docker compose run cli info
# See that docker buildx create succeeds with the default context
docker compose run cli buildx create --use

Build logs

ERROR: could not create a builder instance with TLS data loaded from environment. Please use `docker context create <context-name>` to create a context for current environment and then create a builder instance with context set to <context-name>

Additional info

No response

@crazy-max
Copy link
Member

crazy-max commented Jul 29, 2024

@chrisbecke Can you show the output of docker context inspect?

Edit: I can find it with docker compose run cli context inspect:

[
    {
        "Name": "default",
        "Metadata": {},
        "Endpoints": {
            "docker": {
                "Host": "tcp://docker:2376",
                "SkipTLSVerify": false
            }
        },
        "TLSMaterial": {
            "docker": [
                "ca.pem",
                "cert.pem",
                "key.pem"
            ]
        },
        "Storage": {
            "MetadataPath": "\u003cIN MEMORY\u003e",
            "TLSPath": "\u003cIN MEMORY\u003e"
        }
    }
]

@crazy-max
Copy link
Member

crazy-max commented Jul 29, 2024

Similar to docker/setup-buildx-action#105 (comment), you need to create a context for current environment when current context has TLS data loaded and set the endpoint when creating the builder.

With your example:

volumes:
  certs:
services:
  docker:
    image: docker:dind
    privileged: true
    volumes:
    - certs:/certs

  cli:
    image: docker:cli
    depends_on:
      - docker
    volumes:
    - certs:/certs
$ docker compose run --rm -it cli sh
/ # 
/ # 
/ # docker context ls
NAME        DESCRIPTION                               DOCKER ENDPOINT     ERROR
default *   Current DOCKER_HOST based configuration   tcp://docker:2376
Warning: DOCKER_HOST environment variable overrides the active context. To use a context, either set the global --context flag, or unset DOCKER_HOST environment variable.
/ # 
/ # 
/ # docker context inspect
[
    {
        "Name": "default",
        "Metadata": {},
        "Endpoints": {
            "docker": {
                "Host": "tcp://docker:2376",
                "SkipTLSVerify": false
            }
        },
        "TLSMaterial": {
            "docker": [
                "key.pem",
                "ca.pem",
                "cert.pem"
            ]
        },
        "Storage": {
            "MetadataPath": "\u003cIN MEMORY\u003e",
            "TLSPath": "\u003cIN MEMORY\u003e"
        }
    }
]
/ # 
/ # 
/ # docker context create builder
builder
Successfully created context "builder"
/ # 
/ # 
/ # docker buildx create --name foo --bootstrap --use builder
[+] Building 6.2s (1/1) FINISHED
 => [internal] booting buildkit                                                                                                                                                                           6.1s 
 => => pulling image moby/buildkit:buildx-stable-1                                                                                                                                                        5.5s
 => => creating container buildx_buildkit_foo0                                                                                                                                                            0.7s
foo
/ # 
/ # 
/ # docker buildx ls
NAME/NODE     DRIVER/ENDPOINT    STATUS    BUILDKIT   PLATFORMS
foo*          docker-container
 \_ foo0       \_ builder        running   v0.15.1    linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
builder       docker
 \_ builder    \_ builder        running   v0.13.2    linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default       docker
 \_ default    \_ default        running   v0.13.2    linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
/ # 
/ # 
/ # docker buildx build .

@chrisbecke
Copy link
Author

There is no dispute that the workaround is easy :- in the context of CI pipelines however it becomes messy as it adds another unexpected failure mode when (trying) to update from an unencrypted to a tls based docker:dind connection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants