fedora docker build fails

[jaeheum]

I have tried the docker build on Fedora31 (5.8.13-100.fc31.x86_64, gcc 9.3.1) with a small modification to the docker incantation: append :Z to -v and add -u root:root to share managarm dir.

xbstrap fails with configuring vim (wide char issue)

checking size of wchar_t is 2 bytes... configure: error: failed to compile test program
xbstrap: Action configure of package vim failed

and xbstrap install --all --keep-going ends with

xbstrap: Skipping action configure of base-devel due to failed prerequisites [264/266]
xbstrap: Skipping action build of base-devel due to failed prerequisites [265/266]
xbstrap: Skipping action install of base-devel due to failed prerequisites [266/266]
xbstrap: The following steps failed:
    configure      coreutils
    build          coreutils (prerequisites failed)
    install        coreutils (prerequisites failed)
    configure      vim
    build          vim (prerequisites failed)
    install        vim (prerequisites failed)
    configure      base (prerequisites failed)
    build          base (prerequisites failed)
    install        base (prerequisites failed)
    configure      base-devel (prerequisites failed)
    build          base-devel (prerequisites failed)
    install        base-devel (prerequisites failed)
xbstrap: Plan failed

I have never used xbstrap before and I'd appreciate some pointer. Thanks!

[avdgrinten]

Thanks you for the report. For clarification: you built the docker image from our Dockerfile and only modified the docker command line, right?

[jaeheum]

Yes I followed https://github.com/managarm/bootstrap-managarm#creating-docker-image-and-container
That gave me permissions error because of host/docker sharing of managarm, and I amended the docker command line.

docker run --group-add root -v $(realpath "$MANAGARM_DIR"):/home/managarm_buildenv/managarm:Z -itu root:root  managarm_buildenv

[Dennisbonke]

So I think I know why vim fails, and I’ll try to fix that tomorrow. I’m curious why coreutils failed for you, do you have a error for that as well?

[jaeheum]

: ~/m/build 12 ; docker --version
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
podman version 2.1.1

.... # start docker

root@88d6f33f8eba:/home/managarm_buildenv# ls -la
total 28
drwxr-xr-x. 3 managarm_buildenv managarm_buildenv 4096 Oct 10 21:02 .
drwxr-xr-x. 3 root              root              4096 Oct 10 21:02 ..
-rw-r--r--. 1 managarm_buildenv managarm_buildenv  220 Apr 18  2019 .bash_logout
-rw-r--r--. 1 managarm_buildenv managarm_buildenv 3526 Apr 18  2019 .bashrc
-rw-r--r--. 1 managarm_buildenv managarm_buildenv   70 Oct  9 21:56 .gitconfig
-rw-r--r--. 1 managarm_buildenv managarm_buildenv  807 Apr 18  2019 .profile
drwxr-xr-x. 4 root              root              4096 Oct 10 00:13 managarm
root@88d6f33f8eba:/home/managarm_buildenv# xbstrap install --rebuild coreutils
...
checking for mkfifo... yes
checking whether mkfifo rejects trailing slashes... yes
checking whether mknod can create fifo without root privileges... configure: error: in `/home/managarm_buildenv/managarm/build/pkg-builds/coreutils':
configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check)
See `config.log' for more details
xbstrap: Action configure of package coreutils failed

https://gist.github.com/jaeheum/544c72f5fc6bdfd46d9f9b30a4c7dc47 has the config.log

I don't like running as root even inside a docker image either... can we get the managarm chown'ed at the image creation time?

[Dennisbonke]

Ah, so for the coreutils issue I think you need @avdgrinten. I don’t think we need root at any time during the build (image creation excluded, and even that can be done without due to libguestfs). I’ll need to verify the vim fix, I’m planning on having that done tomorrow

[Dennisbonke]

As of 19a3074 the vim configure issue should be fixed, leaving this open for the docker issue that @avdgrinten is looking at

[jaeheum]

just tried running xbstrap with commit c226694 (HEAD -> master, origin/master, origin/HEAD)

• xbstrap install --rebuild coreutils runs into the same problem still.
• something in tzdata is also wonky

xbstrap: checkout tzdata [164/287]
Traceback (most recent call last):
  File "/usr/local/bin/xbstrap", line 484, in <module>
    do_install(args)
  File "/usr/local/bin/xbstrap", line 358, in do_install
    plan.run_plan()
  File "/usr/local/lib/python3.7/dist-packages/xbstrap/base.py", line 2297, in run_plan
    checkout_src(self._cfg, subject, self._settings)
  File "/usr/local/lib/python3.7/dist-packages/xbstrap/base.py", line 1365, in checkout_src
    prefix = source['extract_path'] + '/'
KeyError: 'extract_path'
root@df8f7b7270dc:/home/managarm_buildenv/managarm/build# 

[Dennisbonke]

Seems like you have an old xbstrap version, can you try upgrading xbstrap with pip3 install -U xbstrap and try it again?

[Dennisbonke]

The docker root issue is indeed not yet fixed, pinging @avdgrinten for that again

[jaeheum]

is xbstrap 0.16 reasonably up-to-date? (it is giving me the same error)

with src/ at 3ad1f4b xbstrap isn't too happy with the yml files anyways

xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.yml, YAML element: general
           Additional properties are not allowed ('everything_by_default' was unexpected)
         * Error in file: ../src/bootstrap.yml, YAML element: packages/26
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/app-admin.yml, YAML element: packages/0
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/dev-libs.yml, YAML element: packages/7
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/dev-util.yml, YAML element: packages/2
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/sys-apps.yml, YAML element: packages/6
           Additional properties are not allowed ('stability_level' was unexpected)
         * Error in file: ../src/bootstrap.d/sys-apps.yml, YAML element: packages/8
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/sys-devel.yml, YAML element: packages/1
           Additional properties are not allowed ('stability_level' was unexpected)
         * Error in file: ../src/bootstrap.d/sys-devel.yml, YAML element: packages/2
           Additional properties are not allowed ('stability_level' was unexpected)
         * Error in file: ../src/bootstrap.d/sys-devel.yml, YAML element: packages/3
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future
xbstrap: Failed to validate boostrap.yml
         * Error in file: ../src/bootstrap.d/x11-misc.yml, YAML element: packages/0
           Additional properties are not allowed ('stability_level' was unexpected)
xbstrap: Validation issues will become hard errors in the future

[avdgrinten]

xbstrap 0.16 is up to date. It should not result in these validation errors. Is xbstrap inside the Docker image up to date?

Regarding the configure error in binutils: it seems that the configure script is not happy when run as root. Is there any specific reason why you run as root in the container and not as a normal user?

[jaeheum]

Running the documented docker build and run incatations on Fedora ends with ~/managarm that is chown root:root. (Sigh) I think I have a solution, but it will be at least a few days before I can document it accurately. Building everything takes a long time...

[ArsenArsen]

post the exact commands you ran. the docker image has been used for development and in production for building for a few years, it should not result in bad file ownership if your users id is 1000

[jaeheum]

On Fedora 31, docker is an alias for podman ("rootless docker") and running the documented

docker build -t managarm_buildenv src/docker
docker run -v $(realpath "$MANAGARM_DIR"):/home/managarm_buildenv/managarm -it managarm_buildenv

ends with ~/managarm chown root:root.

If I modify the docker/podman incantation to

sudo docker build -t managarm_buildenv src/docker
sudo docker run -v $(realpath "$MANAGARM_DIR"):/home/managarm_buildenv/managarm:Z -itu managarm_buildenv:managarm_buildenv  managarm_buildenv

then ~/managarm chown managarm_buildenv:managarm_buildenv and xbstrap runs ok.

On Arch, I prefixed the documented docker incantation with sudo.

[avdgrinten]

Thank you for figuring this out. Strangely, on Debian (with Docker CE instead of podman), sudo is not required. We should converge to a solution that does not require sudo on any distribution.

[ArsenArsen]

I assume this is a breakage in podman at the end of the day. I will have to investigate personally at some point. podmans intention clearly is to stay compatible with docker, but rootless, so this would seem like a problem.
as for whether sudo is required, it is quite simple: /var/run/docker.sock is how docker communicates with it's daemon, access to this file is access to all of docker itself. it tends to be owned by a docker group (example from my system: Access: (0660/srw-rw----) Uid: ( 0/ root) Gid: ( 994/ docker)), so adding yourself to the docker group is sufficient. I do not think there needs to be any method change here to accommodate for that.
perhaps the reason why this worked with, but not without sudo, if you have actual docker installed, is because sudo will, for reasons obvious, not account for shell aliases, so sudo docker invokes actual docker.

EDIT: aliasing docker to podman in /etc/bash/bashrc, or whatever fedora does to make that alias a default, seems quite short sighted to me. try again, launching docker directly.
EDIT 2: :Z on docker-runs -v should have no effect, I believe