From ba45b296ec11734bafcae7728915016f17137a3d Mon Sep 17 00:00:00 2001 From: Jonathan Lebon Date: Tue, 13 Aug 2024 10:50:13 -0400 Subject: [PATCH] cmdlib: workaround rofiles-fuse mounts leaking This started happening in CI: ``` + mount -o remount,ro /srv/cache mount: /srv/cache: mount point is busy. dmesg(1) may have more information after failed mount system call. [ 321.026584] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00002000 ``` Digging into it, it seems to be due to leftover rofiles-fuse mounts: ``` + ps aux ... root 306 0.0 0.1 401092 3972 ? Ssl 15:46 0:00 rofiles-fuse --copyup usr /tmp/rpmostree-rofiles-fuse0QroLi root 311 0.0 0.1 251564 2488 ? Ssl 15:46 0:00 rofiles-fuse --copyup etc /tmp/rpmostree-rofiles-fuseANAAgx root 330 0.0 0.2 474828 4120 ? Ssl 15:46 0:00 rofiles-fuse --copyup usr /tmp/rpmostree-rofiles-fuseioSZ7r root 334 0.0 0.1 251564 2712 ? Ssl 15:46 0:00 rofiles-fuse --copyup etc /tmp/rpmostree-rofiles-fusePPGBR9 ... ``` For some reason, rpm-ostree is not able to unmount those. We should dig into this but as a brutal workaround, just kill them manually. There's no data loss concerns here; the commit has already been written and shipped out via virtiofs. The issue remains tracked at https://github.com/coreos/coreos-assembler/issues/3848. --- src/cmdlib.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/cmdlib.sh b/src/cmdlib.sh index 0f1210615c..442d13f71c 100755 --- a/src/cmdlib.sh +++ b/src/cmdlib.sh @@ -758,6 +758,8 @@ else fi echo \$rc > ${rc_file} if [ -n "\${cachedev}" ]; then + # XXX: brutal workaround for https://github.com/coreos/coreos-assembler/issues/3848 + killall rofiles-fuse || : /sbin/fstrim -v ${workdir}/cache mount -o remount,ro ${workdir}/cache fsfreeze -f ${workdir}/cache