On Mon, Jun 14, 2021 at 11:57 AM Martin Kletzander <mkletzan at redhat.com> wrote:> > On Sun, Jun 13, 2021 at 11:15:31PM +0300, Nir Soffer wrote: > >On Sun, Jun 13, 2021 at 1:38 PM Richard W.M. Jones <rjones at redhat.com> wrote: > >> > >> On Sat, Jun 12, 2021 at 11:11:22PM +0300, Nir Soffer wrote: > >> > I'm running this inside a fedora 34 container in ovirt-imageio CI: > >> > > >> > 'virt-builder', '-v', '-x', 'cirros-0.3.5', '--write', > >> > '/etc/cirros-init/config:DATASOURCE_LIST="nocloud"', > >> > '--root-password', 'password:', '-o', > >> > '/var/tmp/imageio-images/cirros-0.3.5' > >> > > >> > The command fails in: > >> > > >> > supermin: ext2: copying kernel modules > >> > supermin: ext2: creating minimal initrd > >> > '/var/tmp/.guestfs-0/appliance.d.t7nny0l7/initrd' > >> > supermin: exception: > >> > Sys_error("/lib/modules/5.12.9-300.fc34.x86_64/modules.dep: No such > >> > file or directory") > >> > >> We use this file in order to construct the minimal initrd with some > >> virtio modules, which is sufficient to bring up the supermin root > >> filesystem. The file itself is used to calculate the correct set of > >> module dependencies (so that, for example, if a virtio kmod needs > >> another kmod then the other kmod will be pulled in). See: > >> > >> https://github.com/libguestfs/supermin/blob/c89e4923909ef84541ac1c047bf64d13b1595b4f/src/format_ext2_initrd.ml > >> > >> I believe in Fedora the file is created via a %post script in > >> kernel-core which runs /bin/kernel-install (from systemd-udev). This > >> in turn is a shell script which runs snippets from > >> /etc/kernel/install.d and /usr/lib/kernel/install.d which are supplied > >> from all kinds of other packages. On my machine one of these is > >> /usr/lib/kernel/install.d/50-depmod.install (also from systemd-udev) > >> which eventually runs depmod -a which creates this file. > >> > >> So those are some of the packages which might be missing. Does > >> modules.dep get created anywhere inside the container? It's possible > >> it moved. > >> > >> It's also possible to set the environment variable KERNEL_INSTALL_VERBOSE=1 > >> which should show (in the dnf install output I guess?) if/when the > >> depmod command runs. > >> > >> > I think I'm missing some package removed from the fedora 34 container, > >> > but I don't have > >> > any idea which one. > >> > > >> > I hope someone has more insight on this. > >> > >> It's curious that things have changed since F33. Could you build > >> containers for both and diff the list of package names? Maybe a > >> dependency is missing or has become a soft dep. > >> > >> rpm -qa --qf '%{name}\n' | sort > > > >I ran this: > > > >$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-34 rpm -qa > >--qf '%{name}\n' | sort > fedora-34.out > >$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-33 rpm -qa > >--qf '%{name}\n' | sort > fedora-33.out > >$ diff -u fedora-33.out fedora-34.out > fedora.diff > >$ egrep -- ^- fedora.diff > removed-in-fedora-34 > >$ egrep -- '^[+]' fedora.diff > added-in-fedora-34 > > > >$ cat removed-in-fedora-34 > >--- fedora-33.out 2021-06-13 22:51:13.156150245 +0300 > >-autogen-libopts > >-genisoimage > >-iptables > >-isl > >-kbd-legacy > >-libcbor > >-libdb-utils > >-libfido2 > >-libguestfs-tools-c > >-libmodman > >-libusal > >-nmap-ncat > >-pciutils > >-pciutils-libs > >-python3-idna > >-rdma-core > >-selinux-policy-minimum > > > >$ cat added-in-fedora-34 > >+++ fedora-34.out 2021-06-13 22:50:59.650016271 +0300 > >+guestfs-tools > >+inih > >+iptables-legacy > >+iptables-legacy-libs > >+langpacks-core-en_GB > >+langpacks-en_GB > >+libbsd > >+libcanberra > >+libcanberra-gtk3 > >+libcloudproviders > >+libpmemobj > >+libretls > >+libtdb > >+netcat > >+pcsc-lite-libs > >+selinux-policy-targeted > >+sound-theme-freedesktop > > > >I don't see anything that can explain the difference. > > > >I also compared contents of the /lib/modules/xxx/ directories: > > > >$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-33 ls -1 > >/lib/modules/5.12.9-200.fc33.x86_64 > > >fedora-33-lib_modules_5.12.9-200.fc33.x86_64.out > >$ podman run -it --rm quay.io/ovirt/imageio-test-fedora-34 ls -1 > >/lib/modules/5.12.9-300.fc34.x86_64 > > >fedora-34-lib_modules_5.12.9-300.fc34.x86_64.out > >$ diff -u fedora-33-lib_modules_5.12.9-200.fc33.x86_64.out > >fedora-34-lib_modules_5.12.9-300.fc34.x86_64.out > >--- fedora-33-lib_modules_5.12.9-200.fc33.x86_64.out 2021-06-13 > >23:12:04.177539132 +0300 > >+++ fedora-34-lib_modules_5.12.9-300.fc34.x86_64.out 2021-06-13 > >23:12:18.072675988 +0300 > >@@ -3,23 +3,14 @@ > > build > > config > > kernel > >-modules.alias > >-modules.alias.bin > > modules.block > > modules.builtin > > modules.builtin.alias.bin > >-modules.builtin.bin > > modules.builtin.modinfo > >-modules.dep > >-modules.dep.bin > >-modules.devname > > modules.drm > > modules.modesetting > > modules.networking > > modules.order > >-modules.softdep > >-modules.symbols > >-modules.symbols.bin > > source > > symvers.gz > > updates > > > >Looks like some post install step is missing on the fedora 34 container. > > > > Just to be clear, that is on the ovirt/imageio container, not any usual > container out there. No upstream fedora container has anything in /lib/modules > because it usually is pointless to distribute something that is not going to be > needed as the container is not coupled with any kernel because it runs on the > host one.I think this is up to the package installing virt-builder. We cannot know about virt-builder dependencies and make sure they are met.> I tested upstream fedora:33 and fedora:34 and fedora-minimal:34 (with microdnf) > containers and all got modules.dep file from the kernel-(debug-)core package, > although it is not owned by the package, so it is clearly installed in a > postscript or similar. So it has to be something like not installing the > package correctly in the imageio container image.Thanks, I'll test adding this package. But shouldn't it be a dependency of the package installing virt-builder?> I've got a tiny hunch, so I must ask. Is someone, by any stroke of misfortune, > using bazel for building that imageio container image?We build the images in quay.io, I don't know how they build them :-) Nir
Richard W.M. Jones
2021-Jun-14 12:01 UTC
[Libguestfs] virt-builder fails in fedora 34 container:
On Mon, Jun 14, 2021 at 02:20:40PM +0300, Nir Soffer wrote:> On Mon, Jun 14, 2021 at 11:57 AM Martin Kletzander <mkletzan at redhat.com> wrote: > > > I've got a tiny hunch, so I must ask. Is someone, by any stroke > > of misfortune, using bazel for building that imageio container > > image? > > We build the images in quay.io, I don't know how they build them :-)The issue that Martin is hinting at is that bazel does not run %post scripts (or create an RPM database). IOW the tools is broken. If you have an RPM database in the final container then it's probably _not_ bazel. If anyone tells you to use bazel, run away! Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top