Martin Kletzander
2021-Jun-14 22:25 UTC
[Libguestfs] virt-builder fails in fedora 34 container:
On Tue, Jun 15, 2021 at 12:15:35AM +0300, Nir Soffer wrote:>On Mon, Jun 14, 2021 at 11:50 PM Nir Soffer <nsoffer at redhat.com> wrote: >> >> On Mon, Jun 14, 2021 at 7:47 PM Martin Kletzander <mkletzan at redhat.com> wrote: >> > >> > On Mon, Jun 14, 2021 at 01:01:37PM +0100, Richard W.M. Jones wrote: >> > >On Mon, Jun 14, 2021 at 02:20:40PM +0300, Nir Soffer wrote: >> > >> On Mon, Jun 14, 2021 at 11:57 AM Martin Kletzander <mkletzan at redhat.com> wrote: >> > >> >> > >> > I've got a tiny hunch, so I must ask. Is someone, by any stroke >> > >> > of misfortune, using bazel for building that imageio container >> > >> > image? >> > >> >> > >> We build the images in quay.io, I don't know how they build them :-) >> > > >> > >The issue that Martin is hinting at is that bazel does not run %post >> > >scripts (or create an RPM database). IOW the tools is broken. >> > > >> > >If you have an RPM database in the final container then it's probably >> > >_not_ bazel. If anyone tells you to use bazel, run away! >> > > >> > >> > +1M >> > >> > Anyway, I can't stop thinking about what would be the reasonable way of fixing >> > the issue at hand properly. It seems like a huge deal to depend on kernel-core, >> > but if you need a kernel and modules during runtime, then I guess it makes >> > sense. >> >> I tried installing kernel-core package: >> >> $ podman run -it --rm quay.io/ovirt/imageio-test-fedora-34 >> # dnf install kernel-core >> ... >> Package kernel-core-5.12.9-300.fc34.x86_64 is already installed. >> >> So I reinstalled it: >> # dnf reinstall kernel-core >> ... >> >> And after that modules.dep was created: >> >> # ls /lib/modules/5.12.9-300.fc34.x86_64/ >> System.map modules.alias modules.builtin.bin >> modules.drm modules.symbols vdso >> bls.conf modules.alias.bin modules.builtin.modinfo >> modules.modesetting modules.symbols.bin vmlinuz >> build modules.block modules.dep >> modules.networking source weak-updates >> config modules.builtin modules.dep.bin >> modules.order symvers.gz >> kernel modules.builtin.alias.bin modules.devname >> modules.softdep updates >> >> And virt-builder is working: >> >> # LIBGUESTFS_BACKEND=direct virt-builder cirros-0.3.5 -o test.img >> [ 2.2] Downloading: http://builder.libguestfs.org/cirros-0.3.5.xz >> [ 2.2] Planning how to build this image >> [ 2.2] Uncompressing >> [ 2.6] Opening the new disk >> [ 26.6] Setting a random seed >> [ 26.9] Setting passwords >> virt-builder: warning: password: using insecure md5 password encryption for >> guest of type cirros version 0.3. >> If this is incorrect, use --password-crypto option and file a bug. >> virt-builder: Setting random password of root to mbeNkcd6DVbSWmNn >> [ 31.9] Finishing off >> Output file: test.img >> Output size: 39.2M >> Output format: raw >> Total usable space: 22.2M >> Free space: 5.3M (23%) >> >> So kernel-core was already installed, but for some reason its scriptlet >> did not create modules.dep when the container was built. >> >> I looked at the quay.io logs creating this container: >> https://quay.io/repository/ovirt/imageio-test-fedora-34/build/08d6965f-9d8f-4217-9f71-aee1e8267553 >> >> And we do see the scriptlet running: >> >> 6/8/2021, 1:30:44 PMInstalling : kernel-core-5.12.9-300.fc34.x86_64 254/373 >> 6/8/2021, 1:30:44 PMRunning scriptlet: >> kernel-core-5.12.9-300.fc34.x86_64 254/373 >> ... >> 6/8/2021, 1:31:05 PMRunning scriptlet: >> kernel-core-5.12.9-300.fc34.x86_64 373/373 >> >> So maybe this is an issue with the install scriptlet when building in quay.io >> >> When I build the container locally on Fedora 32, modules.dep is created >> and virt-builder works. But I want to build in quay.io, it is much more >> convenient. > >Fixed also by running depmod before running virt-builder: > ># depmod -a 5.12.9-300.fc34.x86_64 >Maybe they are overusing multi-stage builds or something. It is difficult to figure out as I do not have access to those logs (even though it lets me "cancel the build" - yes, the old and finished one) and I do not know from which Containerfile it was built. I looked at GitHub and saw this: https://github.com/oVirt/ovirt-imageio/tree/d163d80f04e5ae64f8e0138463aae5e867fd1fa8/docker if that is where the images are built from, then it clearly is an issue in quay because a) there is nothing that suggests a missing file and b) I tried it locally and it works. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://listman.redhat.com/archives/libguestfs/attachments/20210615/c53645c0/attachment.sig>
On Tue, Jun 15, 2021 at 1:26 AM Martin Kletzander <mkletzan at redhat.com> wrote:> > On Tue, Jun 15, 2021 at 12:15:35AM +0300, Nir Soffer wrote: > >On Mon, Jun 14, 2021 at 11:50 PM Nir Soffer <nsoffer at redhat.com> wrote: > >> > >> On Mon, Jun 14, 2021 at 7:47 PM Martin Kletzander <mkletzan at redhat.com> wrote: > >> > > >> > On Mon, Jun 14, 2021 at 01:01:37PM +0100, Richard W.M. Jones wrote: > >> > >On Mon, Jun 14, 2021 at 02:20:40PM +0300, Nir Soffer wrote: > >> > >> On Mon, Jun 14, 2021 at 11:57 AM Martin Kletzander <mkletzan at redhat.com> wrote: > >> > >> > >> > >> > I've got a tiny hunch, so I must ask. Is someone, by any stroke > >> > >> > of misfortune, using bazel for building that imageio container > >> > >> > image? > >> > >> > >> > >> We build the images in quay.io, I don't know how they build them :-) > >> > > > >> > >The issue that Martin is hinting at is that bazel does not run %post > >> > >scripts (or create an RPM database). IOW the tools is broken. > >> > > > >> > >If you have an RPM database in the final container then it's probably > >> > >_not_ bazel. If anyone tells you to use bazel, run away! > >> > > > >> > > >> > +1M > >> > > >> > Anyway, I can't stop thinking about what would be the reasonable way of fixing > >> > the issue at hand properly. It seems like a huge deal to depend on kernel-core, > >> > but if you need a kernel and modules during runtime, then I guess it makes > >> > sense. > >> > >> I tried installing kernel-core package: > >> > >> $ podman run -it --rm quay.io/ovirt/imageio-test-fedora-34 > >> # dnf install kernel-core > >> ... > >> Package kernel-core-5.12.9-300.fc34.x86_64 is already installed. > >> > >> So I reinstalled it: > >> # dnf reinstall kernel-core > >> ... > >> > >> And after that modules.dep was created: > >> > >> # ls /lib/modules/5.12.9-300.fc34.x86_64/ > >> System.map modules.alias modules.builtin.bin > >> modules.drm modules.symbols vdso > >> bls.conf modules.alias.bin modules.builtin.modinfo > >> modules.modesetting modules.symbols.bin vmlinuz > >> build modules.block modules.dep > >> modules.networking source weak-updates > >> config modules.builtin modules.dep.bin > >> modules.order symvers.gz > >> kernel modules.builtin.alias.bin modules.devname > >> modules.softdep updates > >> > >> And virt-builder is working: > >> > >> # LIBGUESTFS_BACKEND=direct virt-builder cirros-0.3.5 -o test.img > >> [ 2.2] Downloading: http://builder.libguestfs.org/cirros-0.3.5.xz > >> [ 2.2] Planning how to build this image > >> [ 2.2] Uncompressing > >> [ 2.6] Opening the new disk > >> [ 26.6] Setting a random seed > >> [ 26.9] Setting passwords > >> virt-builder: warning: password: using insecure md5 password encryption for > >> guest of type cirros version 0.3. > >> If this is incorrect, use --password-crypto option and file a bug. > >> virt-builder: Setting random password of root to mbeNkcd6DVbSWmNn > >> [ 31.9] Finishing off > >> Output file: test.img > >> Output size: 39.2M > >> Output format: raw > >> Total usable space: 22.2M > >> Free space: 5.3M (23%) > >> > >> So kernel-core was already installed, but for some reason its scriptlet > >> did not create modules.dep when the container was built. > >> > >> I looked at the quay.io logs creating this container: > >> https://quay.io/repository/ovirt/imageio-test-fedora-34/build/08d6965f-9d8f-4217-9f71-aee1e8267553 > >> > >> And we do see the scriptlet running: > >> > >> 6/8/2021, 1:30:44 PMInstalling : kernel-core-5.12.9-300.fc34.x86_64 254/373 > >> 6/8/2021, 1:30:44 PMRunning scriptlet: > >> kernel-core-5.12.9-300.fc34.x86_64 254/373 > >> ... > >> 6/8/2021, 1:31:05 PMRunning scriptlet: > >> kernel-core-5.12.9-300.fc34.x86_64 373/373 > >> > >> So maybe this is an issue with the install scriptlet when building in quay.io > >> > >> When I build the container locally on Fedora 32, modules.dep is created > >> and virt-builder works. But I want to build in quay.io, it is much more > >> convenient. > > > >Fixed also by running depmod before running virt-builder: > > > ># depmod -a 5.12.9-300.fc34.x86_64 > > > > Maybe they are overusing multi-stage builds or something. It is difficult to > figure out as I do not have access to those logs (even though it lets me "cancel > the build" - yes, the old and finished one) and I do not know from which > Containerfile it was built. > > I looked at GitHub and saw this: > > https://github.com/oVirt/ovirt-imageio/tree/d163d80f04e5ae64f8e0138463aae5e867fd1fa8/docker > > if that is where the images are built from,Correct> then it clearly is an issue in quay > because a) there is nothing that suggests a missing file and b) I tried it > locally and it works.Thanks for looking. Attached the secret build logs if you want to look at them. -------------- next part -------------- A non-text attachment was scrubbed... Name: 08d6965f-9d8f-4217-9f71-aee1e8267553.json.xz Type: application/x-xz Size: 29668 bytes Desc: not available URL: <http://listman.redhat.com/archives/libguestfs/attachments/20210615/76026db8/attachment.xz>