Laine Stump
2015-Feb-24 21:05 UTC
Re: [libvirt-users] Broken OS when booting rootfs from 9p share
On 02/24/2015 03:37 PM, Olivier Mauras wrote:>> Hello, >> >> I've been trying to boot a VM with the rootfs being a 9P share from >> the host. The VM OS is centos 7. >> The OS boots but no services can work and it appears that the >> authentication system is broken. >> >> Now the funny thing is that booting the same OS on the same 9P share >> manually with Qemu works as expected with a fully functionnal OS... So >> I'm wondering what could libvirt do that render the OS on the share >> broken - Something with rights ? >> Selinux is disabled on the host, and accessmode to the share is set to >> passthrough in both cases. >> >> Here's my working Qemu line: >> qemu >> -kernel /srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 \ >> >> -initrd /srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img \ >> -fsdev >> local,id=r,path=/srv/overlay/run/irc,security_model=passthrough \ >> -device virtio-9p-pci,fsdev=r,mount_tag=root \ >> -nographic \ >> -m 256M \ >> -machine pc-i440fx-2.1,accel=kvm \ >> -netdev tap,id=net0 -device >> virtio-net,netdev=net0,mac=52:54:00:12:36:34 \ >> -smp 1,sockets=1,cores=1 \ >> -append 'root=virtfs:root rootflags=trans=virtio,version=9p2000.u ro >> rootfstype=9p console=ttyS0' >> >> And here my libvirt VM config: >> <domain type='kvm'> >> <name>irc</name> >> <uuid>ffbd32eb-a693-eadd-8923-18de80137472</uuid> >> <memory unit='KiB'>262144</memory> >> <currentMemory unit='KiB'>262144</currentMemory> >> <vcpu placement='static'>1</vcpu> >> <os> >> <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type> >> >> <kernel>/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64</kernel> >> >> <initrd>/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img</initrd> >> <cmdline>root=virtfs:root rootflags=trans=virtio,version=9p2000.u >> ro rootfstype=9p</cmdline> >> <boot dev='hd'/> >> </os> >> <features> >> <acpi/> >> <apic/> >> <pae/> >> </features> >> <clock offset='utc'/> >> <on_poweroff>destroy</on_poweroff> >> <on_reboot>restart</on_reboot> >> <on_crash>restart</on_crash> >> <devices> >> <emulator>/usr/bin/qemu-system-x86_64</emulator> >> <controller type='usb' index='0'> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' >> function='0x2'/> >> </controller> >> <controller type='pci' index='0' model='pci-root'/> >> <filesystem type='mount' accessmode='passthrough'> >> <source dir='/srv/overlay/run/irc'/> >> <target dir='root'/> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' >> function='0x0'/> >> </filesystem> >> <interface type='bridge'> >> <mac address='52:54:00:00:36:34'/> >> <source bridge='br0'/> >> <model type='virtio'/> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' >> function='0x0'/> >> </interface> >> <serial type='pty'> >> <target port='0'/> >> </serial> >> <console type='pty'> >> <target type='serial' port='0'/> >> </console> >> <input type='mouse' bus='ps2'/> >> <input type='keyboard' bus='ps2'/> >> <graphics type='vnc' port='-1' autoport='yes'/> >> <video> >> <model type='cirrus' vram='9216' heads='1'/> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' >> function='0x0'/> >> </video> >> <memballoon model='virtio'> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' >> function='0x0'/> >> </memballoon> >> </devices> >> </domain> >> >> >> My libvirt version is 1.2.9, and qemu is 2.1.2. >> >> Thanks for your help, >> >> Olivier >> >> >> _______________________________________________ >> libvirt-users mailing list >> libvirt-users@redhat.com >> https://www.redhat.com/mailman/listinfo/libvirt-users > So I updated libvirt to 1.2.12 and qemu to 2.1.3 and it still gives me > the exact same behaviour ... > Could it be that the share was created with an old version of > virt-manager redenring in old/broken XML? From Qemu config example i > don't see how my XML could be incorrect. > > Any idea ?I know nothing about the 9p filesystem, but since you have a qemu commandline that works and a libvirt config that doesn't, the next logical step would be to provide the non-working qemu commandline generated by libvirt. You can find this at the end of the log file /var/log/libvirt/qemu/irc.log.
Olivier Mauras
2015-Feb-24 22:56 UTC
Re: [libvirt-users] Broken OS when booting rootfs from 9p share
On Tue, 2015-02-24 at 16:05 -0500, Laine Stump wrote:> On 02/24/2015 03:37 PM, Olivier Mauras wrote: > >> Hello, > >> > >> I've been trying to boot a VM with the rootfs being a 9P share from > >> the host. The VM OS is centos 7. > >> The OS boots but no services can work and it appears that the > >> authentication system is broken. > >> > >> Now the funny thing is that booting the same OS on the same 9P share > >> manually with Qemu works as expected with a fully functionnal OS... So > >> I'm wondering what could libvirt do that render the OS on the share > >> broken - Something with rights ? > >> Selinux is disabled on the host, and accessmode to the share is set to > >> passthrough in both cases. > >> > >> Here's my working Qemu line: > >> qemu > >> -kernel /srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 \ > >> > >> -initrd /srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img \ > >> -fsdev > >> local,id=r,path=/srv/overlay/run/irc,security_model=passthrough \ > >> -device virtio-9p-pci,fsdev=r,mount_tag=root \ > >> -nographic \ > >> -m 256M \ > >> -machine pc-i440fx-2.1,accel=kvm \ > >> -netdev tap,id=net0 -device > >> virtio-net,netdev=net0,mac=52:54:00:12:36:34 \ > >> -smp 1,sockets=1,cores=1 \ > >> -append 'root=virtfs:root rootflags=trans=virtio,version=9p2000.u ro > >> rootfstype=9p console=ttyS0' > >> > >> And here my libvirt VM config: > >> <domain type='kvm'> > >> <name>irc</name> > >> <uuid>ffbd32eb-a693-eadd-8923-18de80137472</uuid> > >> <memory unit='KiB'>262144</memory> > >> <currentMemory unit='KiB'>262144</currentMemory> > >> <vcpu placement='static'>1</vcpu> > >> <os> > >> <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type> > >> > >> <kernel>/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64</kernel> > >> > >> <initrd>/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img</initrd> > >> <cmdline>root=virtfs:root rootflags=trans=virtio,version=9p2000.u > >> ro rootfstype=9p</cmdline> > >> <boot dev='hd'/> > >> </os> > >> <features> > >> <acpi/> > >> <apic/> > >> <pae/> > >> </features> > >> <clock offset='utc'/> > >> <on_poweroff>destroy</on_poweroff> > >> <on_reboot>restart</on_reboot> > >> <on_crash>restart</on_crash> > >> <devices> > >> <emulator>/usr/bin/qemu-system-x86_64</emulator> > >> <controller type='usb' index='0'> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > >> function='0x2'/> > >> </controller> > >> <controller type='pci' index='0' model='pci-root'/> > >> <filesystem type='mount' accessmode='passthrough'> > >> <source dir='/srv/overlay/run/irc'/> > >> <target dir='root'/> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > >> function='0x0'/> > >> </filesystem> > >> <interface type='bridge'> > >> <mac address='52:54:00:00:36:34'/> > >> <source bridge='br0'/> > >> <model type='virtio'/> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > >> function='0x0'/> > >> </interface> > >> <serial type='pty'> > >> <target port='0'/> > >> </serial> > >> <console type='pty'> > >> <target type='serial' port='0'/> > >> </console> > >> <input type='mouse' bus='ps2'/> > >> <input type='keyboard' bus='ps2'/> > >> <graphics type='vnc' port='-1' autoport='yes'/> > >> <video> > >> <model type='cirrus' vram='9216' heads='1'/> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > >> function='0x0'/> > >> </video> > >> <memballoon model='virtio'> > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > >> function='0x0'/> > >> </memballoon> > >> </devices> > >> </domain> > >> > >> > >> My libvirt version is 1.2.9, and qemu is 2.1.2. > >> > >> Thanks for your help, > >> > >> Olivier > >> > >> > >> _______________________________________________ > >> libvirt-users mailing list > >> libvirt-users@redhat.com > >> https://www.redhat.com/mailman/listinfo/libvirt-users > > So I updated libvirt to 1.2.12 and qemu to 2.1.3 and it still gives me > > the exact same behaviour ... > > Could it be that the share was created with an old version of > > virt-manager redenring in old/broken XML? From Qemu config example i > > don't see how my XML could be incorrect. > > > > Any idea ? > > I know nothing about the 9p filesystem, but since you have a qemu > commandline that works and a libvirt config that doesn't, the next > logical step would be to provide the non-working qemu commandline > generated by libvirt. You can find this at the end of the log file > /var/log/libvirt/qemu/irc.log.Here it is... A bit more complex :) LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-system-x86_64 -name irc -S -machine pc-i440fx-2.1,accel=kvm,usb=off -m 256 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid ffbd32eb-a693-eadd-8923-18de80137472 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/irc.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -no-acpi -boot strict=on -kernel /srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 -initrd /srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img -append root=virtfs:root rootflags=trans=virtio,version=9p2000.u rw rootfstype=9p -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/srv/overlay/run/irc -device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=root,bus=pci.0,addr=0x5 -netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:00:10:10,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:3 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on
Kashyap Chamarthy
2015-Feb-26 15:05 UTC
Re: [libvirt-users] Broken OS when booting rootfs from 9p share
On Tue, Feb 24, 2015 at 11:56:50PM +0100, Olivier Mauras wrote:> > > On Tue, 2015-02-24 at 16:05 -0500, Laine Stump wrote: > > On 02/24/2015 03:37 PM, Olivier Mauras wrote: > > >> Hello, > > >> > > >> I've been trying to boot a VM with the rootfs being a 9P share from > > >> the host. The VM OS is centos 7. > > >> The OS boots but no services can work and it appears that the > > >> authentication system is broken. > > >> > > >> Now the funny thing is that booting the same OS on the same 9P share > > >> manually with Qemu works as expected with a fully functionnal OS... So > > >> I'm wondering what could libvirt do that render the OS on the share > > >> broken - Something with rights ? > > >> Selinux is disabled on the host, and accessmode to the share is set to > > >> passthrough in both cases. > > >> > > >> Here's my working Qemu line: > > >> qemu > > >> -kernel /srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 \ > > >> > > >> -initrd /srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img \ > > >> -fsdev > > >> local,id=r,path=/srv/overlay/run/irc,security_model=passthrough \ > > >> -device virtio-9p-pci,fsdev=r,mount_tag=root \ > > >> -nographic \ > > >> -m 256M \ > > >> -machine pc-i440fx-2.1,accel=kvm \ > > >> -netdev tap,id=net0 -device > > >> virtio-net,netdev=net0,mac=52:54:00:12:36:34 \ > > >> -smp 1,sockets=1,cores=1 \ > > >> -append 'root=virtfs:root rootflags=trans=virtio,version=9p2000.u ro > > >> rootfstype=9p console=ttyS0' > > >> > > >> And here my libvirt VM config: > > >> <domain type='kvm'> > > >> <name>irc</name> > > >> <uuid>ffbd32eb-a693-eadd-8923-18de80137472</uuid> > > >> <memory unit='KiB'>262144</memory> > > >> <currentMemory unit='KiB'>262144</currentMemory> > > >> <vcpu placement='static'>1</vcpu> > > >> <os> > > >> <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type> > > >> > > >> <kernel>/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64</kernel> > > >> > > >> <initrd>/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img</initrd> > > >> <cmdline>root=virtfs:root rootflags=trans=virtio,version=9p2000.u > > >> ro rootfstype=9p</cmdline> > > >> <boot dev='hd'/> > > >> </os> > > >> <features> > > >> <acpi/> > > >> <apic/> > > >> <pae/> > > >> </features> > > >> <clock offset='utc'/> > > >> <on_poweroff>destroy</on_poweroff> > > >> <on_reboot>restart</on_reboot> > > >> <on_crash>restart</on_crash> > > >> <devices> > > >> <emulator>/usr/bin/qemu-system-x86_64</emulator> > > >> <controller type='usb' index='0'> > > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > > >> function='0x2'/> > > >> </controller> > > >> <controller type='pci' index='0' model='pci-root'/> > > >> <filesystem type='mount' accessmode='passthrough'> > > >> <source dir='/srv/overlay/run/irc'/> > > >> <target dir='root'/> > > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > > >> function='0x0'/> > > >> </filesystem> > > >> <interface type='bridge'> > > >> <mac address='52:54:00:00:36:34'/> > > >> <source bridge='br0'/> > > >> <model type='virtio'/> > > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > > >> function='0x0'/> > > >> </interface> > > >> <serial type='pty'> > > >> <target port='0'/> > > >> </serial> > > >> <console type='pty'> > > >> <target type='serial' port='0'/> > > >> </console> > > >> <input type='mouse' bus='ps2'/> > > >> <input type='keyboard' bus='ps2'/> > > >> <graphics type='vnc' port='-1' autoport='yes'/> > > >> <video> > > >> <model type='cirrus' vram='9216' heads='1'/> > > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > > >> function='0x0'/> > > >> </video> > > >> <memballoon model='virtio'> > > >> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > > >> function='0x0'/> > > >> </memballoon> > > >> </devices> > > >> </domain> > > >> > > >> > > >> My libvirt version is 1.2.9, and qemu is 2.1.2. > > >> > > >> Thanks for your help, > > >> > > >> Olivier > > >> > > >> > > >> _______________________________________________ > > >> libvirt-users mailing list > > >> libvirt-users@redhat.com > > >> https://www.redhat.com/mailman/listinfo/libvirt-users > > > So I updated libvirt to 1.2.12 and qemu to 2.1.3 and it still gives me > > > the exact same behaviour ... > > > Could it be that the share was created with an old version of > > > virt-manager redenring in old/broken XML? From Qemu config example i > > > don't see how my XML could be incorrect. > > > > > > Any idea ? > > > > I know nothing about the 9p filesystem, but since you have a qemu > > commandline that works and a libvirt config that doesn't, the next > > logical step would be to provide the non-working qemu commandline > > generated by libvirt. You can find this at the end of the log file > > /var/log/libvirt/qemu/irc.log. > Here it is... A bit more complex :) > > LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin > QEMU_AUDIO_DRV=none /usr/bin/qemu-system-x86_64 -name irc -S -machine > pc-i440fx-2.1,accel=kvm,usb=off -m 256 -realtime mlock=off -smp > 1,sockets=1,cores=1,threads=1 -uuid ffbd32eb-a693-eadd-8923-18de80137472 > -no-user-config -nodefaults -chardev > socket,id=charmonitor,path=/var/lib/libvirt/qemu/irc.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -no-acpi -boot strict=on -kernel /srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 -initrd /srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img -append root=virtfs:root rootflags=trans=virtio,version=9p2000.u rw rootfstype=9p -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -fsdev local,security_model=passthrough,id=fsdev-fs0,path=/srv/overlay/run/irc -device virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=root,bus=pci.0,addr=0x5 -netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:00:10:10,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:3 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 -msg timestamp=on >Maybe you can try enabling Debug logs/filters in libvirt http://wiki.libvirt.org/page/DebugLogs and see if that helps you narrow down? -- /kashyap