libvirt users - Feb 2015 - Re: Broken OS when booting rootfs from 9p share

Hello,

I've been trying to boot a VM with the rootfs being a 9P
share from the host. The VM OS is centos 7.
The OS boots but no services
can work and it appears that the authentication system is broken.

Now
the funny thing is that booting the same OS on the same 9P share
manually with Qemu works as expected with a fully functionnal OS... So
I'm wondering what could libvirt do that render the OS on the share
broken - Something with rights ?
Selinux is disabled on the host, and
accessmode to the share is set to passthrough in both cases.

Here's my
working Qemu line:
qemu -kernel
/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 

-initrd
/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img

 -fsdev
local,id=r,path=/srv/overlay/run/irc,security_model=passthrough 

-device virtio-9p-pci,fsdev=r,mount_tag=root 
 -nographic 
 -m 256M 

-machine pc-i440fx-2.1,accel=kvm 
 -netdev tap,id=net0 -device
virtio-net,netdev=net0,mac=52:54:00:12:36:34 
 -smp 1,sockets=1,cores=1

 -append 'root=virtfs:root rootflags=trans=virtio,version=9p2000.u ro
rootfstype=9p console=ttyS0'

And here my libvirt VM config:
<domain
type='kvm'>
 <name>irc</name>

<uuid>ffbd32eb-a693-eadd-8923-18de80137472</uuid>
 <memory
unit='KiB'>262144</memory>
 <currentMemory
unit='KiB'>262144</currentMemory>
 <vcpu placement='static'>1</vcpu>

<os>
 <type arch='x86_64'
machine='pc-i440fx-2.1'>hvm</type>

<kernel>/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64</kernel>

<initrd>/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img</initrd>

<cmdline>root=virtfs:root rootflags=trans=virtio,version=9p2000.u ro
rootfstype=9p</cmdline>
 <boot dev='hd'/>
 </os>
 <features>
 <acpi/>

<apic/>
 <pae/>
 </features>
 <clock offset='utc'/>

<on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>

<on_crash>restart</on_crash>
 <devices>

<emulator>/usr/bin/qemu-system-x86_64</emulator>
 <controller type='usb'
index='0'>
 <address type='pci' domain='0x0000' bus='0x00'
slot='0x01'
function='0x2'/>
 </controller>
 <controller type='pci' index='0'
model='pci-root'/>
 <filesystem type='mount' accessmode='passthrough'>

<source dir='/srv/overlay/run/irc'/>
 <target dir='root'/>
 <address
type='pci' domain='0x0000' bus='0x00'
slot='0x05' function='0x0'/>

</filesystem>
 <interface type='bridge'>
 <mac
address='52:54:00:00:36:34'/>
 <source bridge='br0'/>
 <model
type='virtio'/>
 <address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
 </interface>
 <serial type='pty'>
 <target
port='0'/>
 </serial>
 <console type='pty'>
 <target type='serial'
port='0'/>
 </console>
 <input type='mouse' bus='ps2'/>
 <input
type='keyboard' bus='ps2'/>
 <graphics type='vnc' port='-1'
autoport='yes'/>
 <video>
 <model type='cirrus' vram='9216' heads='1'/>

<address type='pci' domain='0x0000' bus='0x00'
slot='0x02'
function='0x0'/>
 </video>
 <memballoon model='virtio'>
 <address
type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>

</memballoon>
 </devices>
</domain>

My libvirt version is 1.2.9, and
qemu is 2.1.2.

Thanks for your help,

Olivier

> 
> Hello,
> 
> I've been trying to boot a VM with the rootfs being a 9P share from
> the host. The VM OS is centos 7.
> The OS boots but no services can work and it appears that the
> authentication system is broken.
> 
> Now the funny thing is that booting the same OS on the same 9P share
> manually with Qemu works as expected with a fully functionnal OS... So
> I'm wondering what could libvirt do that render the OS on the share
> broken - Something with rights ?
> Selinux is disabled on the host, and accessmode to the share is set to
> passthrough in both cases.
> 
> Here's my working Qemu line:
> qemu
> -kernel
/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 \
> 
> -initrd
/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img \
>   -fsdev
> local,id=r,path=/srv/overlay/run/irc,security_model=passthrough \
>   -device virtio-9p-pci,fsdev=r,mount_tag=root \
>   -nographic \
>   -m 256M \
>   -machine pc-i440fx-2.1,accel=kvm \
>   -netdev tap,id=net0 -device
> virtio-net,netdev=net0,mac=52:54:00:12:36:34 \
>   -smp 1,sockets=1,cores=1 \
>   -append 'root=virtfs:root rootflags=trans=virtio,version=9p2000.u ro
> rootfstype=9p console=ttyS0'
> 
> And here my libvirt VM config:
> <domain type='kvm'>
>   <name>irc</name>
>   <uuid>ffbd32eb-a693-eadd-8923-18de80137472</uuid>
>   <memory unit='KiB'>262144</memory>
>   <currentMemory unit='KiB'>262144</currentMemory>
>   <vcpu placement='static'>1</vcpu>
>   <os>
>     <type arch='x86_64'
machine='pc-i440fx-2.1'>hvm</type>
> 
>
<kernel>/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64</kernel>
> 
>
<initrd>/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img</initrd>
>     <cmdline>root=virtfs:root rootflags=trans=virtio,version=9p2000.u
> ro rootfstype=9p</cmdline>
>     <boot dev='hd'/>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>   </features>
>   <clock offset='utc'/>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>restart</on_crash>
>   <devices>
>     <emulator>/usr/bin/qemu-system-x86_64</emulator>
>     <controller type='usb' index='0'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x01'
> function='0x2'/>
>     </controller>
>     <controller type='pci' index='0'
model='pci-root'/>
>     <filesystem type='mount'
accessmode='passthrough'>
>       <source dir='/srv/overlay/run/irc'/>
>       <target dir='root'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x05'
> function='0x0'/>
>     </filesystem>
>     <interface type='bridge'>
>       <mac address='52:54:00:00:36:34'/>
>       <source bridge='br0'/>
>       <model type='virtio'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
>     <serial type='pty'>
>       <target port='0'/>
>     </serial>
>     <console type='pty'>
>       <target type='serial' port='0'/>
>     </console>
>     <input type='mouse' bus='ps2'/>
>     <input type='keyboard' bus='ps2'/>
>     <graphics type='vnc' port='-1'
autoport='yes'/>
>     <video>
>       <model type='cirrus' vram='9216'
heads='1'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
>     <memballoon model='virtio'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x04'
> function='0x0'/>
>     </memballoon>
>   </devices>
> </domain>
> 
> 
> My libvirt version is 1.2.9, and qemu is 2.1.2.
> 
> Thanks for your help,
> 
> Olivier
> 
>  
> _______________________________________________
> libvirt-users mailing list
> libvirt-users@redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
So I updated libvirt to 1.2.12 and qemu to 2.1.3 and it still gives me
the exact same behaviour ... 
Could it be that the share was created with an old version of
virt-manager redenring in old/broken XML?  From Qemu config example i
don't see how my XML could be incorrect.

Any idea ?

Thanks,
Olivier

On 02/24/2015 03:37 PM, Olivier Mauras wrote:
>> Hello,
>>
>> I've been trying to boot a VM with the rootfs being a 9P share from
>> the host. The VM OS is centos 7.
>> The OS boots but no services can work and it appears that the
>> authentication system is broken.
>>
>> Now the funny thing is that booting the same OS on the same 9P share
>> manually with Qemu works as expected with a fully functionnal OS... So
>> I'm wondering what could libvirt do that render the OS on the share
>> broken - Something with rights ?
>> Selinux is disabled on the host, and accessmode to the share is set to
>> passthrough in both cases.
>>
>> Here's my working Qemu line:
>> qemu
>> -kernel
/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64 \
>>
>> -initrd
/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img \
>>   -fsdev
>> local,id=r,path=/srv/overlay/run/irc,security_model=passthrough \
>>   -device virtio-9p-pci,fsdev=r,mount_tag=root \
>>   -nographic \
>>   -m 256M \
>>   -machine pc-i440fx-2.1,accel=kvm \
>>   -netdev tap,id=net0 -device
>> virtio-net,netdev=net0,mac=52:54:00:12:36:34 \
>>   -smp 1,sockets=1,cores=1 \
>>   -append 'root=virtfs:root rootflags=trans=virtio,version=9p2000.u
ro
>> rootfstype=9p console=ttyS0'
>>
>> And here my libvirt VM config:
>> <domain type='kvm'>
>>   <name>irc</name>
>>   <uuid>ffbd32eb-a693-eadd-8923-18de80137472</uuid>
>>   <memory unit='KiB'>262144</memory>
>>   <currentMemory unit='KiB'>262144</currentMemory>
>>   <vcpu placement='static'>1</vcpu>
>>   <os>
>>     <type arch='x86_64'
machine='pc-i440fx-2.1'>hvm</type>
>>
>>
<kernel>/srv/overlay/kernels/grsec-3.14.33-101/vmlinuz-3.14.33-101.el6.x86_64</kernel>
>>
>>
<initrd>/srv/overlay/kernels/grsec-3.14.33-101/initramfs-3.14.33-101.el6.x86_64.img</initrd>
>>     <cmdline>root=virtfs:root
rootflags=trans=virtio,version=9p2000.u
>> ro rootfstype=9p</cmdline>
>>     <boot dev='hd'/>
>>   </os>
>>   <features>
>>     <acpi/>
>>     <apic/>
>>     <pae/>
>>   </features>
>>   <clock offset='utc'/>
>>   <on_poweroff>destroy</on_poweroff>
>>   <on_reboot>restart</on_reboot>
>>   <on_crash>restart</on_crash>
>>   <devices>
>>     <emulator>/usr/bin/qemu-system-x86_64</emulator>
>>     <controller type='usb' index='0'>
>>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x01'
>> function='0x2'/>
>>     </controller>
>>     <controller type='pci' index='0'
model='pci-root'/>
>>     <filesystem type='mount'
accessmode='passthrough'>
>>       <source dir='/srv/overlay/run/irc'/>
>>       <target dir='root'/>
>>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x05'
>> function='0x0'/>
>>     </filesystem>
>>     <interface type='bridge'>
>>       <mac address='52:54:00:00:36:34'/>
>>       <source bridge='br0'/>
>>       <model type='virtio'/>
>>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x03'
>> function='0x0'/>
>>     </interface>
>>     <serial type='pty'>
>>       <target port='0'/>
>>     </serial>
>>     <console type='pty'>
>>       <target type='serial' port='0'/>
>>     </console>
>>     <input type='mouse' bus='ps2'/>
>>     <input type='keyboard' bus='ps2'/>
>>     <graphics type='vnc' port='-1'
autoport='yes'/>
>>     <video>
>>       <model type='cirrus' vram='9216'
heads='1'/>
>>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x02'
>> function='0x0'/>
>>     </video>
>>     <memballoon model='virtio'>
>>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x04'
>> function='0x0'/>
>>     </memballoon>
>>   </devices>
>> </domain>
>>
>>
>> My libvirt version is 1.2.9, and qemu is 2.1.2.
>>
>> Thanks for your help,
>>
>> Olivier
>>
>>  
>> _______________________________________________
>> libvirt-users mailing list
>> libvirt-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/libvirt-users
> So I updated libvirt to 1.2.12 and qemu to 2.1.3 and it still gives me
> the exact same behaviour ... 
> Could it be that the share was created with an old version of
> virt-manager redenring in old/broken XML?  From Qemu config example i
> don't see how my XML could be incorrect.
>
> Any idea ?
I know nothing about the 9p filesystem, but since you have a qemu
commandline that works and a libvirt config that doesn't, the next
logical step would be to provide the non-working qemu commandline
generated by libvirt. You can find this at the end of the log file
/var/log/libvirt/qemu/irc.log.