libvirt users - Apr 2019 - Re: [libvirt] Why virsh domxml-to-native qemu-argv changes PCI slot number

Moving to libvirt-users where it belongs.

On Thu, 2019-04-18 at 06:47 +0000, Tal Attaly wrote:
> Hi,
> I have bridge type interface defined with slot 3 -
> <address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
> but after running virsh domxml-to-native qemu-argv I get slot 2-
> -netdev tap,fd=21,id=hostnet0 -device
e1000,netdev=hostnet0,id=net0,mac=f2:ff:ff:ff:ff:07,bus=pci.0,addr=0x2
> Any idea why it's happen and how to keep the slot number?  Libvirt
version is 4.5.0
I can reproduce this, and also confirm that it shouldn't happen :)

qemuDomainAssignAddresses() is supposed to make sure that any address
configured explicitly by the user (or previously by libvirt) is
preserved, and it's doing that correctly when it's called from inside
qemuDomainDefAssignAddresses(); however, for some reason, by the time
we call it again in qemuProcessPrepareDomain() the PCI address for
the network interface is no longer set, and so a fresh one gets
assigned instead.

I'll investigate further and get back to you.
> here is the full domain XML -
>  
> <domain type='kvm'
xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
>   <name>vm1</name>
>   <uuid>dbad303e-fc1b-454e-8861-712bf1766a64</uuid>
>   <memory unit='KiB'>4194304</memory>
>   <currentMemory unit='KiB'>4194304</currentMemory>
>   <vcpu placement='static'>1</vcpu>
>   <os>
>     <type arch='x86_64'
machine='pc-i440fx-2.12'>hvm</type>
>     <boot dev='hd'/>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>   </features>
>   <cpu mode='custom' match='exact'
check='partial'>
>     <model fallback='allow'>SandyBridge-IBRS</model>
>   </cpu>
>   <clock offset='utc'>
>     <timer name='rtc' tickpolicy='catchup'/>
>     <timer name='pit' tickpolicy='delay'/>
>     <timer name='hpet' present='no'/>
>   </clock>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>destroy</on_crash>
>   <pm>
>     <suspend-to-mem enabled='no'/>
>     <suspend-to-disk enabled='no'/>
>   </pm>
>   <devices>
>    
<emulator>/qemu/x86_64-softmmu/qemu-system-x86_64</emulator>
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='qcow2'/>
>       <source file='/var/lib/libvirt/images/vm.img'/>
>       <target dev='hda' bus='ide'/>
>       <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>     </disk>
>     <controller type='usb' index='0'
model='ich9-ehci1'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x04' function='0x7'/>
>     </controller>
>     <controller type='usb' index='0'
model='ich9-uhci1'>
>       <master startport='0'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x04' function='0x0'
multifunction='on'/>
>     </controller>
>     <controller type='usb' index='0'
model='ich9-uhci2'>
>       <master startport='2'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x04' function='0x1'/>
>     </controller>
>     <controller type='usb' index='0'
model='ich9-uhci3'>
>       <master startport='4'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x04' function='0x2'/>
>     </controller>
>     <controller type='ide' index='0'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x1'/>
>     </controller>
>     <controller type='pci' index='0'
model='pci-root'/>
>     <interface type='bridge'>
>       <mac address='f2:ff:ff:ff:ff:07'/>
>       <source bridge='br0:'/>
>       <model type='e1000'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x03' function='0x0'/>
>     </interface>
>     <serial type='tcp'>
>       <source mode='bind' host='10.210.27.119'
service='9999'/>
>       <protocol type='telnet'/>
>       <target type='isa-serial' port='0'>
>         <model name='isa-serial'/>
>       </target>
>     </serial>
>     <console type='tcp'>
>       <source mode='bind' host='10.210.27.119'
service='9999'/>
>       <protocol type='telnet'/>
>       <target type='serial' port='0'/>
>     </console>
>     <input type='mouse' bus='ps2'/>
>     <input type='keyboard' bus='ps2'/>
>     <memballoon model='virtio'>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x05' function='0x0'/>
>     </memballoon>
>   </devices>
>   <qemu:commandline>
>     <qemu:env name='CONFIG_FILE'
value='/opt/cfg/qemu.cfg'/>
>   </qemu:commandline>
> </domain>
-- 
Andrea Bolognani / Red Hat / Virtualization

On Thu, Apr 18, 2019 at 04:23:05PM +0200, Andrea Bolognani
wrote:
> Moving to libvirt-users where it belongs.
> 
> On Thu, 2019-04-18 at 06:47 +0000, Tal Attaly wrote:
> > Hi,
> > I have bridge type interface defined with slot 3 -
> > <address type='pci' domain='0x0000'
bus='0x00' slot='0x03' function='0x0'/>
> > but after running virsh domxml-to-native qemu-argv I get slot 2-
> > -netdev tap,fd=21,id=hostnet0 -device
e1000,netdev=hostnet0,id=net0,mac=f2:ff:ff:ff:ff:07,bus=pci.0,addr=0x2
> > Any idea why it's happen and how to keep the slot number?  Libvirt
version is 4.5.0
> 
> I can reproduce this, and also confirm that it shouldn't happen :)
> 
> qemuDomainAssignAddresses() is supposed to make sure that any address
> configured explicitly by the user (or previously by libvirt) is
> preserved, and it's doing that correctly when it's called from
inside
> qemuDomainDefAssignAddresses(); however, for some reason, by the time
> we call it again in qemuProcessPrepareDomain() the PCI address for
> the network interface is no longer set, and so a fresh one gets
> assigned instead.
> 
> I'll investigate further and get back to you.
It is a bug in qemuConnectDomainXMLToNative. You'll see we replace
the original type=bridge NIC with a type=ethernet, except we fail
to copy the address info & probably other properties too.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Thu, 2019-04-18 at 15:29 +0100, Daniel P. Berrangé
wrote:
> On Thu, Apr 18, 2019 at 04:23:05PM +0200, Andrea Bolognani wrote:
> > qemuDomainAssignAddresses() is supposed to make sure that any address
> > configured explicitly by the user (or previously by libvirt) is
> > preserved, and it's doing that correctly when it's called from
inside
> > qemuDomainDefAssignAddresses(); however, for some reason, by the time
> > we call it again in qemuProcessPrepareDomain() the PCI address for
> > the network interface is no longer set, and so a fresh one gets
> > assigned instead.
> > 
> > I'll investigate further and get back to you.
> 
> It is a bug in qemuConnectDomainXMLToNative. You'll see we replace
> the original type=bridge NIC with a type=ethernet, except we fail
> to copy the address info & probably other properties too.
Yeah, I figured that out after digging a bit more :)

I'm not quite sure how to fix it properly, though. Adding yet more
properties to the list of those we record before calling
virDomainNetDefClear() and restore afterwards would be a stop-gap
measure at best; limiting ourselves to merely changing net->type,
on the other hand, though it seems to work from a quick test, will
likely lead to hitting other corner cases.

Formatting the XML, replacing the type attribute and parsing it
back perhaps? Other, hopefully better, ideas?

-- 
Andrea Bolognani / Red Hat / Virtualization