On 3/21/23 11:05, nospam at godawa.de wrote:> Hi everybody, > > since long time, I'm using Xen on CentOS with XL, currently latest CentOS 7 with > Xen 4.15 from the CentOS-Xen-Project. For several VMs I have to use SR-IOV, to > lower the CPU-usage on Dom0 on the host. > > CentOS 7 comes to an end, Xen is not supported by RHEL nor RockyLinux anymore, > so unfortunately I have to switch to KVM. > > First step will be now, converting all the scripts for managing and running VMs, > that they run with the additional libvirt-layer.What is the libvirt version?> Mostly everything is working, but I do not get a network interface in the VM, > when I start it with "virsh start ..." instead of "xl create ...". > > > First of all, is there documentation how to configure the VMs in the > dom-definition for XEN (all docs I found are KVM-related)?I know it's not complete, but https://libvirt.org/formatdomain.html makes an attempt to identify hypervisor support for the various configuration settings.> The converted xl-config does not do the job: > > virsh -c xen:/// domxml-from-native --format xen-xl vm > vm.xml > > These are some none working examples I tried out: > > ??? ... > ??? <interface type='hostdev' managed='yes'> > ????? <mac address='02:16:32:10:20:30'/> > ????? <driver name='xen'/> > ????? <source> > ??????? <address type='pci' domain='0x0000' bus='0x81' slot='0x02' > function='0x6'/> > ????? </source> > ????? <vlan> > ??????? <tag id='11'/> > ????? </vlan> > ??? </interface> > ??? ... > > > ??? ... > ??? <hostdev mode='subsystem' type='pci' managed='yes'> > ????? <driver name='xen'/> > ????? <source> > ??????? <address domain='0x0000' bus='0x81' slot='0x02' function='0x6'/> > ????? </source> > ????? <address type='pci' domain='0x0000' bus='0x81' slot='0x02' function='0x6'/> > ??? </hostdev> > ??? ... >I've used both configurations successfully. Has the sriov virtual function been unbound from the native driver and bound to xen-pciback? You can check which driver is in use with something like ls /sys/bus/pci/devices/0000\:81\:02.6/driver If bound to xen-pciback, it vf should also appear in the output of 'xl pci-assignable-list'. Regards, Jim
Jim Fehlig schrieb:> > What is the libvirt version?It's the "latest and greatest" I get from this source: [root at xen1 ~]# virsh --version 6.6.0 [root at xen1 ~]# libvirtd --version libvirtd (libvirt) 6.6.0 [root at xengfs1f ~]# yum list | grep libvirt libvirt.x86_64 1:6.6.0-6.xen415.el7 @centos-virt-xen-415 libvirt-bash-completion.x86_64 1:6.6.0-6.xen415.el7 @centos-virt-xen-415 libvirt-client.x86_64 1:6.6.0-6.xen415.el7 @centos-virt-xen-415 libvirt-daemon.x86_64 1:6.6.0-6.xen415.el7 @centos-virt-xen-415> I know it's not complete, but https://libvirt.org/formatdomain.html > makes an attempt to identify hypervisor support for the various > configuration settings.I will "recheck" it, but I don't see anything new, that might be missing.> I've used both configurations successfully. Has the sriov virtual > function been unbound from the native driver and bound to xen-pciback?Should be, otherwise it would not work with XL?> You can check which driver is in use with something like > > ls /sys/bus/pci/devices/0000\:81\:02.6/driverI can't see the exact definition, but again, same output when starting with xl or virsh: [root at xen1 ~]# ls /sys/bus/pci/devices/0000\:81\:02.6/driver 0000:81:02.0 0000:81:02.7 0000:81:03.6 0000:81:04.5 0000:81:05.4 0000:81:06.5 0000:81:07.4 0000:81:08.3 0000:81:09.2 module slots 0000:81:02.1 0000:81:03.0 0000:81:03.7 0000:81:04.6 0000:81:05.5 0000:81:06.6 0000:81:07.5 0000:81:08.4 0000:81:09.3 new_id uevent 0000:81:02.2 0000:81:03.1 0000:81:04.0 0000:81:04.7 0000:81:06.0 0000:81:06.7 0000:81:07.6 0000:81:08.5 0000:81:09.4 new_slot unbind 0000:81:02.3 0000:81:03.2 0000:81:04.1 0000:81:05.0 0000:81:06.1 0000:81:07.0 0000:81:07.7 0000:81:08.6 0000:81:09.5 permissive 0000:81:02.4 0000:81:03.3 0000:81:04.2 0000:81:05.1 0000:81:06.2 0000:81:07.1 0000:81:08.0 0000:81:08.7 bind quirks 0000:81:02.5 0000:81:03.4 0000:81:04.3 0000:81:05.2 0000:81:06.3 0000:81:07.2 0000:81:08.1 0000:81:09.0 irq_handlers remove_id 0000:81:02.6 0000:81:03.5 0000:81:04.4 0000:81:05.3 0000:81:06.4 0000:81:07.3 0000:81:08.2 0000:81:09.1 irq_handler_state remove_slot [root at xen1 ~]# cat /sys/bus/pci/devices/0000\:81\:02.6/device 0x154c [root at xen1 ~]# cat /sys/bus/pci/devices/0000\:81\:02.6/vendor 0x8086 [root at xen1 ~]# cat /sys/bus/pci/devices/0000\:81\:02.6/subsystem_vendor 0x8086 [root at xen1 ~]# ls /sys/bus/pci/devices/0000\:81\:02.6/driver/module/drivers/ pci:pciback xen-backend:xen-pciback [root at xen1 ~]# ls /sys/bus/pci/devices/0000\:81\:02.6/driver/0000:81:02.6 broken_parity_status consistent_dma_mask_bits dma_mask_bits enable local_cpus numa_node reset resource0_wc subsystem uevent class d3cold_allowed driver irq modalias physfn resource resource3 subsystem_device vendor config device driver_override local_cpulist msi_bus power resource0 resource3_wc subsystem_vendor> If bound to xen-pciback, it vf should also appear in the output of 'xl > pci-assignable-list'.Before starting the VM, it's there: [root at xengfs1f ~]# xl pci-assignable-list | grep "0000:81:02.6" 0000:81:02.6 [root at xen1 ~]# [root at xen1 ~]# virsh start vm-test1 Domain vm-test1 gestartet [root at xen1 ~]# xl pci-assignable-list | grep "0000:81:02.6" [root at xen1 ~]# After starting it's "gone", but I assume that it's correct, because it's the same behaviour when starting the with xl create :-) Thanks a lot, -- kind regards, Thorolf