Peter Krempa
2021-Jan-06 15:36 UTC
[ovirt-devel] Issue: Device path changed after adding disks to guest VM
On Wed, Jan 06, 2021 at 17:16:24 +0200, Nir Soffer wrote:> On Wed, Dec 2, 2020 at 4:57 PM Joy Li <joooy.li at gmail.com> wrote:[...]> Comparing to state before reboot: > > # virsh -r domblklist disk-mapping > Target Source > --------------------------------------------------------------------------------------------------------------------------------------------------------------- > sdc - > sda /rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5 > vda /dev/mapper/3600140594af345ed76d42058f2b1a454 > vdb /dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5 > vdc /dev/mapper/36001405b4d0c0b7544d47438b21296ef > > # ls -lh /dev/disk/by-id/virtio-* > lrwxrwxrwx. 1 root root 9 Jan 6 09:42 > /dev/disk/by-id/virtio-b97e68b2-87ea-45ca-9 -> ../../vda > lrwxrwxrwx. 1 root root 9 Jan 6 09:42 > /dev/disk/by-id/virtio-d9a29187-f492-4a0d-a -> ../../vdb > lrwxrwxrwx. 1 root root 9 Jan 6 09:51 > /dev/disk/by-id/virtio-e801c2e4-dc2e-4c53-b -> ../../vdc > > In the guest disks are mapped to the same device name. > > It looks like libivrt domblklist is not correct - vdb and vdc are switched. > Peter, this expected?The names in 'virsh domblklist' are unfortunately and confusingly chosen to match the expected /dev/ device node name, but it's at kernel's discretion to name /dev/ nodes. This means that it's not guaranteed that what you see in 'virsh domblklist' will match the state in the guest. In this case I think the reorder happens as the PCI address of vdb is larger than the address of vdc. A partial workaround can be to use the data provided by the qemu guest agent, for exampe via 'virsh domfsinfo': $ virsh domfsinfo fedora32 Mountpoint Name Type Target ------------------------------------ / dm-0 xfs vda /boot vda1 xfs vda Here the guest-host matching is established via the PCI address so the 'Target' field accurately refers to the target in the VM XML. Similarly the linux kernel recently changed enumeration of SCSI devices so they are not guaranteed to match either. Libguestfs for example needed a workaround. https://github.com/libguestfs/libguestfs/commit/bca9b94fc593771b3801b09b95e477f160517909
Daniel P. Berrangé
2021-Jan-06 15:48 UTC
[ovirt-devel] Issue: Device path changed after adding disks to guest VM
On Wed, Jan 06, 2021 at 04:36:46PM +0100, Peter Krempa wrote:> On Wed, Jan 06, 2021 at 17:16:24 +0200, Nir Soffer wrote: > > On Wed, Dec 2, 2020 at 4:57 PM Joy Li <joooy.li at gmail.com> wrote: > > [...] > > > Comparing to state before reboot: > > > > # virsh -r domblklist disk-mapping > > Target Source > > --------------------------------------------------------------------------------------------------------------------------------------------------------------- > > sdc - > > sda /rhev/data-center/mnt/blockSD/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/images/40018b33-2b11-4d10-82e4-604a5b135fb2/40f455c4-8c92-4f8f-91c2-991b0ddfc2f5 > > vda /dev/mapper/3600140594af345ed76d42058f2b1a454 > > vdb /dev/mapper/360014050058f2f8a0474dc7a8a7cc6a5 > > vdc /dev/mapper/36001405b4d0c0b7544d47438b21296ef > > > > # ls -lh /dev/disk/by-id/virtio-* > > lrwxrwxrwx. 1 root root 9 Jan 6 09:42 > > /dev/disk/by-id/virtio-b97e68b2-87ea-45ca-9 -> ../../vda > > lrwxrwxrwx. 1 root root 9 Jan 6 09:42 > > /dev/disk/by-id/virtio-d9a29187-f492-4a0d-a -> ../../vdb > > lrwxrwxrwx. 1 root root 9 Jan 6 09:51 > > /dev/disk/by-id/virtio-e801c2e4-dc2e-4c53-b -> ../../vdc > > > > In the guest disks are mapped to the same device name. > > > > It looks like libivrt domblklist is not correct - vdb and vdc are switched. > > Peter, this expected? > > The names in 'virsh domblklist' are unfortunately and confusingly chosen > to match the expected /dev/ device node name, but it's at kernel's > discretion to name /dev/ nodes. > > This means that it's not guaranteed that what you see in 'virsh > domblklist' will match the state in the guest.Essentially the only thing the disk device name is used for is sorting the <disk> elements within the XML document. This in turn affects what order PCI addresses (virtio-blk) or SCSI LUNS (virtio-scsi) are assigned in. This influences/hints as to what order the guest OS *might* assign device names in. The device name from the XML is not exposed to the guest directly though. Certainly when hotplugging/unplugging is involved all bets are off wrt what disk names you'll see in the guest vs the XML. Dont expect them to match except by luck. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|