On Thu, Mar 11, 2021 at 10:51:13 +0200, Liran Rotenberg
wrote:> We recently had this bug[1]. The thought that came from it is the handling
> of error code after running virDomainSnapshotCreateXML, we encountered
> VIR_ERR_OPERATION_ABORTED(78).
VIR_ERR_OPERATION_ABORTED is an error code which is emitted by the
migration code only. That means that the error comes from the failure to
take a memory image/snapshot of the VM.
Quick skim through the bugreport seems to mention timeout, so your code
probably aborted the snapshot if it was taking too long.
> Apparently, the new volume is in use. Are there cases where this will
> happen and the new volume won't appear in the volumes chain? Can we
detect
> / know when?
In the vast majority of cases if virDomainSnapshotCreateXML returns
failure the new disk volumes are NOT used at that point.
Libvirt tries very hard to ensure that everything is atomic. The memory
snapshot is taken before installing volumes into the backing chain, so
if that one fails we don't even attempt to do anything with the disks.
There are three extremely unlikely reasons where the snapshot API returns
failure and new images were already installed into the backing chain:
1) resuming of the VM failed after snapshot
2) thawing (domfsthaw) of filesystems has failed
(easily avoided by not using the _QUIESCE flag, but freezing
manually)
3) saving of the internal VM state XML failed
Any error except those above can happen only if the images werent
installed or the VM died while installing the images.
In addition if resuming the cpus after the snapshot fails, the cpus
didn't run so the guest couldn't have written anything to the image.
Since snapshot is supposed to flush qemu caches, in case you destroy the
VM without running the vcpus it's safe to discard the overlays as guest
didn't write anything into them yet.
> Thinking aloud, if we can detect such cases we can prevent rolling back by
> reporting it back from VDSM to ovirt. Or, if it can't be detected to go
on
> the safe side in order to save data corruption and prevent the rollback as
> well.
In general, except for the case when saving of the guest XML has failed,
the new disk images will not be used by the VM so it's safe to delete
them.
> Currently, in ovirt, if the job is aborted, we will look into the chain to
> decide whether to rollback or not.
This is okay, we update the XML only if qemu successfully installed the
overlays.