libvirt users - Feb 2014 - Re: Can I move the disk image of the guest while it is running?

Thank you Eric,

On 2014-02-05 17:23, Eric Blake wrote:
> Yes, live storage migration is possible; although at the moment, qemu is
> lacking a way to restart the operation if it fails midstream, so libvirt
> only allows the operation if you are willing to temporarily make your
> guest transient.
What does this mean? Will I loose anything if - for example - there is
not enough space on the target device? Or it will still use the original
disk image?

AFAIK, a transient guest only means it will disappear after the
virtualisation session ends.
> virsh dumpxml $dom > file
> virsh undefine $dom
> virsh blockcopy $dom /ssd/image.raw /hdd/image.raw \
>   --wait --verbose --pivot
> virsh define file
I could not find anything about "pivot" or "pivoting"? What
does --pivot
do in this case?


Thank you.

--
Üdvözlettel / Best regards

Horváth Gergely | gergely.horvath@inepex.com

IneTrack - Nyomkövetés egyszerűen | Inepex Kft.
Ügyfélszolgálat: support@inetrack.hu | +36 30 825 7646 | support.inetrack.hu
Web: www.inetrack.hu | nyomkovetes-blog.hu | facebook.com/inetrack

Inepex - The White Label GPS fleet-tracking platform | www.inepex.com

On 02/05/2014 02:54 PM, Gergely Horváth wrote:
> Thank you Eric,
> 
> On 2014-02-05 17:23, Eric Blake wrote:
>> Yes, live storage migration is possible; although at the moment, qemu
is
>> lacking a way to restart the operation if it fails midstream, so
libvirt
>> only allows the operation if you are willing to temporarily make your
>> guest transient.
> 
> What does this mean? Will I loose anything if - for example - there is
> not enough space on the target device? Or it will still use the original
> disk image?
If blockcopy fails before mirroring is in sync, such as for insufficient
space on the destination, then the source is still valid, with no data
loss.  If it reaches sync, then both source and destination are copies
of each other until you quit the operation (the --wait flag tells virsh
to automatically wait for sync, and the --pivot flag says to quit the
operation as soon as sync is reached, rather than doing an indefinite
mirror).  As long as the guest doesn't quit in the meantime, you haven't
lost anything.  If the guest powers itself off in the middle of the
blockcopy, the blockcopy will fail, but the original is still intact at
the state it was when the guest shut down.
> 
> AFAIK, a transient guest only means it will disappear after the
> virtualisation session ends.
Correct, which is why you can then re-define the guest with the XML you
saved earlier, so that it is once again persistent and can be safely
powered off.  If blockcopy --pivot succeeded, then the guest uses only
the destination, and you have successfully gotten rid of the need to
refer to the source.
> 
>> virsh dumpxml $dom > file
>> virsh undefine $dom
>> virsh blockcopy $dom /ssd/image.raw /hdd/image.raw \
>>   --wait --verbose --pivot
>> virsh define file
Oh, I just realized a caveat - before redefining the file, you'll need
to edit it to reflect a successful blockcopy (otherwise, if the guest
stops and then is rebooted, the reboot will still use the original
source, rather than the destination); a safer set of steps might be:

virsh dumpxml $dom > file
virsh undefine $dom
if virsh blockcopy $dom /ssd/image.raw /hdd/image.raw \
   --wait --verbose --pivot; then
  virsh dumpxml $dom > file # refresh the xml to reflect new disk source
fi
virsh define file

so that file restores you to the original disk location if blockcopy
fails, but tracks the new location if blockcopy succeeds.

[In my development work, I've been testing on throwaway disks, where I
don't care if I clobber things by starting over again - but in a
production environment, it pays to be more careful, and possibly to
experiment on a safe domain with throwaway disk before doing it on your
real domain]
> 
> I could not find anything about "pivot" or "pivoting"?
What does --pivot
> do in this case?
--pivot says to break mirroring by using just the destination (in other
words, pivot away from the source).  The alternative is to use --finish,
which says break mirroring by using just the source (in other words, the
destination is now a point-in-time snapshot at the time you broke
mirroring - useful for backup purposes).

And if you are terrified of the possible consequences of a transient
domain, there is another possibility for moving your data to a new disk,
except that it forces you to convert to qcow2:

virsh snapshot-create-as $dom --no-metadata --disk-only \
  --diskspec /ssd/image.raw,file=/hdd/image.qcow2
virsh blockpull $dom --wait --verbose

and also has the drawback that if the guest quits in the middle of the
blockpull, then you have to track both the original file and the
snapshot wrapper - but at least blockpull is restartable from where it
left off, for the next time the guest is running, whereas blockcopy has
to start from scratch.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

On Wed, Feb 05, 2014 at 10:54:15PM +0100, Gergely Horváth
wrote:
> Thank you Eric,
> 
> On 2014-02-05 17:23, Eric Blake wrote:
> > Yes, live storage migration is possible; although at the moment, qemu
is
> > lacking a way to restart the operation if it fails midstream, so
libvirt
> > only allows the operation if you are willing to temporarily make your
> > guest transient.
> 
> What does this mean? Will I loose anything if - for example - there is
> not enough space on the target device?
As I understand it, no, you'll not lose anything.

Here's some old notes for live backup with 'virsh blockcopy', this
was
after discussing with Eric

  http://kashyapc.fedorapeople.org/virt/lc-2012/live-backup-with-blockcopy.txt

  Currently due to QEMU's limitation, libvirt expects a domain to be
  trasient. Because, blockcopy jobs last forever until canceled, which
  implies that they should last across domain restarts if the domain
  were persistent. But qemu doesn't yet provide a way to restart a copy
  job on domain restart. So the trick is to temporarily make the domain
  transient

> Or it will still use the original
> disk image?
> 
> AFAIK, a transient guest only means it will disappear after the
> virtualisation session ends.
The disk image stays intact, just it'll disappear from libvirt's view,
hence you take a copy of your XML file below ('virsh dumpxml') does that
> 
> > virsh dumpxml $dom > file
> > virsh undefine $dom
> > virsh blockcopy $dom /ssd/image.raw /hdd/image.raw \
> >   --wait --verbose --pivot
> > virsh define file
> 
> I could not find anything about "pivot" or "pivoting"?
What does --pivot
> do in this case?
When you "pivot", libvirt will use the new copy (i.e. /hdd/image.raw
in this case), and this is done while the guest is running.

Eric, please correct if I said something wrong.

-- 
/kashyap

On Thu, Feb 06, 2014 at 12:02:11PM +0530, Kashyap Chamarthy
wrote:
> On Wed, Feb 05, 2014 at 10:54:15PM +0100, Gergely Horváth wrote:
> > Thank you Eric,
> > 
> > On 2014-02-05 17:23, Eric Blake wrote:
> > > Yes, live storage migration is possible; although at the moment,
qemu is
> > > lacking a way to restart the operation if it fails midstream, so
libvirt
> > > only allows the operation if you are willing to temporarily make
your
> > > guest transient.
> > 
> > What does this mean? Will I loose anything if - for example - there is
> > not enough space on the target device?
> 
> As I understand it, no, you'll not lose anything.
> 
> Here's some old notes for live backup with 'virsh blockcopy',
this was
> after discussing with Eric
> 
>  
http://kashyapc.fedorapeople.org/virt/lc-2012/live-backup-with-blockcopy.txt
> 
>   Currently due to QEMU's limitation, libvirt expects a domain to be
>   trasient. Because, blockcopy jobs last forever until canceled, which
>   implies that they should last across domain restarts if the domain
>   were persistent. But qemu doesn't yet provide a way to restart a copy
>   job on domain restart. So the trick is to temporarily make the domain
>   transient
> 
> 
> > Or it will still use the original
> > disk image?
> > 
> > AFAIK, a transient guest only means it will disappear after the
> > virtualisation session ends.
> 
> The disk image stays intact, just it'll disappear from libvirt's
view,
> hence you take a copy of your XML file below ('virsh dumpxml') does
that
> 
> > 
> > > virsh dumpxml $dom > file
> > > virsh undefine $dom
> > > virsh blockcopy $dom /ssd/image.raw /hdd/image.raw \
> > >   --wait --verbose --pivot
> > > virsh define file
> > 
> > I could not find anything about "pivot" or
"pivoting"? What does --pivot
> > do in this case?
> 
> When you "pivot", libvirt will use the new copy (i.e.
/hdd/image.raw
> in this case), and this is done while the guest is running.
> 
> Eric, please correct if I said something wrong.
Ah, Eric already explained it more eloquently (I hit send before seeing
his email).

-- 
/kashyap

----- Original Message -----
> From: "Eric Blake" <eblake@redhat.com>
> To: "Gergely Horváth" <gergely.horvath@inepex.com>,
libvirt-users@redhat.com
> Sent: Wednesday, February 5, 2014 4:47:47 PM
> Subject: Re: [libvirt-users] Can I move the disk image of the guest while
it is running?
> 
> On 02/05/2014 02:54 PM, Gergely Horváth wrote:
> > Thank you Eric,
> > 
> > On 2014-02-05 17:23, Eric Blake wrote:
> >> Yes, live storage migration is possible; although at the moment,
qemu is
> >> lacking a way to restart the operation if it fails midstream, so
libvirt
> >> only allows the operation if you are willing to temporarily make
your
> >> guest transient.
> > 
> > What does this mean? Will I loose anything if - for example - there is
> > not enough space on the target device? Or it will still use the
original
> > disk image?
> 
> If blockcopy fails before mirroring is in sync, such as for insufficient
> space on the destination, then the source is still valid, with no data
> loss.  If it reaches sync, then both source and destination are copies
> of each other until you quit the operation (the --wait flag tells virsh
> to automatically wait for sync, and the --pivot flag says to quit the
> operation as soon as sync is reached, rather than doing an indefinite
> mirror).  As long as the guest doesn't quit in the meantime, you
haven't
> lost anything.  If the guest powers itself off in the middle of the
> blockcopy, the blockcopy will fail, but the original is still intact at
> the state it was when the guest shut down.
> 
> > 
> > AFAIK, a transient guest only means it will disappear after the
> > virtualisation session ends.
> 
> Correct, which is why you can then re-define the guest with the XML you
> saved earlier, so that it is once again persistent and can be safely
> powered off.  If blockcopy --pivot succeeded, then the guest uses only
> the destination, and you have successfully gotten rid of the need to
> refer to the source.
> 
> > 
> >> virsh dumpxml $dom > file
> >> virsh undefine $dom
> >> virsh blockcopy $dom /ssd/image.raw /hdd/image.raw \
> >>   --wait --verbose --pivot
> >> virsh define file
> 
> Oh, I just realized a caveat - before redefining the file, you'll need
> to edit it to reflect a successful blockcopy (otherwise, if the guest
> stops and then is rebooted, the reboot will still use the original
> source, rather than the destination); a safer set of steps might be:
> 
> virsh dumpxml $dom > file
> virsh undefine $dom
> if virsh blockcopy $dom /ssd/image.raw /hdd/image.raw \
>    --wait --verbose --pivot; then
>   virsh dumpxml $dom > file # refresh the xml to reflect new disk source
> fi
> virsh define file
> 
> so that file restores you to the original disk location if blockcopy
> fails, but tracks the new location if blockcopy succeeds.
> 
> [In my development work, I've been testing on throwaway disks, where I
> don't care if I clobber things by starting over again - but in a
> production environment, it pays to be more careful, and possibly to
> experiment on a safe domain with throwaway disk before doing it on your
> real domain]
> 
> > 
> > I could not find anything about "pivot" or
"pivoting"? What does --pivot
> > do in this case?
> 
> --pivot says to break mirroring by using just the destination (in other
> words, pivot away from the source).  The alternative is to use --finish,
> which says break mirroring by using just the source (in other words, the
> destination is now a point-in-time snapshot at the time you broke
> mirroring - useful for backup purposes).
> 
> And if you are terrified of the possible consequences of a transient
> domain, there is another possibility for moving your data to a new disk,
> except that it forces you to convert to qcow2:
> 
> virsh snapshot-create-as $dom --no-metadata --disk-only \
>   --diskspec /ssd/image.raw,file=/hdd/image.qcow2
> virsh blockpull $dom --wait --verbose
> 
> and also has the drawback that if the guest quits in the middle of the
> blockpull, then you have to track both the original file and the
> snapshot wrapper - but at least blockpull is restartable from where it
> left off, for the next time the guest is running, whereas blockcopy has
> to start from scratch.
Eric,

For scripted live backups, which non-transient technique would you prefer:
1. snapshot-create-as followed by blockpull, which results in the original 
file being left as a standalone copy of the VM from that point in time; it
can then be copied off to a backup disk once blockpull completes
2. snapshot-create-as followed by copying the backing file (since it is now
read-only) and then using blockcommit to pull the interim changes back into
the original/backing file (and removing the snapshot file)

For #2 I suppose you could hash the backing file and the copy once it 
finishes to verify integrity of the backup copy.

Thanks,

Andrew