libvirt users - Oct 2014 - Re: KVM incremental backup using CBT

On Fri, Oct 10, 2014 at 07:32:06PM -0600, Eric Blake
wrote:
> On 10/10/2014 11:37 AM, Jd wrote:
> > Hi
> >     Looking in to implementing (CBT like) delta backup for KVM.
> 
> Not quite sure what you mean by CBT.
> 
> > 
> >     The following looks promising..(last paragraph)
> >      http://wiki.qemu.org/Features/Snapshots2
> > 
> 
> Libvirt hasn't yet been patched to take advantage of all the latest
qemu
> features.  Patches are welcome.  But libvirt already knows how to create
> external snapshots and do blockpull (backing files pulled into the
> active image so the backing isn't needed any more) and commit (active
> files pushed into the backing files), and even though it is not as
> full-featured as where we'd like to get, it can already do quite a bit.
To expand an example of what Eric wrote here about external snapshots,
some commands to try:

1. Check the current acitve disk image in use:

    $ virsh domblklist vm1


2. Create an external live snapshot:

    $ virsh snapshot-create-as --domain vm1 sn1 \
        --diskspec vda,file=/export/images/sn1.qcow2 \
        --disk-only --atomic --no-metadata

    Note (thanks Eric Blake): Above, if you have QEMU guest agent
    installed in your virtual machine, try --quiesce option with
    'snapshot-create-as' to ensure you have a consistent disk state.

    Now, you have a disk image chain:
    
        base <-sn1 (live QEMU)


3. Take a backup of the original disk in backround:

    $ rsync -avh --progress /export/images/base.img \
        /export/images/copy.img


4. Commit the sn1 contents back into base:

    $ virsh blockcommit vm1 vda --active --verbose --pivot

    Now, the chain is: 
    
        base (live QEMU) 

    with contents from sn1 live committed into base.


5.  Check the current acitve disk image in use again:

    $ virsh domblklist vm1


-- 
/kashyap

Thanks Kashyap,
The command line examples makes thing quite clear :)
rsync for image (large) files create a new file for every little 
change,  that was the reason I started looking in to using dirty bitmaps.

/Jd

On 10/11/14, 1:13 AM, Kashyap Chamarthy wrote:
> On Fri, Oct 10, 2014 at 07:32:06PM -0600, Eric Blake wrote:
>> On 10/10/2014 11:37 AM, Jd wrote:
>>> Hi
>>>      Looking in to implementing (CBT like) delta backup for KVM.
>> Not quite sure what you mean by CBT.
>>
>>>      The following looks promising..(last paragraph)
>>>       http://wiki.qemu.org/Features/Snapshots2
>>>
>> Libvirt hasn't yet been patched to take advantage of all the latest
qemu
>> features.  Patches are welcome.  But libvirt already knows how to
create
>> external snapshots and do blockpull (backing files pulled into the
>> active image so the backing isn't needed any more) and commit
(active
>> files pushed into the backing files), and even though it is not as
>> full-featured as where we'd like to get, it can already do quite a
bit.
> To expand an example of what Eric wrote here about external snapshots,
> some commands to try:
>
> 1. Check the current acitve disk image in use:
>
>      $ virsh domblklist vm1
>
>
> 2. Create an external live snapshot:
>
>      $ virsh snapshot-create-as --domain vm1 sn1 \
>          --diskspec vda,file=/export/images/sn1.qcow2 \
>          --disk-only --atomic --no-metadata
>
>      Note (thanks Eric Blake): Above, if you have QEMU guest agent
>      installed in your virtual machine, try --quiesce option with
>      'snapshot-create-as' to ensure you have a consistent disk
state.
>
>      Now, you have a disk image chain:
>      
>          base <-sn1 (live QEMU)
>
>
> 3. Take a backup of the original disk in backround:
>
>      $ rsync -avh --progress /export/images/base.img \
>          /export/images/copy.img
>
>
> 4. Commit the sn1 contents back into base:
>
>      $ virsh blockcommit vm1 vda --active --verbose --pivot
>
>      Now, the chain is:
>      
>          base (live QEMU)
>
>      with contents from sn1 live committed into base.
>
>
> 5.  Check the current acitve disk image in use again:
>
>      $ virsh domblklist vm1
>
>

Hello.
> blockcommit vm1 vda --active --verbose --pivot
That's where libvirt stops working at the moment, right?

virsh # blockcommit puppet-test vda --active --verbose --pivot
error: unsupported flags (0x4) in function qemuDomainBlockCommit

cheers
t.

On 10/12/2014 10:19 PM, Jd wrote:

[please don't top-post on technical lists]
> Thanks Kashyap,
> The command line examples makes thing quite clear :)
> rsync for image (large) files create a new file for every little
> change,  that was the reason I started looking in to using dirty bitmaps.
Upstream qemu still doesn't have persistent dirty bitmap tracking
working yet; once patches land there, then we also need to get libvirt
to interact with those new qemu features.  But yes, dirty bitmap
tracking, so that you can track which portions of a file have changed
rather than copying the entire file, is something that is technically
feasible once the patches materialize.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

On Saturday 11 October 2014 10:13:14 Kashyap Chamarthy wrote:
> 2. Create an external live snapshot:
> 
>     $ virsh snapshot-create-as --domain vm1 sn1 \
>         --diskspec vda,file=/export/images/sn1.qcow2 \
>         --disk-only --atomic --no-metadata
> 
>     Note (thanks Eric Blake): Above, if you have QEMU guest agent
>     installed in your virtual machine, try --quiesce option with
>     'snapshot-create-as' to ensure you have a consistent disk
state.
About the --quiesce option. Do i need to do something inside the vm? The most 
commonly would probably be a sql server running inside the vm. Do i need to 
tell the sql server something about the --quiesce option i use? I read this 
article here which suggests such a procedure. Okay, it's vmware, but... Is 
that right?

http://www.virtuallifestyle.nl/2013/03/back-up-mysql-on-linux-without-stopping-services-or-dumping-the-database/

thanks and cheers
t.

On 10/29/2014 01:07 PM, Thomas Stein wrote:
> About the --quiesce option. Do i need to do something inside the vm? The
most
> commonly would probably be a sql server running inside the vm. Do i need to
> tell the sql server something about the --quiesce option i use? I read this
> article here which suggests such a procedure. Okay, it's vmware, but...
Is
> that right?
For --quiesce to work, you have to have qemu-guest-agent installed and
running in your guest, and also inform libvirt via the domain XML to
establish a channel to the guest agent.  Furthermore, if you want your
sql database to be at a stable point, you can install freeze hooks in
your guest that get invoked prior to freezing the file system and just
after thawing it (that is, qemu-guest-agent already has documented hooks
that let you do any additional prep work beyond just freezing the file
system).

A quick google search found this:
http://callecalle.uach.cl/ovirt-engine/docs/manual/en_US/html/Technical_Guide/QEMU_Guest_Agent_Overview.html

Sadly, it didn't spell out the name of where you install hook scripts
into the guest.  But this commit is pretty telling:
https://github.com/qemu/qemu/blob/master/scripts/qemu-guest-agent/fsfreeze-hook

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org