Xen devel - Apr 2009 - Does XEN have any hot backup/snapshot features or plan?

Hello everybody,
  My name is Zign and I would like to ask a simple question.
  For example, I want to backup my VMs everyday. But I do not
want them to be stopped while the backup. For this reason, I can''t
use a traditional way like:  freeze the VM  -> backup everything ->
restore the VM.
  So is there any solution for this?

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Zhao Yue
   Esaki Laboratory,
   Information Physics & Communication Engineering,
   Graduate School of Information Science and Technology,
   The University of Tokyo
Email: zign@hongo.wide.ad.jp
        zign@zign.cn

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Zing,

 You can use LVM2 as "virtual hard discs" for your virtual machines.
So you
can use the LVM tools to backup, to make snapshots, etc...

 http://www.ibm.com/developerworks/linux/library/l-lvm2/index.html

Regards,
Thiago

2009/4/5 zign <zign@hongo.wide.ad.jp>
> Hello everybody,
>  My name is Zign and I would like to ask a simple question.
>  For example, I want to backup my VMs everyday. But I do not
> want them to be stopped while the backup. For this reason, I can''t
> use a traditional way like:  freeze the VM  -> backup everything ->
> restore the VM.
>  So is there any solution for this?
>
> --
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Zhao Yue
>  Esaki Laboratory,
>  Information Physics & Communication Engineering,
>  Graduate School of Information Science and Technology,
>  The University of Tokyo
> Email: zign@hongo.wide.ad.jp
>       zign@zign.cn
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

It doesn''t have this feature right now, but I''m currently
working on a
memory copy-on-write mechanism for Xen that can be used for taking 
snapshots of VMs (without needing to stop them). Used in combination with a 
disk snapshotting tool (like Parallax) you could create backups of your 
VMs. I''ll be my initial implementation to this list very soon, in fact.

There are other people working on this sort of thing too (Mike Sun, for 
example).


Patrick


zign wrote:
> Hello everybody,
>  My name is Zign and I would like to ask a simple question.
>  For example, I want to backup my VMs everyday. But I do not
> want them to be stopped while the backup. For this reason, I can''t
> use a traditional way like:  freeze the VM  -> backup everything ->
> restore the VM.
>  So is there any solution for this?
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Monday, 06 April 2009 at 08:47, zign wrote:
> Hello everybody,
>  My name is Zign and I would like to ask a simple question.
>  For example, I want to backup my VMs everyday. But I do not
> want them to be stopped while the backup. For this reason, I can''t
> use a traditional way like:  freeze the VM  -> backup everything ->
> restore the VM.
>  So is there any solution for this?
I don''t know of a complete tool, but the parts are all there. xm save
-c creates a checkpoint of a VM without destroying it, and LVM lets
you easily snapshot a disk. You can tie them together using the
external device migration hook in xend -- I think I posted a demo
script to this list many months ago that does this.

I can''t remember whether xm save -c also supports --live in xend, but
if not it''s a trivial change that can reduce the time that the VM is
paused for backup to almost nothing. The checkpoints may be larger in
that case, but they can be folded down to simple images afterwards. I
have a script somewhere around here that does this rewriting, but I
haven''t used it in some time.

You might also think about running your VM on parallax, and saving its
images there too -- parallax can figure out the deltas between series
of images for you automatically.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Mr Brendan Cully,
Thank you for your fast reply!
LVM + checkpoint sounds reasonable, but there is problem:
if the storage and the MEM checkpoint are not saved in the
very same time, I am not sure if this is a "correct" snapshot.
You mentioned something about the migration hook that can tie
the MEM checkpoint and the LVM snapshot together. Can this
migration hook ensure that there is no "time-shift" between the
MEM and storage?

Brendan Cully 写道:
> On Monday, 06 April 2009 at 08:47, zign wrote:
>   
>> Hello everybody,
>>  My name is Zign and I would like to ask a simple question.
>>  For example, I want to backup my VMs everyday. But I do not
>> want them to be stopped while the backup. For this reason, I
can''t
>> use a traditional way like:  freeze the VM  -> backup everything
->
>> restore the VM.
>>  So is there any solution for this?
>>     
>
> I don''t know of a complete tool, but the parts are all there. xm
save
> -c creates a checkpoint of a VM without destroying it, and LVM lets
> you easily snapshot a disk. You can tie them together using the
> external device migration hook in xend -- I think I posted a demo
> script to this list many months ago that does this.
>
> I can''t remember whether xm save -c also supports --live in xend,
but
> if not it''s a trivial change that can reduce the time that the VM
is
> paused for backup to almost nothing. The checkpoints may be larger in
> that case, but they can be folded down to simple images afterwards. I
> have a script somewhere around here that does this rewriting, but I
> haven''t used it in some time.
>
> You might also think about running your VM on parallax, and saving its
> images there too -- parallax can figure out the deltas between series
> of images for you automatically.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>   

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Zhao Yue
  Esaki Laboratory,
  Information Physics & Communication Engineering,
  Graduate School of Information Science and Technology,
  The University of Tokyo
Email: zign@hongo.wide.ad.jp
       zign@zign.cn


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Monday, 06 April 2009 at 09:16, zign wrote:
> Mr Brendan Cully,
> Thank you for your fast reply!
> LVM + checkpoint sounds reasonable, but there is problem:
> if the storage and the MEM checkpoint are not saved in the
> very same time, I am not sure if this is a "correct" snapshot.
> You mentioned something about the migration hook that can tie
> the MEM checkpoint and the LVM snapshot together. Can this
> migration hook ensure that there is no "time-shift" between the
> MEM and storage?
As you say, the memory and disk images must be synchronized, and you
can use the external device migration hook to do it. I posted some
demo (that is, not production!) code to do it here:

http://lists.xensource.com/archives/html/xen-devel/2006-12/msg00685.html

But I haven''t looked at this problem in quite some time.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

You can use this:

http://rogerioferreira.objectis.net/projetos/xen-backup-image

-- 
Rogerio Ferreira
Fone: +55 11 7566-1424
http://rogerioferreira.objectis.net

LPI 302 (Mixed Environment) Specialty
Senior Level Linux Professional (LPIC-3)
Advanced Level Linux Professional (LPIC-2)
Junior Level Linux Professional (LPIC-1)


2009/4/5 zign <zign@hongo.wide.ad.jp>:
> Hello everybody,
>  My name is Zign and I would like to ask a simple question.
>  For example, I want to backup my VMs everyday. But I do not
> want them to be stopped while the backup. For this reason, I can''t
> use a traditional way like:  freeze the VM  -> backup everything ->
> restore the VM.
>  So is there any solution for this?
>
> --
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Zhao Yue
>  Esaki Laboratory,
>  Information Physics & Communication Engineering,
>  Graduate School of Information Science and Technology,
>  The University of Tokyo
> Email: zign@hongo.wide.ad.jp
>       zign@zign.cn
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,
I was thinking of prototyping "hot snapshotting" of blktap backed
vbds.
The idea would be for example : for a domain called MyDomain having
/var/xen/data/mydisk.vmdk attached to hda

xm vbd-snapshot-create MyDomain hda /var/xen/snapshot/snapshot1.vmdk

xend will then synchronise the domain, pause it and signal blktap daemon to
create a disk snapshot : Current block device attached to hda would be
closed and moved to /var/backup/snapshot1.vmdk than recreate mydisk.vmdk as
a cow copy of snapshot1.vmdk then unpause the domain. snapshot1.vmdk could
be backed up safely as it receives no writes. After backup imagine some
thing like :

xm vbd-snapshot-merge MyDomain hda

That would "commit" mydisk.vmdk into snapshot1.vmdk, either with a
paused
domain or (more complex but more confortable) live.

- How interesting this would be for general use ? Interface should of course
be generic enought to support large variety of tap storage backends (cow,
vmdk, vhd ?). Would it be interesting enought to some day get integrated in
mainstream xen ? Or philosophy of lean means that we won''t implement in
xen
something that LVM would do fine ?

- Is there already some work like this ?

- As of xen 3.2 (and hopefully 3.3) i think blktap daemon is the right place
to do it, adding the right callbacks to tap_disk structure and implementing
it in convinent drivers. But i saw plans with xen 3.4 (or after ?) with the
idea of deep rework trying to merge everything in qemu device model? What is
the best place to do this ?



Regards,

Omar BENHAMID


On Mon, Apr 6, 2009 at 1:47 AM, zign <zign@hongo.wide.ad.jp> wrote:
> Hello everybody,
>  My name is Zign and I would like to ask a simple question.
>  For example, I want to backup my VMs everyday. But I do not
> want them to be stopped while the backup. For this reason, I can''t
> use a traditional way like:  freeze the VM  -> backup everything ->
> restore the VM.
>  So is there any solution for this?
>
> --
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Zhao Yue
>  Esaki Laboratory,
>  Information Physics & Communication Engineering,
>  Graduate School of Information Science and Technology,
>  The University of Tokyo
> Email: zign@hongo.wide.ad.jp
>       zign@zign.cn
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
Omar BENHAMID
Email: o.benhamid@bewigo.fr
Tel: +33 6 09 31 60 60
http://www.bewigo.fr


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hot snapshot capabilities are in blktap2 (in xen-unstable) for qcow and 
vhd.  However, there''s no unified snapshot interface.  Each driver has
its
own interface, but you could script something up for one or more disk 
types pretty easily. Check out tools/blktap2/vhd/vhd-util.

--Dutch

On Sun, 14 Jun 2009, Omar BENHAMID wrote:
> Hi,
> I was thinking of prototyping "hot snapshotting" of blktap backed
vbds.
> The idea would be for example : for a domain called MyDomain having
> /var/xen/data/mydisk.vmdk attached to hda
>
> xm vbd-snapshot-create MyDomain hda /var/xen/snapshot/snapshot1.vmdk
>
> xend will then synchronise the domain, pause it and signal blktap daemon to
> create a disk snapshot : Current block device attached to hda would be
> closed and moved to /var/backup/snapshot1.vmdk than recreate mydisk.vmdk as
> a cow copy of snapshot1.vmdk then unpause the domain. snapshot1.vmdk could
> be backed up safely as it receives no writes. After backup imagine some
> thing like :
>
> xm vbd-snapshot-merge MyDomain hda
>
> That would "commit" mydisk.vmdk into snapshot1.vmdk, either with
a paused
> domain or (more complex but more confortable) live.
>
> - How interesting this would be for general use ? Interface should of
course
> be generic enought to support large variety of tap storage backends (cow,
> vmdk, vhd ?). Would it be interesting enought to some day get integrated in
> mainstream xen ? Or philosophy of lean means that we won''t
implement in xen
> something that LVM would do fine ?
>
> - Is there already some work like this ?
>
> - As of xen 3.2 (and hopefully 3.3) i think blktap daemon is the right
place
> to do it, adding the right callbacks to tap_disk structure and implementing
> it in convinent drivers. But i saw plans with xen 3.4 (or after ?) with the
> idea of deep rework trying to merge everything in qemu device model? What
is
> the best place to do this ?
>
>
>
> Regards,
>
> Omar BENHAMID
>
>
> On Mon, Apr 6, 2009 at 1:47 AM, zign <zign@hongo.wide.ad.jp> wrote:
>
>> Hello everybody,
>>  My name is Zign and I would like to ask a simple question.
>>  For example, I want to backup my VMs everyday. But I do not
>> want them to be stopped while the backup. For this reason, I
can''t
>> use a traditional way like:  freeze the VM  -> backup everything
->
>> restore the VM.
>>  So is there any solution for this?
>>
>> --
>> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>> Zhao Yue
>>  Esaki Laboratory,
>>  Information Physics & Communication Engineering,
>>  Graduate School of Information Science and Technology,
>>  The University of Tokyo
>> Email: zign@hongo.wide.ad.jp
>>       zign@zign.cn
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>
>
>
> --
> Omar BENHAMID
> Email: o.benhamid@bewigo.fr
> Tel: +33 6 09 31 60 60
> http://www.bewigo.fr
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel