Xen users - Oct 2006 - FAQ: How to backup for recovery?

Hi!

DomUs can be backed up just fine, and there''s no problem to restore
files as long
as the DomU boots. Now I wonder how to recover hard (desaster recovery)...
Backing up the disk images or partitions (from Dom0) seems to be not an option, 
because there are changes to them all the time, so incremental backups would 
always save them, and all saves would be inconsistent.

Now the idea is this: Provide a tool that extracts the "structure of a
partition
or image file" into some data file or script that can be used to recrerate
that
container later. Then my idea was to loop-mount the thing (yet unsure how to do 
that for partitions and files that are "harddisks" for DomU), and
restore the
backup to those.

Maybe a "mini-root" (rescue system) as tar archive would be helpful as
well.
I''d like to see these issues discussed at least in the docs.

Regards,
Ulrich


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Ulrich,
> DomUs can be backed up just fine, and there''s no problem to
restore files
> as long
> as the DomU boots. Now I wonder how to recover hard (desaster recovery)...
> Backing up the disk images or partitions (from Dom0) seems to be not an
> option,
> because there are changes to them all the time, so incremental backups
> would always save them, and all saves would be inconsistent.
I''m using LVM for DomUs: On backup time, I create a LVM snapshot, mount
the snapshot in Dom0 and running rdiff-backup to create a new increment.
On desaster, I create a new logical volume and restore the rdiff-backup
into the mounted volume after creating a filesystem in the new LV. Now you
can umount the volume and startup the DomU.
> Now the idea is this: Provide a tool that extracts the "structure of a
> partition
> or image file" into some data file or script that can be used to
recrerate
> that
> container later. Then my idea was to loop-mount the thing (yet unsure how
> to do
> that for partitions and files that are "harddisks" for DomU), and
restore
> the backup to those.
You may create a snapshot, and dd it somewhere to s save place. But there
are two disadvantages:

- This needs a lot of backup space
- You can''t recover from hidden filesystem corruptions
> Maybe a "mini-root" (rescue system) as tar archive would be
helpful as
> well. I''d like to see these issues discussed at least in the docs.
I tar Dom0. Since it has a very small footprint this is the ideal rescue
system for me.

Gruß,
cp

-- 
Die Homepage meines Vertrauens: www.cpur.de


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On 11 Oct 2006 at 8:24, Christoph Purrucker wrote:
> I tar Dom0. Since it has a very small footprint this is the ideal rescue
> system for me.
> 
I wasn''t really successful with a minimal Dom0 and SLES10:
# rpm -qa |wc
    544     544   11988


Regards,
Ulrich


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 11 October 2006 1:24 am, Christoph Purrucker
wrote:
> I''m using LVM for DomUs: On backup time, I create a LVM snapshot,
mount
> the snapshot in Dom0 and running rdiff-backup to create a new increment.
i guess that if the snapshot is done while the DomU is running, mounting the 
snapshot would find it as ''not cleanly unmounted'' and might
require an fsck.
have you encountered this issue?

-- 
Javier


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Javier,
>> I''m using LVM for DomUs: On backup time, I create a LVM
snapshot, mount
>> the snapshot in Dom0 and running rdiff-backup to create a new
increment.
> 
> i guess that if the snapshot is done while the DomU is running, mounting
the
> snapshot would find it as ''not cleanly unmounted'' and
might require an fsck.
> have you encountered this issue?
of course, the filesystem was not cleanly unmounted. But as long as you 
use a journaling filesystem, the log run on mount time will correct 
nearly all errors: If you can mount the snapshot, if will be consistent. 
If the DomU did some wired things during snapshot, and the journal will 
not help to recover, I''d skip the backup and try it later. I''m
doing
such backups with several ext3 and one xfs (a cyrus partition with lots 
of files), and the snapshots always were mountable after journal recovery.

cu cp

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Julian,
>> of course, the filesystem was not cleanly unmounted. But as long as 
>> you use a journaling filesystem, the log run on mount time will 
>> correct nearly all errors: If you can mount the snapshot, if will be 
>> consistent. If the DomU did some wired things during snapshot, and the 
>> journal will not help to recover, I''d skip the backup and try
it
>> later. I''m doing such backups with several ext3 and one xfs (a
cyrus
>> partition with lots of files), and the snapshots always were mountable 
>> after journal recovery.
> 
> How do you know if the backup mounts? Do you just try it and
> see what happens?
Yes, mount returns useful error codes when trying. In your shell-script 
you can easily case the error codes and decide what to do.
> (I''m wondering how an automated system will ''try it
later'')
I currently only echo a warning, so I''ll see the error in the log the 
cronjob is mailing me. ''try it later'' means, that I skip the
domU at
this run. Because I do 4 runs a day, it wouldn''t be that bad. But - as
I
wrote above - it always recovered successfully since installation of the 
script.

cu cp

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On 11 Oct 2006 at 22:53, Christoph Purrucker wrote:
> Hi Javier,
> 
> >> I''m using LVM for DomUs: On backup time, I create a LVM
snapshot, mount
> >> the snapshot in Dom0 and running rdiff-backup to create a new
increment.
> > 
> > i guess that if the snapshot is done while the DomU is running,
mounting the
> > snapshot would find it as ''not cleanly unmounted''
and might require an fsck.
> > have you encountered this issue?
> 
> of course, the filesystem was not cleanly unmounted. But as long as you 
> use a journaling filesystem, the log run on mount time will correct 
...and XEN preserves write ordering...
> nearly all errors: If you can mount the snapshot, if will be consistent. 
> If the DomU did some wired things during snapshot, and the journal will 
> not help to recover, I''d skip the backup and try it later.
I''m doing
> such backups with several ext3 and one xfs (a cyrus partition with lots 
> of files), and the snapshots always were mountable after journal recovery.
...mountable does not mean all the data is still tehre, however...

Ulrich
> 
> cu cp
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ulrich Windl wrote:
> On 11 Oct 2006 at 8:24, Christoph Purrucker wrote:
> 
>> I tar Dom0. Since it has a very small footprint this is the ideal
rescue
>> system for me.
>>
> 
> I wasn''t really successful with a minimal Dom0 and SLES10:
> # rpm -qa |wc
>     544     544   11988
I had more success with FC5:

# rpm -qa | wc
    172     172    3569

And this includes some stuff I had to install to customise SELinux
policy. I could probably reduce this somewhat by removing the a few
dev/admin tools (gcc, lsof, m4, etc.)

R.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Did your setup of taking lvm snapshots and then backing them up work without
any issues ? I heard lvm snapshots weren''t stable.

On 10/11/06, Christoph Purrucker <cp+ml-xen@bnmsp.de>
wrote:
>
> I''m using LVM for DomUs: On backup time, I create a LVM snapshot,
mount
> the snapshot in Dom0 and running rdiff-backup to create a new increment.
> On desaster, I create a new logical volume and restore the rdiff-backup
> into the mounted volume after creating a filesystem in the new LV. Now you
> can umount the volume and startup the DomU.
>
-- 
regards,

Anand Gupta


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hello Anand,
> Did your setup of taking lvm snapshots and then backing them up work 
> without any issues ? I heard lvm snapshots weren''t stable.
as I wrote above, I''m using this setup for a while now - no probs so
far
(and yes - I do verify my backups :-)). I''m using LVM2 of course with 
Debian Etch. Where do you heard other voices? Do you have references?

cu cp

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users