Xen users - Feb 2011 - Backup running Windows machines - redundancy

Hi,

I know that there are a lot of tools to successfully backup windows guest
machine, using samba shares, dd, rsync etc. They all work, but they
don''t
solve one problem: redundancy.

In my current setup, backups are done using LVM snapshots, and rsync over
the network. The problem is that the rsync is not incremental, and every
time a new copy of the entire lv is sent to the backup server. I could mount
on the Dom0 the disk of the windows guest, and send the files, but the
restore process doesn''t work.

Do you know/use other method to do incremental backups for Windows machines,
that is more reliable or safer than this one?

Thanks alot.

Paul Piscuc


<http://www.appnor.com>


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Paul,
currently I''m trying to backup Windows machines using rdiff-backup for
incremental backups.
I mount the Windows fs in dom0 and then rdiff-backup performs the
incremental backup.
To restore a backup, just mount the Windows fs, rm all files in the mount
point, and use rdiff-backup to restore an old backup to the mounted Windows
fs.

Roberto



2011/2/2 Paul PISCUC <paul.piscuc@appnor.com>
> Hi,
>
> I know that there are a lot of tools to successfully backup windows guest
> machine, using samba shares, dd, rsync etc. They all work, but they
don''t
> solve one problem: redundancy.
>
> In my current setup, backups are done using LVM snapshots, and rsync over
> the network. The problem is that the rsync is not incremental, and every
> time a new copy of the entire lv is sent to the backup server. I could
mount
> on the Dom0 the disk of the windows guest, and send the files, but the
> restore process doesn''t work.
>
> Do you know/use other method to do incremental backups for Windows
> machines, that is more reliable or safer than this one?
>
> Thanks alot.
>
> Paul Piscuc
>
>
> <http://www.appnor.com>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


-- 
Roberto Bifulco, Ph.D. Student
robertobifulco.it
COMICS Lab - www.comics.unina.it


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Roberto,

could you give us some more informations concerning your
backup-strategy.

I''m working on the same subject and worked out some kind of
"universal
backup-script" which 

uses different backup-methods, like dd with gzip, partimage, and
ntfsclone for different operating systems.

 

Since nearly all of our domu have their base on LVM Storage, the process
of creating a snapshot 

and afterwards working with different backup tools on the snapshot seems
to be the right way.

For desaster recovery an image created with dd and gzip works great but
as stated by Paul this 

kind of backup is not the right tool for daily (file-) backup.

 

Do you use ntfs-3g for mounting the snapshot or the windows file system?

What about the windows acls on those files, is rdiff able to include
them in the backup?

 

Guido

 

________________________________

Von: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von Roberto
Bifulco
Gesendet: Mittwoch, 2. Februar 2011 15:14
An: sysadmin@appnor.com
Cc: xen-users@lists.xensource.com
Betreff: Re: [Xen-users] Backup running Windows machines - redundancy

 

Hi Paul,

currently I''m trying to backup Windows machines using rdiff-backup for
incremental backups.

I mount the Windows fs in dom0 and then rdiff-backup performs the
incremental backup.

To restore a backup, just mount the Windows fs, rm all files in the
mount point, and use rdiff-backup to restore an old backup to the
mounted Windows fs.

 

Roberto

 

 

 

2011/2/2 Paul PISCUC <paul.piscuc@appnor.com>

Hi,

I know that there are a lot of tools to successfully backup windows
guest machine, using samba shares, dd, rsync etc. They all work, but
they don''t solve one problem: redundancy. 

In my current setup, backups are done using LVM snapshots, and rsync
over the network. The problem is that the rsync is not incremental, and
every time a new copy of the entire lv is sent to the backup server. I
could mount on the Dom0 the disk of the windows guest, and send the
files, but the restore process doesn''t work.

Do you know/use other method to do incremental backups for Windows
machines, that is more reliable or safer than this one?

Thanks alot.

Paul Piscuc


<http://www.appnor.com> 
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users




-- 
Roberto Bifulco, Ph.D. Student
robertobifulco.it
COMICS Lab - www.comics.unina.it



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Also, what I like to know: if you do a snapshot of a drive of a running
Windows DomU and copy (backup) the files in it after mounting, can you
assume they are all intact and safe to put aside. Obviously this is only
about regular files, not database stuff and things like that.

thx,

B.



On 02/02/11 18:39, Guido Hecken wrote:
> Roberto,
> 
> could you give us some more informations concerning your backup-strategy.
> 
> I’m working on the same subject and worked out some kind of “universal
> backup-script” which
> 
> uses different backup-methods, like dd with gzip, partimage, and
> ntfsclone for different operating systems.
> 
>  
> 
> Since nearly all of our domu have their base on LVM Storage, the process
> of creating a snapshot
> 
> and afterwards working with different backup tools on the snapshot seems
> to be the right way.
> 
> For desaster recovery an image created with dd and gzip works great but
> as stated by Paul this
> 
> kind of backup is not the right tool for daily (file-) backup.
> 
>  
> 
> Do you use ntfs-3g for mounting the snapshot or the windows file system?
> 
> What about the windows acls on those files, is rdiff able to include
> them in the backup?
> 
>  
> 
> Guido
> 
>  
> 
> ------------------------------------------------------------------------
> 
> *Von:* xen-users-bounces@lists.xensource.com
> [mailto:xen-users-bounces@lists.xensource.com] *Im Auftrag von *Roberto
> Bifulco
> *Gesendet:* Mittwoch, 2. Februar 2011 15:14
> *An:* sysadmin@appnor.com
> *Cc:* xen-users@lists.xensource.com
> *Betreff:* Re: [Xen-users] Backup running Windows machines - redundancy
> 
>  
> 
> Hi Paul,
> 
> currently I''m trying to backup Windows machines using rdiff-backup
for
> incremental backups.
> 
> I mount the Windows fs in dom0 and then rdiff-backup performs the
> incremental backup.
> 
> To restore a backup, just mount the Windows fs, rm all files in the
> mount point, and use rdiff-backup to restore an old backup to the
> mounted Windows fs.
> 
>  
> 
> Roberto
> 
>  
> 
>  
> 
>  
> 
> 2011/2/2 Paul PISCUC <paul.piscuc@appnor.com
> <mailto:paul.piscuc@appnor.com>>
> 
> Hi,
> 
> I know that there are a lot of tools to successfully backup windows
> guest machine, using samba shares, dd, rsync etc. They all work, but
> they don''t solve one problem: redundancy.
> 
> In my current setup, backups are done using LVM snapshots, and rsync
> over the network. The problem is that the rsync is not incremental, and
> every time a new copy of the entire lv is sent to the backup server. I
> could mount on the Dom0 the disk of the windows guest, and send the
> files, but the restore process doesn''t work.
> 
> Do you know/use other method to do incremental backups for Windows
> machines, that is more reliable or safer than this one?
> 
> Thanks alot.
> 
> Paul Piscuc
> 
> 
> <http://www.appnor.com>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com <mailto:Xen-users@lists.xensource.com>
> http://lists.xensource.com/xen-users
> 
> 
> 
> 
> -- 
> Roberto Bifulco, Ph.D. Student
> robertobifulco.it <http://robertobifulco.it>
> COMICS Lab - www.comics.unina.it <http://www.comics.unina.it>
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Bart,

you''re right, databases should be backed up with their own
backup-technologies inside the domu e.g. mysqldump or ntbackup for exchange to
have them in a consistent state.
One should perform the main-backup right after the db-backup script successfully
finished.
If a short downtime of the db is not a problem, one could even stop the db and
restart it when backup is done.
Maybe, if accessibility of the server is no problem too, you could even flush
all(?) memory pages to disk, pause the whole machine and do your backup-job.
But all of these thoughts are not specific to virtualization technologies except
the one with pausing the server.
;-)
Here some links to the topic:
http://en.wikipedia.org/wiki/Near_Continuous_Backup#Continuous_vs_near_continuous
http://wiki.r1soft.com/display/TP/near-Continuous+V.S.+True+CDP

Guido


-----Ursprüngliche Nachricht-----
Von: Bart Coninckx [mailto:bart.coninckx@telenet.be] 
Gesendet: Mittwoch, 2. Februar 2011 22:43
An: Guido Hecken
Cc: xen-users@lists.xensource.com
Betreff: Re: [Xen-users] Backup running Windows machines - redundancy

Also, what I like to know: if you do a snapshot of a drive of a running
Windows DomU and copy (backup) the files in it after mounting, can you
assume they are all intact and safe to put aside. Obviously this is only
about regular files, not database stuff and things like that.

thx,

B.



On 02/02/11 18:39, Guido Hecken wrote:
> Roberto,
> 
> could you give us some more informations concerning your backup-strategy.
> 
> I''m working on the same subject and worked out some kind of
"universal
> backup-script" which
> 
> uses different backup-methods, like dd with gzip, partimage, and
> ntfsclone for different operating systems.
> 
>  
> 
> Since nearly all of our domu have their base on LVM Storage, the process
> of creating a snapshot
> 
> and afterwards working with different backup tools on the snapshot seems
> to be the right way.
> 
> For desaster recovery an image created with dd and gzip works great but
> as stated by Paul this
> 
> kind of backup is not the right tool for daily (file-) backup.
> 
>  
> 
> Do you use ntfs-3g for mounting the snapshot or the windows file system?
> 
> What about the windows acls on those files, is rdiff able to include
> them in the backup?
> 
>  
> 
> Guido
> 
>  
> 
> ------------------------------------------------------------------------
> 
> *Von:* xen-users-bounces@lists.xensource.com
> [mailto:xen-users-bounces@lists.xensource.com] *Im Auftrag von *Roberto
> Bifulco
> *Gesendet:* Mittwoch, 2. Februar 2011 15:14
> *An:* sysadmin@appnor.com
> *Cc:* xen-users@lists.xensource.com
> *Betreff:* Re: [Xen-users] Backup running Windows machines - redundancy
> 
>  
> 
> Hi Paul,
> 
> currently I''m trying to backup Windows machines using rdiff-backup
for
> incremental backups.
> 
> I mount the Windows fs in dom0 and then rdiff-backup performs the
> incremental backup.
> 
> To restore a backup, just mount the Windows fs, rm all files in the
> mount point, and use rdiff-backup to restore an old backup to the
> mounted Windows fs.
> 
>  
> 
> Roberto
> 
>  
> 
>  
> 
>  
> 
> 2011/2/2 Paul PISCUC <paul.piscuc@appnor.com
> <mailto:paul.piscuc@appnor.com>>
> 
> Hi,
> 
> I know that there are a lot of tools to successfully backup windows
> guest machine, using samba shares, dd, rsync etc. They all work, but
> they don''t solve one problem: redundancy.
> 
> In my current setup, backups are done using LVM snapshots, and rsync
> over the network. The problem is that the rsync is not incremental, and
> every time a new copy of the entire lv is sent to the backup server. I
> could mount on the Dom0 the disk of the windows guest, and send the
> files, but the restore process doesn''t work.
> 
> Do you know/use other method to do incremental backups for Windows
> machines, that is more reliable or safer than this one?
> 
> Thanks alot.
> 
> Paul Piscuc
> 
> 
> <http://www.appnor.com>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com <mailto:Xen-users@lists.xensource.com>
> http://lists.xensource.com/xen-users
> 
> 
> 
> 
> -- 
> Roberto Bifulco, Ph.D. Student
> robertobifulco.it <http://robertobifulco.it>
> COMICS Lab - www.comics.unina.it <http://www.comics.unina.it>
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Guido,
currently I''m performing a "raw" file-level backup of LV
volumes.

These are the steps:

1. Pause VM
2. Snapshot LV
3. Unpause VM
4. Use kpartx to find partitions on LV-snapshot
5. Mount windows partition using ntfs-3g
6. Use rdiff-backup over the mounted partition, targeting a backup-storage
7. Umount
8. Remove snapshot


 > What about the windows acls on those files, is rdiff able to include them
in the backup?

I''m totally dependent on rdiff-backup for such things. So you can refer
to
rdiff-backup documentation.
In http://www.nongnu.org/rdiff-backup/features.html you can find this:

"Filesystem feature autodetection: People use rdiff-backup in many
different
environments. The filesystem they want to back up may be on Linux, Windows,
or Mac. It may or may not be case sensitive, support characters like
":",
have resource forks, extended attributes, or access control lists. Moreover,
the file system they are backing up to may or may not support these
features.

rdiff-backup tries to handle these situations automatically without the need
for switches like --acl --ea --no-ownership, etc. When run it will run tests
on both the source and destination filesystems to see what features each
supports like case sensitivity, changing uid/gid ownership, resource forks,
extended attributes, or access control lists. To see the results of this
testing, run rdiff-backup with verbosity 4 or higher, as in -v4."

Roberto.

2011/2/2 Guido Hecken <guido.hecken@gwsnettech.de>
>  Roberto,
>
> could you give us some more informations concerning your backup-strategy.
>
> I’m working on the same subject and worked out some kind of “universal
> backup-script” which
>
> uses different backup-methods, like dd with gzip, partimage, and ntfsclone
> for different operating systems.
>
>
>
> Since nearly all of our domu have their base on LVM Storage, the process of
> creating a snapshot
>
> and afterwards working with different backup tools on the snapshot seems to
> be the right way.
>
> For desaster recovery an image created with dd and gzip works great but as
> stated by Paul this
>
> kind of backup is not the right tool for daily (file-) backup.
>
>
>
> Do you use ntfs-3g for mounting the snapshot or the windows file system?
>
> What about the windows acls on those files, is rdiff able to include them
> in the backup?
>
>
>
> Guido
>
>
>  ------------------------------
>
> *Von:* xen-users-bounces@lists.xensource.com [mailto:
> xen-users-bounces@lists.xensource.com] *Im Auftrag von *Roberto Bifulco
> *Gesendet:* Mittwoch, 2. Februar 2011 15:14
> *An:* sysadmin@appnor.com
> *Cc:* xen-users@lists.xensource.com
> *Betreff:* Re: [Xen-users] Backup running Windows machines - redundancy
>
>
>
> Hi Paul,
>
> currently I''m trying to backup Windows machines using rdiff-backup
for
> incremental backups.
>
> I mount the Windows fs in dom0 and then rdiff-backup performs the
> incremental backup.
>
> To restore a backup, just mount the Windows fs, rm all files in the mount
> point, and use rdiff-backup to restore an old backup to the mounted Windows
> fs.
>
>
>
> Roberto
>
>
>
>
>
>
>
> 2011/2/2 Paul PISCUC <paul.piscuc@appnor.com>
>
> Hi,
>
> I know that there are a lot of tools to successfully backup windows guest
> machine, using samba shares, dd, rsync etc. They all work, but they
don''t
> solve one problem: redundancy.
>
> In my current setup, backups are done using LVM snapshots, and rsync over
> the network. The problem is that the rsync is not incremental, and every
> time a new copy of the entire lv is sent to the backup server. I could
mount
> on the Dom0 the disk of the windows guest, and send the files, but the
> restore process doesn''t work.
>
> Do you know/use other method to do incremental backups for Windows
> machines, that is more reliable or safer than this one?
>
> Thanks alot.
>
> Paul Piscuc
>
>
>  <http://www.appnor.com>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
>
>
>
> --
> Roberto Bifulco, Ph.D. Student
> robertobifulco.it
> COMICS Lab - www.comics.unina.it
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


-- 
Roberto Bifulco, Ph.D. Student
robertobifulco.it
COMICS Lab - www.comics.unina.it


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Roberto,

good to see others are doing things in a very similiar way.

:-)

 

My steps differ from your''s only in step one (three) and in step six.

Since it certainly does not harm, what are the advantages of your step one?

Afaik, creating a snapshot should already suffice to get a consistent state of
the volume.

Does the pausing of the VM have some "extra magic" or even extra
safety on the snapshot creation?

 

Thanks for clearing things about the acls in the use of rdiff-backup.

I''ll do some tests with some "really ugly" acls on the files
to be backed up.

 

In my step six I normally do things like ntfsclone, dd through gzip, partimage
and hopefully

soon rdiff-backup.

 

Surely this thread will assist others in finding the right solution for their
backup strategy.

 

Guido

 

gwsNetTech
Guido Hecken

Quirrenbacher Str. 36
53639 Königswinter
Germany


fon     +49(2244) 870663
fax     +49(2244) 870664
mobil  +49(179) 1267353
web     http://www.gwsnettech.de <http://www.gwsnettech.de/> 
mail    guido.hecken@gwsnettech.de <mailto:iguido.hecken@gwsnettech.de> 

 

________________________________

Von: Roberto Bifulco [mailto:roberto.bifulco2@unina.it] 
Gesendet: Donnerstag, 3. Februar 2011 15:53
An: Guido Hecken
Cc: xen-users@lists.xensource.com
Betreff: Re: [Xen-users] Backup running Windows machines - redundancy

 

Guido,
currently I''m performing a "raw" file-level backup of LV
volumes.

These are the steps:

1. Pause VM
2. Snapshot LV
3. Unpause VM
4. Use kpartx to find partitions on LV-snapshot
5. Mount windows partition using ntfs-3g
6. Use rdiff-backup over the mounted partition, targeting a backup-storage
7. Umount
8. Remove snapshot


 > What about the windows acls on those files, is rdiff able to include them
in the backup?

I''m totally dependent on rdiff-backup for such things. So you can refer
to rdiff-backup documentation.
In http://www.nongnu.org/rdiff-backup/features.html you can find this:

"Filesystem feature autodetection: People use rdiff-backup in many
different environments. The filesystem they want to back up may be on Linux,
Windows, or Mac. It may or may not be case sensitive, support characters like
":", have resource forks, extended attributes, or access control
lists. Moreover, the file system they are backing up to may or may not support
these features.

rdiff-backup tries to handle these situations automatically without the need for
switches like --acl --ea --no-ownership, etc. When run it will run tests on both
the source and destination filesystems to see what features each supports like
case sensitivity, changing uid/gid ownership, resource forks, extended
attributes, or access control lists. To see the results of this testing, run
rdiff-backup with verbosity 4 or higher, as in -v4."

Roberto.

2011/2/2 Guido Hecken <guido.hecken@gwsnettech.de>

Roberto,

could you give us some more informations concerning your backup-strategy.

I''m working on the same subject and worked out some kind of
"universal backup-script" which

uses different backup-methods, like dd with gzip, partimage, and ntfsclone for
different operating systems.

 

Since nearly all of our domu have their base on LVM Storage, the process of
creating a snapshot

and afterwards working with different backup tools on the snapshot seems to be
the right way.

For desaster recovery an image created with dd and gzip works great but as
stated by Paul this

kind of backup is not the right tool for daily (file-) backup.

 

Do you use ntfs-3g for mounting the snapshot or the windows file system?

What about the windows acls on those files, is rdiff able to include them in the
backup?

 

Guido

 

________________________________

Von: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von Roberto Bifulco
Gesendet: Mittwoch, 2. Februar 2011 15:14
An: sysadmin@appnor.com
Cc: xen-users@lists.xensource.com
Betreff: Re: [Xen-users] Backup running Windows machines - redundancy

 

Hi Paul,

currently I''m trying to backup Windows machines using rdiff-backup for
incremental backups.

I mount the Windows fs in dom0 and then rdiff-backup performs the incremental
backup.

To restore a backup, just mount the Windows fs, rm all files in the mount point,
and use rdiff-backup to restore an old backup to the mounted Windows fs.

 

Roberto

 

 

 

2011/2/2 Paul PISCUC <paul.piscuc@appnor.com>

Hi,

I know that there are a lot of tools to successfully backup windows guest
machine, using samba shares, dd, rsync etc. They all work, but they
don''t solve one problem: redundancy.

In my current setup, backups are done using LVM snapshots, and rsync over the
network. The problem is that the rsync is not incremental, and every time a new
copy of the entire lv is sent to the backup server. I could mount on the Dom0
the disk of the windows guest, and send the files, but the restore process
doesn''t work.

Do you know/use other method to do incremental backups for Windows machines,
that is more reliable or safer than this one?

Thanks alot.

Paul Piscuc


<http://www.appnor.com> 
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users




-- 
Roberto Bifulco, Ph.D. Student
robertobifulco.it
COMICS Lab - www.comics.unina.it


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users




-- 
Roberto Bifulco, Ph.D. Student
robertobifulco.it
COMICS Lab - www.comics.unina.it



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Guido,


it is not directly related no, though I can see not a lot of other
circumstances where you would snapshot an underlying LVM to backup
Windows files, except for with virtualization. Not sure where questions
like this would be best asked. Anyway, mine was about regular Windows
files, not databases. I have no possibility to pause the machine, so
that was why I was curious. I will test it and see what gives.

Thx for the links,

B.


On 02/03/11 01:08, Guido Hecken wrote:
> Bart,
> 
> you''re right, databases should be backed up with their own
backup-technologies inside the domu e.g. mysqldump or ntbackup for exchange to
have them in a consistent state.
> One should perform the main-backup right after the db-backup script
successfully finished.
> If a short downtime of the db is not a problem, one could even stop the db
and restart it when backup is done.
> Maybe, if accessibility of the server is no problem too, you could even
flush all(?) memory pages to disk, pause the whole machine and do your
backup-job.
> But all of these thoughts are not specific to virtualization technologies
except the one with pausing the server.
> ;-)
> Here some links to the topic:
>
http://en.wikipedia.org/wiki/Near_Continuous_Backup#Continuous_vs_near_continuous
> http://wiki.r1soft.com/display/TP/near-Continuous+V.S.+True+CDP
> 
> Guido
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: Bart Coninckx [mailto:bart.coninckx@telenet.be] 
> Gesendet: Mittwoch, 2. Februar 2011 22:43
> An: Guido Hecken
> Cc: xen-users@lists.xensource.com
> Betreff: Re: [Xen-users] Backup running Windows machines - redundancy
> 
> Also, what I like to know: if you do a snapshot of a drive of a running
> Windows DomU and copy (backup) the files in it after mounting, can you
> assume they are all intact and safe to put aside. Obviously this is only
> about regular files, not database stuff and things like that.
> 
> thx,
> 
> B.
> 
> 
> 
> On 02/02/11 18:39, Guido Hecken wrote:
>> Roberto,
>>
>> could you give us some more informations concerning your
backup-strategy.
>>
>> I''m working on the same subject and worked out some kind of
"universal
>> backup-script" which
>>
>> uses different backup-methods, like dd with gzip, partimage, and
>> ntfsclone for different operating systems.
>>
>>  
>>
>> Since nearly all of our domu have their base on LVM Storage, the
process
>> of creating a snapshot
>>
>> and afterwards working with different backup tools on the snapshot
seems
>> to be the right way.
>>
>> For desaster recovery an image created with dd and gzip works great but
>> as stated by Paul this
>>
>> kind of backup is not the right tool for daily (file-) backup.
>>
>>  
>>
>> Do you use ntfs-3g for mounting the snapshot or the windows file
system?
>>
>> What about the windows acls on those files, is rdiff able to include
>> them in the backup?
>>
>>  
>>
>> Guido
>>
>>  
>>
>>
------------------------------------------------------------------------
>>
>> *Von:* xen-users-bounces@lists.xensource.com
>> [mailto:xen-users-bounces@lists.xensource.com] *Im Auftrag von *Roberto
>> Bifulco
>> *Gesendet:* Mittwoch, 2. Februar 2011 15:14
>> *An:* sysadmin@appnor.com
>> *Cc:* xen-users@lists.xensource.com
>> *Betreff:* Re: [Xen-users] Backup running Windows machines - redundancy
>>
>>  
>>
>> Hi Paul,
>>
>> currently I''m trying to backup Windows machines using
rdiff-backup for
>> incremental backups.
>>
>> I mount the Windows fs in dom0 and then rdiff-backup performs the
>> incremental backup.
>>
>> To restore a backup, just mount the Windows fs, rm all files in the
>> mount point, and use rdiff-backup to restore an old backup to the
>> mounted Windows fs.
>>
>>  
>>
>> Roberto
>>
>>  
>>
>>  
>>
>>  
>>
>> 2011/2/2 Paul PISCUC <paul.piscuc@appnor.com
>> <mailto:paul.piscuc@appnor.com>>
>>
>> Hi,
>>
>> I know that there are a lot of tools to successfully backup windows
>> guest machine, using samba shares, dd, rsync etc. They all work, but
>> they don''t solve one problem: redundancy.
>>
>> In my current setup, backups are done using LVM snapshots, and rsync
>> over the network. The problem is that the rsync is not incremental, and
>> every time a new copy of the entire lv is sent to the backup server. I
>> could mount on the Dom0 the disk of the windows guest, and send the
>> files, but the restore process doesn''t work.
>>
>> Do you know/use other method to do incremental backups for Windows
>> machines, that is more reliable or safer than this one?
>>
>> Thanks alot.
>>
>> Paul Piscuc
>>
>>
>> <http://www.appnor.com>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
<mailto:Xen-users@lists.xensource.com>
>> http://lists.xensource.com/xen-users
>>
>>
>>
>>
>> -- 
>> Roberto Bifulco, Ph.D. Student
>> robertobifulco.it <http://robertobifulco.it>
>> COMICS Lab - www.comics.unina.it <http://www.comics.unina.it>
>>
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,

Regarding the backups made to the Windows fs using rdiff ... does rdiff 
preserve Windows ACLs and all those permissions?  What about hidden 
files, or files with different permissions between incremental backups?

Sorry for replying so late.

Paul

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

When you backup to a Linux system I don''t believe these will be
preserved, there is no way to. 
>From Windows to Windows this might be OK though.
B.


----- Originele e-mail  -----
Van: "Paul Piscuc" <paul.piscuc@appnor.com>
Aan: xen-users@lists.xensource.com
Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam / Berlijn /
Bern / Rome / Stockholm / Wenen
Onderwerp: Re: [Xen-users] Backup running Windows machines - redundancy

Hi,

Regarding the backups made to the Windows fs using rdiff ... does rdiff 
preserve Windows ACLs and all those permissions?  What about hidden 
files, or files with different permissions between incremental backups?

Sorry for replying so late.

Paul

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wed, Feb 9, 2011 at 6:06 PM,  <bart.coninckx@telenet.be>
wrote:
> When you backup to a Linux system I don''t believe these will be
preserved, there is no way to.
> >From Windows to Windows this might be OK though.
This also leaves a question, can you recover a complete system (OS
included) with rdiff. If anyone has implemented this setup please
share your experience.

Personally, I''m testing using native zfs on Linux
(https://github.com/zfs-linux) on Xen dom0, and use the zvol as domU
storage.
It''s on RHEL dom0, with xen 4.0.1 rpm from http://www.gitco.de/repo/,
plus custom kernel from
http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=shortlog;h=refs/heads/xen/stable-2.6.32.x
. With this setup, incremental backup using "zfs send" is easy, as
well as restoring the block device using "zfs receive".

-- 
Fajar
>
> B.
>
>
> ----- Originele e-mail  -----
> Van: "Paul Piscuc" <paul.piscuc@appnor.com>
> Aan: xen-users@lists.xensource.com
> Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam / Berlijn
/ Bern / Rome / Stockholm / Wenen
> Onderwerp: Re: [Xen-users] Backup running Windows machines - redundancy
>
> Hi,
>
> Regarding the backups made to the Windows fs using rdiff ... does rdiff
> preserve Windows ACLs and all those permissions?  What about hidden
> files, or files with different permissions between incremental backups?
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Well, according to http://www.nongnu.org/rdiff-backup/features.html
one of the main advantages of rdiff over other forms of file based 
backups is that it preserves all meta data.

I have no personal experience with it though.

-henrik

On Wed, Feb 09, 2011 at 12:06:25PM +0100, bart.coninckx@telenet.be
wrote:
> When you backup to a Linux system I don''t believe these will be
preserved, there is no way to.
> >From Windows to Windows this might be OK though.
> 
> B.
> 
> 
> ----- Originele e-mail  -----
> Van: "Paul Piscuc" <paul.piscuc@appnor.com>
> Aan: xen-users@lists.xensource.com
> Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam / Berlijn
/ Bern / Rome / Stockholm / Wenen
> Onderwerp: Re: [Xen-users] Backup running Windows machines - redundancy
> 
> Hi,
> 
> Regarding the backups made to the Windows fs using rdiff ... does rdiff 
> preserve Windows ACLs and all those permissions?  What about hidden 
> files, or files with different permissions between incremental backups?
> 
> Sorry for replying so late.
> 
> Paul
> 
> -- 
> Best regards,
> Paul PISCUC
> System Administrator - Appnor MSP S.A.
> US: (+1) 650 336 57 50
> RO: (+4) 021 569 46 56
> http://www.appnor.com
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wed, Feb 9, 2011 at 6:37 PM, Henrik Langos
<hlangos-xen@innominate.com> wrote:
> Well, according to http://www.nongnu.org/rdiff-backup/features.html
> one of the main advantages of rdiff over other forms of file based
> backups is that it preserves all meta data.

Hmmm ... from that page

ACL and EA support:
...
ACLs are not supported, however, on Mac OS X or Windows as those
systems do not use standard POSIX.1e access controls.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wed, Feb 09, 2011 at 06:43:40PM +0700, Fajar A. Nugraha
wrote:
> On Wed, Feb 9, 2011 at 6:37 PM, Henrik Langos
> <hlangos-xen@innominate.com> wrote:
> > Well, according to http://www.nongnu.org/rdiff-backup/features.html
> > one of the main advantages of rdiff over other forms of file based
> > backups is that it preserves all meta data.
> 
> 
> Hmmm ... from that page
> 
> ACL and EA support:
> ...
> ACLs are not supported, however, on Mac OS X or Windows as those
> systems do not use standard POSIX.1e access controls.

You are right. So there is no complete preservation of meta data. 
Though I would still defend that there is a way to do it. It just 
hasn''t been done in rdiff :-)

Full OS backup that you can be restored into a bootable state is 
VERY hard to do though. At the very least you''d need to make sure
that the partition''s/disk''s boot record is also restored and
put
into the right spot.

Also when done from within the windows system there are always files
that are locked by the operating system and most Windows backup
programs use the "Volume Shadow Copy Service" to work around those
problems.

cheers
-henrik



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,

Has anyone been able to backup a Windows machine using rdiff, or other 
tool, that also preserves the meta data and allows restoring the VM?

On 02/09/2011 01:06 PM, bart.coninckx@telenet.be wrote:
> When you backup to a Linux system I don''t believe these will be
preserved, there is no way to.
>  From Windows to Windows this might be OK though.
>
> B.
>
>
> ----- Originele e-mail  -----
> Van: "Paul Piscuc"<paul.piscuc@appnor.com>
> Aan: xen-users@lists.xensource.com
> Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam / Berlijn
/ Bern / Rome / Stockholm / Wenen
> Onderwerp: Re: [Xen-users] Backup running Windows machines - redundancy
>
> Hi,
>
> Regarding the backups made to the Windows fs using rdiff ... does rdiff
> preserve Windows ACLs and all those permissions?  What about hidden
> files, or files with different permissions between incremental backups?
>
> Sorry for replying so late.
>
> Paul
>

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Paul,

me again. Imaging the virtual disks takes care of this. You can first perform a
snapshot for this. In order to get a 100% "safe" state I save the
machine first, snaphost it, dd it and then restore it again.

For backing up just the files I see two possibilities:

- use a share on the Windows DomU
- snapshot the volume, mount it and rdiff from it.

As for as ACLs preserving is concerned, tools like SetACL
(http://helgeklein.com/) can export them fpr later restore.

I don''t think there are any other ways to backup the files with ACLs
included that include any sort of copy to Linux file systems.

B.


----- "Paul Piscuc" <paul.piscuc@appnor.com> schreef:
> Hi,
> 
> Has anyone been able to backup a Windows machine using rdiff, or other
> 
> tool, that also preserves the meta data and allows restoring the VM?
> 
> On 02/09/2011 01:06 PM, bart.coninckx@telenet.be wrote:
> > When you backup to a Linux system I don''t believe these will
be
> preserved, there is no way to.
> >  From Windows to Windows this might be OK though.
> >
> > B.
> >
> >
> > ----- Originele e-mail  -----
> > Van: "Paul Piscuc"<paul.piscuc@appnor.com>
> > Aan: xen-users@lists.xensource.com
> > Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam /
> Berlijn / Bern / Rome / Stockholm / Wenen
> > Onderwerp: Re: [Xen-users] Backup running Windows machines -
> redundancy
> >
> > Hi,
> >
> > Regarding the backups made to the Windows fs using rdiff ... does
> rdiff
> > preserve Windows ACLs and all those permissions?  What about hidden
> > files, or files with different permissions between incremental
> backups?
> >
> > Sorry for replying so late.
> >
> > Paul
> >
> 
> 
> -- 
> Best regards,
> Paul PISCUC
> System Administrator - Appnor MSP S.A.
> US: (+1) 650 336 57 50
> RO: (+4) 021 569 46 56
> http://www.appnor.com
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Paul,

I did some tests on different tools like ntfsclone, partimage and dd through
gzip.
In combination with LVM Snapshots they produce a backup which should preserve
all meta data since these methods are not filebased.
One of the disadvantages are, these are not increment backup solutions, so you
will have multiple big archive files.
Commercial tools like acronis backup&recovery or R1soft''s cdp can
do the backup job image-based and in an differential or incremental way.
So you will get smaller archives.
But these are commercial solutions and not really cheap.
BTW R1Soft has a great and free(!) tool called Linux Hot Copy.
With this tool you can create snapshots from regular, not LVM based filesystems.
It''s a great enhancement for your daily backup stuff and it''s
free.

I think one should have at least two backup strategies, an image based for
disaster recovery scenarios and one for the "oops I deleted file xy
problem" as a filebased solution. Perhaps the filebased backup should run
directly in the VM and use the OS specific tools, at least in Windows systems.

Guido


-----Ursprüngliche Nachricht-----
Von: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von Paul Piscuc
Gesendet: Mittwoch, 9. Februar 2011 23:56
An: bart.coninckx@telenet.be
Cc: xen-users@lists.xensource.com
Betreff: Re: [Xen-users] Backup running Windows machines - redundancy

Hi,

Has anyone been able to backup a Windows machine using rdiff, or other 
tool, that also preserves the meta data and allows restoring the VM?

On 02/09/2011 01:06 PM, bart.coninckx@telenet.be wrote:
> When you backup to a Linux system I don''t believe these will be
preserved, there is no way to.
>  From Windows to Windows this might be OK though.
>
> B.
>
>
> ----- Originele e-mail  -----
> Van: "Paul Piscuc"<paul.piscuc@appnor.com>
> Aan: xen-users@lists.xensource.com
> Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam / Berlijn
/ Bern / Rome / Stockholm / Wenen
> Onderwerp: Re: [Xen-users] Backup running Windows machines - redundancy
>
> Hi,
>
> Regarding the backups made to the Windows fs using rdiff ... does rdiff
> preserve Windows ACLs and all those permissions?  What about hidden
> files, or files with different permissions between incremental backups?
>
> Sorry for replying so late.
>
> Paul
>

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi  Guido,

Thanks for the helpful insight. I think I''m doing to do a weekly
dd/gzip
image( or maybe twice a week), and daily incremental file backups. Still,
this doesn''t look like a clean method to backup. It''s more
like a
workaround.

ZFS would be a solution, but like every solution, comes with good and bads.

And one more thing ... what about saving the machine''s state or pausing
it,
and then try to backup it?

On Thu, Feb 10, 2011 at 10:46 AM, Guido Hecken
<guido.hecken@gwsnettech.de>wrote:
> Paul,
>
> I did some tests on different tools like ntfsclone, partimage and dd
> through gzip.
> In combination with LVM Snapshots they produce a backup which should
> preserve all meta data since these methods are not filebased.
> One of the disadvantages are, these are not increment backup solutions, so
> you will have multiple big archive files.
> Commercial tools like acronis backup&recovery or R1soft''s cdp
can do the
> backup job image-based and in an differential or incremental way.
> So you will get smaller archives.
> But these are commercial solutions and not really cheap.
> BTW R1Soft has a great and free(!) tool called Linux Hot Copy.
> With this tool you can create snapshots from regular, not LVM based
> filesystems.
> It''s a great enhancement for your daily backup stuff and
it''s free.
>
> I think one should have at least two backup strategies, an image based for
> disaster recovery scenarios and one for the "oops I deleted file xy
problem"
> as a filebased solution. Perhaps the filebased backup should run directly
in
> the VM and use the OS specific tools, at least in Windows systems.
>
> Guido
>
>
> -----Ursprüngliche Nachricht-----
> Von: xen-users-bounces@lists.xensource.com [mailto:
> xen-users-bounces@lists.xensource.com] Im Auftrag von Paul Piscuc
> Gesendet: Mittwoch, 9. Februar 2011 23:56
> An: bart.coninckx@telenet.be
> Cc: xen-users@lists.xensource.com
> Betreff: Re: [Xen-users] Backup running Windows machines - redundancy
>
> Hi,
>
> Has anyone been able to backup a Windows machine using rdiff, or other
> tool, that also preserves the meta data and allows restoring the VM?
>
> On 02/09/2011 01:06 PM, bart.coninckx@telenet.be wrote:
> > When you backup to a Linux system I don''t believe these will
be
> preserved, there is no way to.
> >  From Windows to Windows this might be OK though.
> >
> > B.
> >
> >
> > ----- Originele e-mail  -----
> > Van: "Paul Piscuc"<paul.piscuc@appnor.com>
> > Aan: xen-users@lists.xensource.com
> > Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam /
> Berlijn / Bern / Rome / Stockholm / Wenen
> > Onderwerp: Re: [Xen-users] Backup running Windows machines -
redundancy
> >
> > Hi,
> >
> > Regarding the backups made to the Windows fs using rdiff ... does
rdiff
> > preserve Windows ACLs and all those permissions?  What about hidden
> > files, or files with different permissions between incremental
backups?
> >
> > Sorry for replying so late.
> >
> > Paul
> >
>
>
> --
> Best regards,
> Paul PISCUC
> System Administrator - Appnor MSP S.A.
> US: (+1) 650 336 57 50
> RO: (+4) 021 569 46 56
> http://www.appnor.com
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
>

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

That''s what I actually suggested in previous post: do a xm save -f,
take
a snapshot of the underlying LV, do a xm restore and then dd the snapshot.
This is 100% safe for restore if you also backup the memory state file.


On 02/10/11 11:18, Paul PISCUC wrote:
> Hi  Guido,
> 
> Thanks for the helpful insight. I think I''m doing to do a weekly
dd/gzip
> image( or maybe twice a week), and daily incremental file backups.
> Still, this doesn''t look like a clean method to backup.
It''s more like a
> workaround.
> 
> ZFS would be a solution, but like every solution, comes with good and bads.
> 
> And one more thing ... what about saving the machine''s state or
pausing
> it, and then try to backup it?
> 
> On Thu, Feb 10, 2011 at 10:46 AM, Guido Hecken
> <guido.hecken@gwsnettech.de
<mailto:guido.hecken@gwsnettech.de>> wrote:
> 
>     Paul,
> 
>     I did some tests on different tools like ntfsclone, partimage and dd
>     through gzip.
>     In combination with LVM Snapshots they produce a backup which should
>     preserve all meta data since these methods are not filebased.
>     One of the disadvantages are, these are not increment backup
>     solutions, so you will have multiple big archive files.
>     Commercial tools like acronis backup&recovery or R1soft''s
cdp can do
>     the backup job image-based and in an differential or incremental way.
>     So you will get smaller archives.
>     But these are commercial solutions and not really cheap.
>     BTW R1Soft has a great and free(!) tool called Linux Hot Copy.
>     With this tool you can create snapshots from regular, not LVM based
>     filesystems.
>     It''s a great enhancement for your daily backup stuff and
it''s free.
> 
>     I think one should have at least two backup strategies, an image
>     based for disaster recovery scenarios and one for the "oops I
>     deleted file xy problem" as a filebased solution. Perhaps the
>     filebased backup should run directly in the VM and use the OS
>     specific tools, at least in Windows systems.
> 
>     Guido
> 
> 
>     -----Ursprüngliche Nachricht-----
>     Von: xen-users-bounces@lists.xensource.com
>     <mailto:xen-users-bounces@lists.xensource.com>
>     [mailto:xen-users-bounces@lists.xensource.com
>     <mailto:xen-users-bounces@lists.xensource.com>] Im Auftrag von
Paul
>     Piscuc
>     Gesendet: Mittwoch, 9. Februar 2011 23:56
>     An: bart.coninckx@telenet.be <mailto:bart.coninckx@telenet.be>
>     Cc: xen-users@lists.xensource.com
<mailto:xen-users@lists.xensource.com>
>     Betreff: Re: [Xen-users] Backup running Windows machines - redundancy
> 
>     Hi,
> 
>     Has anyone been able to backup a Windows machine using rdiff, or other
>     tool, that also preserves the meta data and allows restoring the VM?
> 
>     On 02/09/2011 01:06 PM, bart.coninckx@telenet.be
>     <mailto:bart.coninckx@telenet.be> wrote:
>     > When you backup to a Linux system I don''t believe these
will be
>     preserved, there is no way to.
>     >  From Windows to Windows this might be OK though.
>     >
>     > B.
>     >
>     >
>     > ----- Originele e-mail  -----
>     > Van: "Paul Piscuc"<paul.piscuc@appnor.com
>     <mailto:paul.piscuc@appnor.com>>
>     > Aan: xen-users@lists.xensource.com
>     <mailto:xen-users@lists.xensource.com>
>     > Verzonden: Woensdag 9 februari 2011 11:09:08 GMT +01:00 Amsterdam
>     / Berlijn / Bern / Rome / Stockholm / Wenen
>     > Onderwerp: Re: [Xen-users] Backup running Windows machines -
>     redundancy
>     >
>     > Hi,
>     >
>     > Regarding the backups made to the Windows fs using rdiff ... does
>     rdiff
>     > preserve Windows ACLs and all those permissions?  What about
hidden
>     > files, or files with different permissions between incremental
>     backups?
>     >
>     > Sorry for replying so late.
>     >
>     > Paul
>     >
> 
> 
>     --
>     Best regards,
>     Paul PISCUC
>     System Administrator - Appnor MSP S.A.
>     US: (+1) 650 336 57 50
>     RO: (+4) 021 569 46 56
>     http://www.appnor.com
> 
> 
>     _______________________________________________
>     Xen-users mailing list
>     Xen-users@lists.xensource.com
<mailto:Xen-users@lists.xensource.com>
>     http://lists.xensource.com/xen-users
> 
> 
> 
> 
> -- 
> Best regards,
> Paul PISCUC
> System Administrator - Appnor MSP S.A.
> US: (+1) 650 336 57 50
> RO: (+4) 021 569 46 56
> http://www.appnor.com
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Bart,
> That''s what I actually suggested in previous post: do a xm save
-f, take
> a snapshot of the underlying LV, do a xm restore and then dd the snapshot.
> This is 100% safe for restore if you also backup the memory state file.
You''re right, it''s only 100% safe if you backup the *memory*
state as
well.  Otherwise, from all the other proposed methods I''ve seen in
this thread, just by pausing or saving the VM does not guarantee that
the file system will be put into a consistent state, which is
important for those who care about file system backups and not VM
snapshots.

Mike

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Mike,

agreed. For situations however where data files that are typically
opened with user applications like Word, Excel and other, I think it
might be safe to just snapshot the underlying LVM while the Windows DomU
is running and the users are not accessing the files (typically at
night), mount this and copy from it. I''m actually testing this now. If
there''s interest I will post my findings.

B.

On 02/10/11 20:36, Mike Sun wrote:
> Hi Bart,
> 
>> That''s what I actually suggested in previous post: do a xm
save -f, take
>> a snapshot of the underlying LV, do a xm restore and then dd the
snapshot.
>> This is 100% safe for restore if you also backup the memory state file.
> 
> You''re right, it''s only 100% safe if you backup the
*memory* state as
> well.  Otherwise, from all the other proposed methods I''ve seen in
> this thread, just by pausing or saving the VM does not guarantee that
> the file system will be put into a consistent state, which is
> important for those who care about file system backups and not VM
> snapshots.
> 
> Mike
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> I did some tests on different tools like ntfsclone, partimage and dd
through gzip.
> In combination with LVM Snapshots they produce a backup which should
preserve all meta data since these
> methods are not filebased.
> One of the disadvantages are, these are not increment backup solutions, so
you will have multiple big archive
> files.
What if you used file-backed sparse images as the VM''s virtual disk
and then used rsync/rdiff-backup on that file itself?  That way, you
don''t have to worry about rsync/rdiff-backup being aware of NTFS
metadata, but still take advantage of the incremental binary diffs
that rsync/rdiff-backup provide at the file level.

Is that a feasible solution?

Mike

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> Hi Mike,
> 
> agreed. For situations however where data files that are typically
> opened with user applications like Word, Excel and other, I think it
> might be safe to just snapshot the underlying LVM while the Windows DomU
> is running and the users are not accessing the files (typically at
> night), mount this and copy from it. I'm actually testing this now. If
> there's interest I will post my findings.
> 
> B.
The Citrix Windows PV drivers include a VSS provider that (I think) allows you
to ensure that your snapshot is consistent at both a filesystem and application
level. I've thought about this for GPLPV but have never actually done
anything about it.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,

I''ll be testing the method also, for different Windows versions, with 
and without memory state backed up. One more question thought ... what 
are the performance and latency aspects of doing a xm save on a Windows 
machine? Does is save only the memory state, or other information 
regarding the state of the machine?

And one more thing we can test. What about restoring a rdiff backup(+MBR 
backup) to a fresh lvm disk ... maybe it will boot in blue screen, but 
it might work.

On 02/11/2011 12:36 AM, James Harper wrote:
>> Hi Mike,
>>
>> agreed. For situations however where data files that are typically
>> opened with user applications like Word, Excel and other, I think it
>> might be safe to just snapshot the underlying LVM while the Windows
DomU
>> is running and the users are not accessing the files (typically at
>> night), mount this and copy from it. I''m actually testing this
now. If
>> there''s interest I will post my findings.
>>
>> B.
> The Citrix Windows PV drivers include a VSS provider that (I think) allows
you to ensure that your snapshot is consistent at both a filesystem and
application level. I''ve thought about this for GPLPV but have never
actually done anything about it.
>
> James

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Paul,

but the problem with rdiff-backup on windows is the lack of suppport for
metadata like acls, as discussed earlier in this thread.
I think, what you''re trying to achive is some kind of disaster recovery
and the methods mentioned before like dd, ntfsclone and probably other image
based tools will do a better (more consistent) job on your data.

As Bart stated earlier in this thread, one could dump the acls in some kind of
controlfile, do some rdiff-backup and in case of a restore scenario, put them
back in place.
I found the tool "icacls" which should dump and later *restore* the
acls.
http://technet.microsoft.com/en-us/library/cc753525%28WS.10%29.aspx

This should bring us, together with other methods from this thread, on the way
to an incremental backup solution with rdiff and windows acl support.
I''ll give it a try...

Guido


-----Ursprüngliche Nachricht-----
Von: Paul Piscuc [mailto:paul.piscuc@appnor.com] 
Gesendet: Freitag, 11. Februar 2011 00:02
An: James Harper
Cc: Bart Coninckx; Mike Sun; Guido Hecken; xen-users@lists.xensource.com
Betreff: Re: [Xen-users] Backup running Windows machines - redundancy

Hi,

I''ll be testing the method also, for different Windows versions, with 
and without memory state backed up. One more question thought ... what 
are the performance and latency aspects of doing a xm save on a Windows 
machine? Does is save only the memory state, or other information 
regarding the state of the machine?

And one more thing we can test. What about restoring a rdiff backup(+MBR 
backup) to a fresh lvm disk ... maybe it will boot in blue screen, but 
it might work.

On 02/11/2011 12:36 AM, James Harper wrote:
>> Hi Mike,
>>
>> agreed. For situations however where data files that are typically
>> opened with user applications like Word, Excel and other, I think it
>> might be safe to just snapshot the underlying LVM while the Windows
DomU
>> is running and the users are not accessing the files (typically at
>> night), mount this and copy from it. I''m actually testing this
now. If
>> there''s interest I will post my findings.
>>
>> B.
> The Citrix Windows PV drivers include a VSS provider that (I think) allows
you to ensure that your snapshot is consistent at both a filesystem and
application level. I''ve thought about this for GPLPV but have never
actually done anything about it.
>
> James

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

No way. Rdiff will not take care of things like what you need to boot
Windows. You need dd or another imaging tool there. If you use Xen
devices on image files you can obviously easily copy them. If you use
block devices or LVM, you''re back to dd or something else.


On 02/11/11 00:01, Paul Piscuc wrote:
> Hi,
> 
> I''ll be testing the method also, for different Windows versions,
with
> and without memory state backed up. One more question thought ... what
> are the performance and latency aspects of doing a xm save on a Windows
> machine? Does is save only the memory state, or other information
> regarding the state of the machine?
> 
> And one more thing we can test. What about restoring a rdiff backup(+MBR
> backup) to a fresh lvm disk ... maybe it will boot in blue screen, but
> it might work.
> 
> On 02/11/2011 12:36 AM, James Harper wrote:
>>> Hi Mike,
>>>
>>> agreed. For situations however where data files that are typically
>>> opened with user applications like Word, Excel and other, I think
it
>>> might be safe to just snapshot the underlying LVM while the Windows
DomU
>>> is running and the users are not accessing the files (typically at
>>> night), mount this and copy from it. I''m actually testing
this now. If
>>> there''s interest I will post my findings.
>>>
>>> B.
>> The Citrix Windows PV drivers include a VSS provider that (I think)
>> allows you to ensure that your snapshot is consistent at both a
>> filesystem and application level. I''ve thought about this for
GPLPV
>> but have never actually done anything about it.
>>
>> James
> 
> 
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ok, cool. What about reinstalling the system, to rebuild the boot sector 
( or better, restore with dd from an earlier backup - let''s assume a 
week old), and then use rdiff to bring the system up to date, from a 
more recent backup ? To be tested.

Again, the purpose of this thread is to find an incremental backup 
solution. DD and other methods work fine, but you cannot do daily 
backups on 8 machines with 100GB storage, and keep those backups for a 
week or more. Also, if one would like to restore a single file from the 
image, things will get ugly.

On 02/11/2011 10:00 AM, Bart Coninckx wrote:
> No way. Rdiff will not take care of things like what you need to boot
> Windows. You need dd or another imaging tool there. If you use Xen
> devices on image files you can obviously easily copy them. If you use
> block devices or LVM, you''re back to dd or something else.
>
>
> On 02/11/11 00:01, Paul Piscuc wrote:
>> Hi,
>>
>> I''ll be testing the method also, for different Windows
versions, with
>> and without memory state backed up. One more question thought ... what
>> are the performance and latency aspects of doing a xm save on a Windows
>> machine? Does is save only the memory state, or other information
>> regarding the state of the machine?
>>
>> And one more thing we can test. What about restoring a rdiff
backup(+MBR
>> backup) to a fresh lvm disk ... maybe it will boot in blue screen, but
>> it might work.
>>
>> On 02/11/2011 12:36 AM, James Harper wrote:
>>>> Hi Mike,
>>>>
>>>> agreed. For situations however where data files that are
typically
>>>> opened with user applications like Word, Excel and other, I
think it
>>>> might be safe to just snapshot the underlying LVM while the
Windows DomU
>>>> is running and the users are not accessing the files (typically
at
>>>> night), mount this and copy from it. I''m actually
testing this now. If
>>>> there''s interest I will post my findings.
>>>>
>>>> B.
>>> The Citrix Windows PV drivers include a VSS provider that (I think)
>>> allows you to ensure that your snapshot is consistent at both a
>>> filesystem and application level. I''ve thought about this
for GPLPV
>>> but have never actually done anything about it.
>>>
>>> James
>>

-- 
Best regards,
Paul PISCUC
System Administrator - Appnor MSP S.A.
US: (+1) 650 336 57 50
RO: (+4) 021 569 46 56
http://www.appnor.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Feb 11, 2011 at 3:50 PM, Paul Piscuc <paul.piscuc@appnor.com>
wrote:
> Ok, cool. What about reinstalling the system, to rebuild the boot sector (
> or better, restore with dd from an earlier backup - let''s assume a
week
> old), and then use rdiff to bring the system up to date, from a more recent
> backup ? To be tested.
>
> Again, the purpose of this thread is to find an incremental backup
> solution. DD and other methods work fine, but you cannot do daily backups
on
> 8 machines with 100GB storage, and keep those backups for a week or more.
> Also, if one would like to restore a single file from the image, things
will
> get ugly.
>
>
The "easy" solution form your problem is zfs :)

Have you considered using a separate storage server, with
nexenta/solaris/whatever-os-that-supports-zfs, and exporting them to your
dom0 using iscsi?

-- 
Fajar


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Friday 11 February 2011 09:56:47 Fajar A. Nugraha
wrote:
> On Fri, Feb 11, 2011 at 3:50 PM, Paul Piscuc <paul.piscuc@appnor.com>
wrote:
> > Ok, cool. What about reinstalling the system, to rebuild the boot
sector
> > ( or better, restore with dd from an earlier backup - let''s
assume a
> > week old), and then use rdiff to bring the system up to date, from a
> > more recent backup ? To be tested.
> > 
> > Again, the purpose of this thread is to find an incremental backup
> > solution. DD and other methods work fine, but you cannot do daily
backups
> > on 8 machines with 100GB storage, and keep those backups for a week or
> > more. Also, if one would like to restore a single file from the image,
> > things will get ugly.
> 
> The "easy" solution form your problem is zfs :)
> 
> Have you considered using a separate storage server, with
> nexenta/solaris/whatever-os-that-supports-zfs, and exporting them to your
> dom0 using iscsi?


Fajar,

you mean because of it''s snapshotting capabilities? Would you have the
DomU
live on this filesystem and snapshot it like that? Does this work faster than 
LVM snapshots (I noticed a horrible performance drop)? And what about the 
problem of integrity?

thx!!

B.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Feb 11, 2011 at 4:19 PM, Bart Coninckx
<bart.coninckx@telenet.be>wrote:
> On Friday 11 February 2011 09:56:47 Fajar A. Nugraha wrote:
> > On Fri, Feb 11, 2011 at 3:50 PM, Paul Piscuc
<paul.piscuc@appnor.com>
> wrote:
> > > Again, the purpose of this thread is to find an incremental
backup
> > > solution. DD and other methods work fine, but you cannot do daily
> backups
> > > on 8 machines with 100GB storage, and keep those backups for a
week or
> > > more. Also, if one would like to restore a single file from the
image,
> > > things will get ugly.
> >
> > The "easy" solution form your problem is zfs :)
> >
> > Have you considered using a separate storage server, with
> > nexenta/solaris/whatever-os-that-supports-zfs, and exporting them to
your
> > dom0 using iscsi?
>
>
>
> Fajar,
>
> you mean because of it''s snapshotting capabilities?

snapshot and clone, to be exact.

> Would you have the DomU
> live on this filesystem and snapshot it like that?

domU will use zvol as a block device (instead of the usual LVM or
partition).
The zvol can be on the on the dom0 (if you''re "brave" enough
to use
zfsonlinux) or iscsi-imported from the storage server (which can run any OS
that officially support zfs).

 Does this work faster than
> LVM snapshots (I noticed a horrible performance drop)?

The performance drop will be there (compared to LVM), but it might or might
not be acceptible depending on your needs.

If you want a supported system (and possibly someone which can help you size
your system to meet your I/O performance needs) then a separate storage
using nexenta/Sun Unified Storage (or even netapp) might be better.

And what about the
> problem of integrity?
>
>
If you''re already using lvm snapshot, then the consistency/integrity is
the
same.
If the application can recover a power failure, then it can recover
it''s
data using the snapshot/clone.

-- 
Fajar


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> Ok, cool. What about reinstalling the system, to rebuild the boot sector
> ( or better, restore with dd from an earlier backup - let's assume a
> week old), and then use rdiff to bring the system up to date, from a
> more recent backup ? To be tested.
> 
> Again, the purpose of this thread is to find an incremental backup
> solution. DD and other methods work fine, but you cannot do daily
> backups on 8 machines with 100GB storage, and keep those backups for a
> week or more. Also, if one would like to restore a single file from the
> image, things will get ugly.
FWIW, I just use Bacula. It invokes a VSS snapshot and backs up from that so
backing up a running machine is not a problem. To restore, you should just be
able to mount the DomU volumes in another DomU and restore to that (bacula
allows redirection). There is nothing particularly special about making a
windows system bootable. Just format as NTFS, mark the partition as bootable
(active I think in disk manager) and then restore the files.

The only gotcha with incremental backups is that Windows can be a little lazy
about updating the 'last modified time'. I think it might only do that
when the file is closed so database files tend not to be picked up reliably.
Everything else works great though.

Given a full backup and subsequent differential/incremental backups, Bacula can
even synthesize the equivalent full backup without touching the source machine.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi Fajar,
> domU will use zvol as a block device (instead of the usual LVM or
> partition).
> The zvol can be on the on the dom0 (if you''re "brave"
enough to use
> zfsonlinux) or iscsi-imported from the storage server (which can run any OS
> that officially support zfs).
I''ve worked with ZFS a bit, but I''m confused as to how you
would use
it in this case.  How do you go from creating a ZFS volume to using it
as a raw block device?

Mike

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Feb 11, 2011 at 10:38 PM, Mike Sun <msun@gatech.edu>
wrote:
> I''ve worked with ZFS a bit, but I''m confused as to how
you would use
> it in this case.  How do you go from creating a ZFS volume to using it
> as a raw block device?
>From Zfs Administration Guide
(http://download.oracle.com/docs/cd/E19082-01/817-2271/817-2271.pdf):
A ZFS volume is a dataset that represents a block device. ZFS volumes
are identiied as devices
in the /dev/zvol/{dsk,rdsk}/pool directory.

So a zvol acts just like any block device (partition, LVM). On Linux,
see http://zfsonlinux.org/example-zvol.html. Simply use phy:/ to use
it in domU config file.

Of course, if you use a separate storage server, the dom0 will see
zvols as just another iscsi-imported block device.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Feb 11, 2011 at 4:50 AM, Mike Sun <msun@bluespot.org>
wrote:
>> I did some tests on different tools like ntfsclone, partimage and dd
through gzip.
>> In combination with LVM Snapshots they produce a backup which should
preserve all meta data since these
>> methods are not filebased.
>> One of the disadvantages are, these are not increment backup solutions,
so you will have multiple big archive
>> files.
>
> What if you used file-backed sparse images as the VM''s virtual
disk
> and then used rsync/rdiff-backup on that file itself?  That way, you
> don''t have to worry about rsync/rdiff-backup being aware of NTFS
> metadata, but still take advantage of the incremental binary diffs
> that rsync/rdiff-backup provide at the file level.
>
> Is that a feasible solution?
Assuming:
- the image is on LVM and a snapshot is taken when doing backup
- rsync is invoked with -S
- the image is used on Xen using tap:aoi (or preferably the new
tap2:tapdisk:aio)

then it might be feasible.

On the other hand I just found this link:
http://serverfault.com/questions/27397/sync-lvm-snapshots-to-backup-server
The script on that page looks promising, as well as ddsnap.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users