Hi all,
I''m doing my first xen,
running with 3
domU perfect.
I need to know which way to
backup domU to restore these
to another server if i''ve problem
with this machine.
There
are 3 LVM with 10G
in VolGroup01 for each domU, with each
of these partitions:
/dev/xvda3 8,7G 1,4G 6,9G 17% /
/dev/xvda1 99M 14M 81M 14% /boot
/dev/svda2 swap
tmpfs 256M 0 256M 0% /dev/shm
I need to know how easiest method for
backup these images
and restore them.
Thank you all
Sorry for my english....
-Kalz-
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Hi, Kalil Costa - Brasilsite wrote:> I''m doing my first xen, running with 3 domU perfect. > > I need to know which way to backup domU to restore these to another > server if i''ve problem with this machine.What I did was create a script that takes each machine in turn. It shuts down the machine, creates an lvm snapshot and then immediately restarts the machine. I backup off the snapshot, nice and clean, then destroy the snapshot when finished. This process makes my DomU unavailable for a very short period of time whilst I get a full and complete, consistent backup of the DomU. -- Kind Regards AndrewM Andrew McGlashan Broadband Solutions now including VoIP _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andrew, thanks you could spend the steps of how did backup ?> Hi, > > Kalil Costa - Brasilsite wrote: >> I''m doing my first xen, running with 3 domU perfect. >> >> I need to know which way to backup domU to restore these to another >> server if i''ve problem with this machine. > > What I did was create a script that takes each machine in turn. > > It shuts down the machine, creates an lvm snapshot and then immediately > restarts the machine. > > I backup off the snapshot, nice and clean, then destroy the snapshot > when finished. > > This process makes my DomU unavailable for a very short period of time > whilst I get a full and complete, consistent backup of the DomU. > > -- > Kind Regards > AndrewM > > Andrew McGlashan > Broadband Solutions now including VoIP > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Kalil Costa - Brasilsite wrote:>I need to know which way to backup domU to restore these to another >server if i''ve problem with this machine.There are many ways, all with their pros and cons. One suggestion given here in the past was to send a signal (with an xm command IIRC) to the guest (assuming guest with Xen support included) to tell the guest to sync it''s unwritten buffers to disk. He then did a snapshot live via LVM. Upside - no downtime on guest. Downside, your backups are of a mounted filesystem with open files, partially written files, whatever. Getting the guest to sync first reduces the impact so it''s not quite the same as the backup being analogous to what you''d find on disk after pulling the power cord on a real machine. The backup is also on the same disks as the live system. You''ve just had a suggestion of stopping the guest before the LVM snapshot. Upside - it''s a ''clean'' backup. Downsides - downtime for guest, backup still on the same disks. As a variation, you could pause the guest and snapshot the LVM volumes **PLUS** save the guest saved state file. If you need to restore then the guest would unpause in the same state as when it was paused. Personally I backup my guests as though they were real (non virtualised) machines. There are many options for this - both free and commercial. At work I''ve setup a system where I have a VM dedicated just to holding backups of the other machines - each of which uses rsync to update a backup copy of itself on the backup server (the server runs rsync in server mode). Thus I have a server holding a complete image of each of my servers at the point they last backed up. Should a host go down, I can move the guests to another host by creating volumes for them and using rsync to pull their files back (mount the guest filesystems on the host, use rsync on the host to pull the files, unmount the filesystems and start the guest). Of course, once you are using rsync, then it doesnt'' matter whether the destination is on the same host, another host in the same rack, or half way round the world - as long as you have enough bandwidth. On my backup machine I then copy the copies to create various levels of historical backups. Again there are various ways of doing this, I settled on StoreBackup which if you disable compression creates full copies which you can just navigate into and use your normal unix/linux tools to access files*. It saves space by hard-linking identical files so it''s fairly efficient. You can do something similar with rsync and some scripting. Another tool I looked at was rdiff-backup - but I didn''t like the way you can''t thin out your backups, and they aren''t readable without going through the restore process. * Just the other day I found this useful as I wanted to find out when a DNS record had changed. I was able to grep <something>/*/var/named/zones/<somezonefile> for the name in question and find out that it changed about a month ago - yes no-one had noticed a service was broken ! -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 09/07/2011 08:54, Simon Hobson wrote:> Kalil Costa - Brasilsite wrote: > > The backup is also on the same disks as the live system.I don''t know why you make this point, as, at least in this thread, the target of the backup is not mentioned and is trivial to have it point to another system.> Personally I backup my guests as though they were real (non > virtualised) machines. There are many options for this - both free and > commercial. > > At work I''ve setup a system where I have a VM dedicated just to > holding backups of the other machines - each of which uses rsync to > update a backup copy of itself on the backup server (the server runs > rsync in server mode). Thus I have a server holding a complete image > of each of my servers at the point they last backed up. Should a host > go down, I can move the guests to another host by creating volumes for > them and using rsync to pull their files back (mount the guest > filesystems on the host, use rsync on the host to pull the files, > unmount the filesystems and start the guest). > Of course, once you are using rsync, then it doesnt'' matter whether > the destination is on the same host, another host in the same rack, or > half way round the world - as long as you have enough bandwidth. > > On my backup machine I then copy the copies to create various levels > of historical backups. Again there are various ways of doing this, I > settled on StoreBackup which if you disable compression creates full > copies which you can just navigate into and use your normal unix/linux > tools to access files*. It saves space by hard-linking identical files > so it''s fairly efficient. > You can do something similar with rsync and some scripting. Another > tool I looked at was rdiff-backup - but I didn''t like the way you > can''t thin out your backups, and they aren''t readable without going > through the restore process.A perfect tool for this is backuppc. http://backuppc.sourceforge.net/ Quick summary of benefits, - rsync, smb, ftp support - uses hardlinks to create incremental backups, - reduces disk space further by compressing and only storing one copy of each file across all backups (generates hashes of new files nightly to determine duplicates) - We have 29 hosts backed up nightly, with historical backups of some going back 13 months. Available data in full: 491GB, actual stored space: 65GB - full set of cmdline tools to manipulate backups - restore is trivially easy - web interface gives statistics - retries failed attempts - mails you if a machine is not backed up for X hours. - can give user web access to just their own backups (we don''t use this though, I beleive no web interface is safe) -- May the ping be with you .. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Steve Allison wrote:>>The backup is also on the same disks as the live system. > >I don''t know why you make this point, as, at least in this thread, >the target of the backup is not mentioned and is trivial to have it >point to another system.I wasn''t aware LVM could create a snapshot which puts a complete copy of the volume on another disk. In fact, AUIU, a snapshot doesn''t actually copy the whole volume, it creates a COW setup which saves any blocks subsequently written to. If I''m wrong then I''m happy to be corrected.>A perfect tool for this is backuppc. http://backuppc.sourceforge.net/Like I say, there are **many** tools for the job. But thanks for the pointer, I''ll take a look at that sometime and see if it''s better than what I use now. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Ursprüngliche Nachricht----- > Von: xen-users-bounces@lists.xensource.com [mailto:xen-users- > bounces@lists.xensource.com] Im Auftrag von Steve Allison > Gesendet: Samstag, 9. Juli 2011 11:27 > An: xen-users@lists.xensource.com > Betreff: Re: [Xen-users] Backup domU > A perfect tool for this is backuppc. http://backuppc.sourceforge.net/ > > Quick summary of benefits, > - rsync, smb, ftp support > - uses hardlinks to create incremental backups, > - reduces disk space further by compressing and only storing one copy > of each file across all backups (generates hashes of new files nightly > to determine duplicates) > - We have 29 hosts backed up nightly, with historical backups of some > going back 13 months. Available data in full: 491GB, actual stored > space: 65GB > - full set of cmdline tools to manipulate backups > - restore is trivially easy > - web interface gives statistics > - retries failed attempts > - mails you if a machine is not backed up for X hours. > - can give user web access to just their own backups (we don''t use > this though, I beleive no web interface is safe)I agree, BackupPc does a good job, once installed and correctly configured, you can forget about it and simple let it do it''s job. :-) IMHO, you have to break every backup scenario regardless of it is a virtualized or a real machine, into two pieces, Disaster Recovery (DR) and normal file- and/or databasebackup. With BackupPc you "only" get a good filebackup solution. While using LVM one could do snapshots in the sense of DR, and BackupPc with some additional scripts (dump databases, stop certain services etc.). As others already posted in this thread, there a lot''s of additional options, one could use to achieve the goal of a perfect backupsolution. Perhaps, in question of DR also have a look at Rear http://rear.sourceforge.net/. If money or perhaps "disposition" is no object, look at the commercial acronis backup&recovery solutions. --- Guido _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, Simon Hobson wrote:> Kalil Costa - Brasilsite wrote: > >> I need to know which way to backup domU to restore these to another >> server if i''ve problem with this machine. > > There are many ways, all with their pros and cons.True.> One suggestion given here in the past was to send a signal (with an xm > command IIRC) to the guest (assuming guest with Xen support included) to > tell the guest to sync it''s unwritten buffers to disk. He then did a > snapshot live via LVM.Interesting, but still not clean enough for me.> Upside - no downtime on guest. Downside, your backups are of a mounted > filesystem with open files, partially written files, whatever. Getting > the guest to sync first reduces the impact so it''s not quite the same as > the backup being analogous to what you''d find on disk after pulling the > power cord on a real machine. > The backup is also on the same disks as the live system.No, it is not on the same disk [as already mentioned] -- actually, rsync is used with the lvm snapshot and better still it uses rsnapshot to have historical versions -- many restore points. Then the rsanpshot is rsynced off-site with again, you guessed it, rsnapshot using rsync. It''s complicated and works very well, but the downtime is so minimal to me that it isn''t an issue. It could probably be expanded upon using multi-mysql master or similar, then shutting down one of the masters, but not shutting down the main server instance.> As a variation, you could pause the guest and snapshot the LVM volumes > **PLUS** save the guest saved state file. If you need to restore then > the guest would unpause in the same state as when it was paused.Interesting too, but you still have a "running" system backup, although it might be fine doing that.> At work I''ve setup a system where I have a VM dedicated just to holding > backups of the other machines - each of which uses rsync to update a > backup copy of itself on the backup server (the server runs rsync in > server mode). Thus I have a server holding a complete image of each of > my servers at the point they last backed up. Should a host go down, I > can move the guests to another host by creating volumes for them and > using rsync to pull their files back (mount the guest filesystems on the > host, use rsync on the host to pull the files, unmount the filesystems > and start the guest). > Of course, once you are using rsync, then it doesnt'' matter whether the > destination is on the same host, another host in the same rack, or half > way round the world - as long as you have enough bandwidth.The only issue with rsync with rsnapshot from what I can see, is for very large files that change frequently. You might end up having loads of copies of these large files and that would be painful in terms of storage and transfer. Take a 2TB Oracle datafile for instance; don''t want to touch that right now with this method, but Oracle has other backup methods anyway .....> On my backup machine I then copy the copies to create various levels of > historical backups. Again there are various ways of doing this, I > settled on StoreBackup which if you disable compression creates full > copies which you can just navigate into and use your normal unix/linux > tools to access files*. It saves space by hard-linking identical files > so it''s fairly efficient.Sounds exactly like rsnapshot does. Get the flags right for rsync and you don''t need to re-invent anything. I''m also selectively using .rsync-filter as well for files / dirs that I don''t require such full backups of. This is an interesting feature of StoreBackup: "recognizes when files have been copied, moved or renamed and does not waste time or space to duplicate the backup of such files" I''m considering these ideas too, it would help with maildir files that only change the dir they live in or the flags at the end of the file name. Duplicates .... hmmm, another whole area, md5 every file being backed up, check if you have the md5 on record -- then binary compare the two files and if they really are absolutely identical, hard link them and only keep one copy. But lots more work for the backup job / tool to consider.> * Just the other day I found this useful as I wanted to find out when a > DNS record had changed. I was able to grep > <something>/*/var/named/zones/<somezonefile> for the name in question > and find out that it changed about a month ago - yes no-one had noticed > a service was broken !Yep, can do that fine with rsnapshot backups too. See all versions kept of a file then vim -d any variants that are interesting. -- Kind Regards AndrewM Andrew McGlashan Broadband Solutions now including VoIP _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Am 09.07.2011 um 12:17 schrieb Andrew McGlashan:> The only issue with rsync with rsnapshot from what I can see, is for very large files that change frequently. You might end up having loads of copies of these large files and that would be painful in terms of storage and transfer. Take a 2TB Oracle datafile for instance; don''t want to touch that right now with this method, but Oracle has other backup methods anyway .....Yes, rsnapshot does deduplication by using hard links. This is file based deduplication; what we really want is deduplication on block level, like e.g. ZFS does. For Linux, I came across lessfs (http://sourceforge.net/projects/lessfs/files/). Anyone here who can provide experiences with lessfs? Rainer _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Rainer Sokoll wrote:> Am 09.07.2011 um 12:17 schrieb Andrew McGlashan: > >> The only issue with rsync with rsnapshot from what I can see, is for very large files that change frequently. You might end up having loads of copies of these large files and that would be painful in terms of storage and transfer. Take a 2TB Oracle datafile for instance; don''t want to touch that right now with this method, but Oracle has other backup methods anyway ..... > > Yes, rsnapshot does deduplication by using hard links. This is file based deduplication; what we really want is deduplication on block level, like e.g. ZFS does. > For Linux, I came across lessfs (http://sourceforge.net/projects/lessfs/files/). > Anyone here who can provide experiences with lessfs?If the file name changes whilst none if it''s content does ... or the file''s location changes, then rsnapshot doesn''t de-duplicate those files. Even files that are very slightly modified mean a whole new copy of them is created (no hard link). Of course the old copies that are no longer on the system being backed up will "age out" via the cycling of rsnapshot directories over time. -- Kind Regards AndrewM Andrew McGlashan Broadband Solutions now including VoIP _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 09/07/2011 10:42, Simon Hobson wrote:> Steve Allison wrote: > >>> The backup is also on the same disks as the live system. >> >> I don''t know why you make this point, as, at least in this thread, >> the target of the backup is not mentioned and is trivial to have it >> point to another system. > > I wasn''t aware LVM could create a snapshot which puts a complete copy > of the volume on another disk. In fact, AUIU, a snapshot doesn''t > actually copy the whole volume, it creates a COW setup which saves any > blocks subsequently written to. >Exactly. So you put the LVM into a snapshot state, and "backup off the snapshot, nice and clean then destroy the snapshot when finished". -- May the ping be with you .. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 09/07/2011 21:58, Steve Allison wrote:> On 09/07/2011 10:42, Simon Hobson wrote: >> Steve Allison wrote: >> >>>> The backup is also on the same disks as the live system. >>> >>> I don''t know why you make this point, as, at least in this thread, >>> the target of the backup is not mentioned and is trivial to have it >>> point to another system. >> >> I wasn''t aware LVM could create a snapshot which puts a complete copy >> of the volume on another disk. In fact, AUIU, a snapshot doesn''t >> actually copy the whole volume, it creates a COW setup which saves any >> blocks subsequently written to. >> > > Exactly. So you put the LVM into a snapshot state, and "backup off the > snapshot, nice and clean then destroy the snapshot when finished". >To be honest, IMO, I think the best method is a mixed solution: 1) Create snapshot-based backups as described above (using rsync, rsnapshot, dd, cp, or whatever). This will take care of most files OK, and will certainly allow for a DomU to be booted up ok in the event of a system crash. and 2) have a seperate solution for applications that require your filesystem to be in a consistent state - Simply making a snapshot as in 1) doesn''t provide this guarantee. Good options here are the tools that come with most good database applications (pgdump for PostgreSQL is an easy choice). By creating snapshot backups alone (as in 1)), unless your DomU has been cleanly shutdown beforehand, will never guarantee a consistent filesystem. Expect to run "e2fsck -f" before using such a backup again. At least that''s my grasp on the subject... _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Steve Allison wrote:>Exactly. So you put the LVM into a snapshot state, and "backup off >the snapshot, nice and clean then destroy the snapshot when >finished".Sorry, I''d missed that detail :-( -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi all, Why my question ... see the problem I Use LVM on Domain0 and create an LVM for each domU, the installation also create partitions in LVM. When I try to access a partition for backup domU can''t, so do i create a physical partition /boot and the root ''/'' in LVM and it can''t mount the partition in domainU for backup. Could this be the best way to work with LVM in domU or is it better to create the physical partitions in domU /dev /xvda ''etc. ..? How do you do?> > Am 09.07.2011 um 12:17 schrieb Andrew McGlashan: > >> The only issue with rsync with rsnapshot from what I can see, is for >> very large files that change frequently. You might end up having loads >> of copies of these large files and that would be painful in terms of >> storage and transfer. Take a 2TB Oracle datafile for instance; don''t >> want to touch that right now with this method, but Oracle has other >> backup methods anyway ..... > > Yes, rsnapshot does deduplication by using hard links. This is file based > deduplication; what we really want is deduplication on block level, like > e.g. ZFS does. > For Linux, I came across lessfs > (http://sourceforge.net/projects/lessfs/files/). > Anyone here who can provide experiences with lessfs? > > Rainer > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I Use LVM on Domain0 and create an LVM for each domU, the installation > also create partitions in LVM.User kpartx before using rsync (or whatever tool you''ve decided to use). kpartx allows you to mount partitions within LVM LVs _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
OK thks So one question, i need backup ''/boot'' too or only root ''/'' ? I have three partitions /boot, / and swap. thks again kalz> >> I Use LVM on Domain0 and create an LVM for each domU, the installation >> also create partitions in LVM. > > User kpartx before using rsync (or whatever tool you''ve decided to use). > kpartx allows you to mount partitions within LVM LVs > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Kalil Costa - Brasilsite wrote:>So one question, i need backup ''/boot'' too or only root ''/'' ? > >I have three partitions /boot, / and swap.Depends on your needs and how much work you want to be doing if/when you need to use your backups. Normally, for minimum fuss you''d need to back all your filesystems (ie both / and /boot in this case). If you don''t backup /boot then after a recovery you''ll find yourself having to do some manual steps before you have a bootable system again. But at least /boot doesn''t change much or often. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, 2011-07-09 at 08:54 +0100, Simon Hobson wrote:> > One suggestion given here in the past was to send a signal (with an > xm command IIRC) to the guest (assuming guest with Xen support > included) to tell the guest to sync it''s unwritten buffers to disk. > He then did a snapshot live via LVM.I have been exploring this myself. I started out by pausing the guest and copying its image. That of course gives one of those semi-corrupted images that you would get from pulling the plug on a real machine, plus it causes a temporary service outage on the guest. To avoid the service outages, I switched to using an LVM snapshot (I had to rearrange a lot of things to make this work, as you have to have extra space on the LVM volume to allow for snapshots and I didn''t have this initially). That works and eliminates the service outage on the guest, but still produces a semi-corrupted image. So after reading this (and taking a vacation), I did some more reading and discovered that using "xm sysrq name-of-guest s" will cause the guest to sync, assuming that the guest has been set up to allow this (kernel.sysrq = 16 in the sysctl.conf file). So I tried syncing the guest this way and then immediately taking a snapshot: xm sysrq name-of-guest s; vm snap ("vm" is a script I wrote that sets up and mounts the snapshot, among many other possible operations). This is going to create the snapshot as soon after the sync as is realistically possible. The /var/log/messages file on the guest shows that the sync did occur. But if I run ''file'' on the guest image from the snapshot, it still says that the ext3 file system "needs journal recovery". So I am concerned that I may still be getting a semi-corrupted image. Is there a way to be certain that the image is clean? Is there a way to force the guest to run the journal as well as syncing? --Greg _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Jul 20, 2011 at 10:59 AM, Greg Woods <woods@ucar.edu> wrote:> But if I run ''file'' on the guest image from the snapshot, it still says > that the ext3 file system "needs journal recovery".sync occurred, but the filesystem was still mounted at the snapshot time. also, there''s still the issue of application level caches. AFAIK, glibc doesn''t flush on every write. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Greg Woods wrote:>So after reading this (and taking a vacation), I did some more reading >and discovered that using "xm sysrq name-of-guest s" will cause the >guest to sync, assuming that the guest has been set up to allow this >(kernel.sysrq = 16 in the sysctl.conf file). So I tried syncing the >guest this way and then immediately taking a snapshot: > >xm sysrq name-of-guest s; vm snap > >("vm" is a script I wrote that sets up and mounts the snapshot, among >many other possible operations). This is going to create the snapshot as >soon after the sync as is realistically possible. > >The /var/log/messages file on the guest shows that the sync did occur. >But if I run ''file'' on the guest image from the snapshot, it still says >that the ext3 file system "needs journal recovery". So I am concerned >that I may still be getting a semi-corrupted image. Is there a way to be >certain that the image is clean? Is there a way to force the guest to >run the journal as well as syncing?The only way to get a truly clean filesystem would be to close all files and unmount the volume (or at least remount it read only). As long as it''s mounted r/w, then it''s going to be dirty as far as any image/snapshot taken of it is concerned. And as Javier Guerra Giraldez says, you still have the issue of what any applications might not have actually written to disk - if it''s still in the application cache then it''s not going to be written by an OS level sync. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Greg, On Wed, Jul 20, 2011 at 09:59:59AM -0600, Greg Woods wrote:> The /var/log/messages file on the guest shows that the sync did > occur. But if I run ''file'' on the guest image from the snapshot, > it still says that the ext3 file system "needs journal recovery".The filesystem journal will always need recovery unless the filesystem was cleanly unmounted (by the guest).> So I am concerned that I may still be getting a semi-corrupted > image. Is there a way to be certain that the image is clean? Is > there a way to force the guest to run the journal as well as > syncing?Not without shutting it all down cleanly. As others have pointed out, there may well also be applications in your guest holding data in their memory that they haven''t yet flushed to disk; sync isn''t going to make them do it. Think about relational databases and the like. In any backup strategy (for virtualised hosts or those running on bare metal) if you want it to be the best it can be then you need to take applications into account. For example for a relational database this may mean periodically taking a lock and dumping out the data to a file which is later backed up by your regular backup process. Such a dump file would have a consistent point-in-time view of your data, unlike trying to back up the raw data files of the database. As the operator of hardware with virtualised guests on it you may not have that level of insight into just what applications the guests are running, in which case the administrator of the guest itself needs to care about that. Your scheme of snapshotting the filesystem may be the best you can offer without knowing what applications are running. It is worth considering running a backup scheme for virtual hosts that is the same as for any other host, just for the win of being able to generalise the process. Cheers, Andy _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andy Smith wrote:>It is worth considering running a backup scheme for virtual hosts >that is the same as for any other host, just for the win of being >able to generalise the process.That''s what I do - each guest does it''s own backups using the same script that all the non-virtualised machines use. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, 2011-07-21 at 07:50 +0100, Simon Hobson wrote:> Andy Smith wrote: > > >It is worth considering running a backup scheme for virtual hosts > >that is the same as for any other host, just for the win of being > >able to generalise the process. > > That''s what I do - each guest does it''s own backups using the same > script that all the non-virtualised machines use.I''ll have to think about this some. It appears there is no way to get a completely clean image without a service outage; I was afraid of that, but I wanted to be sure. The trouble with backing up the guests the same way as the "real" machines is that it removes one of the great advantages of virtualization. Now I have to install and configure the backup software 100 times, once for each guest (yes I know there are ways of at least partially automating this), instead of something that can be done maybe half a dozen times on my host OS''s. I may choose to just continue with the snapshot backups, understanding that there is a slight possibility that the file I need to recover might not be available on a given day''s backup and I might have to resort to an older copy. In my case, the applications being run are mostly simpler things like DNS and SMTP servers, rather than complex things like databases, so the risk that the applications may be holding data in memory is much less. --Greg _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Greg Woods wrote:>The trouble with backing up the guests the same way as the "real" >machines is that it removes one of the great advantages of >virtualization.Personally, I''ve never seen that as one of the advantages of virtualisation - but that''s a separate debate ;-) -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Reading this thread I wonder about one thing: What happens with the system time in a resumed domU? I have never tried that but I''m going to soon (tomorrow when I get back to work). See, my domUs and dom0 have xen.independent_clock = 0 so that I only run ntp in my dom0 and all domUs (pv) keep very (not exactly) synced. So If I resume a domU with this configuration which had been backed up a couple of days ago, then the domU has the "ilusion" of time having been changed by hand (like with date -s) because of this? This could be disastrous to a DBMS (any!) which in the precise moment of backup was running a silly system query (e.g., a mantainence procedure, vacuuming, etc.). Even something as innocent as that could be dangerous. You know how important is time to DBMSs. And in the case the time keeps going normally from the point it stopped, then that domU would have to have set xen.independent_clock 1, and thus any access to the DBMS could be disastrous as well. You couldn''t connect to it using SSL. Just picture it. I mean, it shouldn''t be the end of the world but when you *have* to use a backup ($hit happens) you want things to go back to normal as fast as it is possible, without worrying about this kind of things. We run multiple DBMSs so that is a concern to us. If anyone faces a scenario like that, it may be better to try other options. If an OS backup then choose the cold backup. If DBMS data backup make a dump. And if availability needed then try HA tools (like Slony-I for PostgreSQL), that would allow you to stop one server, back it up, then the other. Cheers 2011/7/21, Simon Hobson <linux@thehobsons.co.uk>:> Greg Woods wrote: > >>The trouble with backing up the guests the same way as the "real" >>machines is that it removes one of the great advantages of >>virtualization. > > Personally, I''ve never seen that as one of the advantages of > virtualisation - but that''s a separate debate ;-) > -- > Simon Hobson > > Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed > author Gladys Hobson. Novels - poetry - short stories - ideal as > Christmas stocking fillers. Some available as e-books. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- Diego Augusto Molina diegoaugustomolina@gmail.com ES: Por favor, evite adjuntar documentos de Microsoft Office. Serán desestimados. EN: Please, avoid attaching Microsoft Office documents. They shall be discarded. LINK: http://www.gnu.org/philosophy/no-word-attachments.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users