Hello! I''m running xen 3.4.1, I''ve a partition with lots of virtual machines mount at /vserver/images/domains. Inside that folder I''ve all the img disk (created with dd) of each virtual machine. # ls /vserver/images/domains # win_xp.img win7.img ubuntu.img debian.img w2008.img gentoo.img Could I run a cp or rdiff-backup over /vserver/images/domains when these virtual machines are running? Thanks in advance, Alberto _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Fri, Mar 9, 2012 at 2:50 PM, Alberto Asuero <albertoasuero@gmail.com> wrote:> Hello! > > I''m running xen 3.4.1, I''ve a partition with lots of virtual machines mount > at /vserver/images/domains. Inside that folder I''ve all the img disk > (created with dd) of each virtual machine. > > # ls /vserver/images/domains > > # win_xp.img win7.img ubuntu.img debian.img w2008.img gentoo.img > > Could I run a cp or rdiff-backup over /vserver/images/domains when these > virtual machines are running?No. At the very least, you need to either: - pause the vms first (xm pause ..., xm resume ...), or - if the "partition" is on LVM, create an LVM snapshot first, mount it, and THEN back it up (don''t forget do delete the snapshot afterwards). -- Fajar
Yes, It has worked fine for me. There may be some danger though. Centos handles it fine. From: xen-users-bounces@lists.xen.org [mailto:xen-users-bounces@lists.xen.org] On Behalf Of Alberto Asuero Sent: Thursday, March 08, 2012 11:50 PM To: xen-users@lists.xen.org Subject: [Xen-users] Backup domU image at runtime Hello! I''m running xen 3.4.1, I''ve a partition with lots of virtual machines mount at /vserver/images/domains. Inside that folder I''ve all the img disk (created with dd) of each virtual machine. # ls /vserver/images/domains # win_xp.img win7.img ubuntu.img debian.img w2008.img gentoo.img Could I run a cp or rdiff-backup over /vserver/images/domains when these virtual machines are running? Thanks in advance, Alberto _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
OK, Thank you very much! And how could I make a backup of a whole domU at runtime? is that possible? Now what I''m doing is backing up the files of domU with rdiff-backup but I''d like a system that allow to restore a domU quickly (now I''ve to reinstall the whole SO and restore the files and applications) Thanks! On Fri, Mar 9, 2012 at 9:17 AM, Fajar A. Nugraha <list@fajar.net> wrote:> On Fri, Mar 9, 2012 at 2:50 PM, Alberto Asuero <albertoasuero@gmail.com> > wrote: > > Hello! > > > > I''m running xen 3.4.1, I''ve a partition with lots of virtual machines > mount > > at /vserver/images/domains. Inside that folder I''ve all the img disk > > (created with dd) of each virtual machine. > > > > # ls /vserver/images/domains > > > > # win_xp.img win7.img ubuntu.img debian.img w2008.img gentoo.img > > > > Could I run a cp or rdiff-backup over /vserver/images/domains when these > > virtual machines are running? > > No. > > At the very least, you need to either: > - pause the vms first (xm pause ..., xm resume ...), or > - if the "partition" is on LVM, create an LVM snapshot first, mount > it, and THEN back it up (don''t forget do delete the snapshot > afterwards). > > -- > Fajar >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Todd H. Foster wrote:>Yes, It has worked fine for me. There may be some danger though. >Centos handles it fine.Not "may be some", but "there is a definite" danger. If you make a backup of the virtual disk (by whatever means) from teh Som0 while a DomU is running then what you copy will be the equivalent of what you''d have on a real machine if you just yanked the power cord. Ie, you''d not include anything cached in the machine''s memory and not yet written to disk. That''s not a theoretical risk, it is a most definite and demonstrable problem. How big of an issue it is to you depends very much on what the virtual machine is doing, how it''s set up, what other measures you have in place, and what your attitude is to risk. At one extreme, if you have a guest with little disk I/O (particularly writes), uses journalled filesystems, and you trigger the guest to flush it''s write cache* before making a snapshot - then the risk would be low. At the other extreme, if you had a very active database, then the risk of a corrupted database is quite high. * IIRC from a previous discussion of the topic, there''s a command you can give from Dom0 to tell the DomU to flush it''s cache. What you really, really, really cannot do is mount one volume in two places (eg DomU and Dom0) - unless you take the right precautions. That may mean mounting it read-only in all places (generally not very useful), or it may mean running a clustered filesystem of some sort. If you try and mount a volume in two places, with a "standard" filesystem, then you are pretty well guaranteed a completely trashed filesystem - I did it once by accident**, it wasn''t pretty or recoverable. ** I was investigating drivers for a Dell RAID card - for this card there was a choice of 2. One was loaded, and when I loaded the other I had two drivers working the same disk - I just wiped it and started again. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books.
On Fri, Mar 9, 2012 at 4:19 PM, Alberto Asuero <albertoasuero@gmail.com> wrote:> OK, Thank you very much! > > And how could I make a backup of a whole domU at runtime? is that possible?Yes. Read my previous reply (hint: snapshot) -- Fajar
Alberto Asuero wrote:>And how could I make a backup of a whole domU at runtime? is that possible?You would have to pause the DomU and image both it''s disk(s) image(s) **AND** it''s current state. In the event of restoring such a backup, you''d kill the guest, restore the image, and unpause the restored image - the guest would continue from the exact state it was in when it was paused before the backup. Your only easy option with this type of backup is to restore the entire machine & state to a point in time. It''s hard to retrieve a single file - you''d need to restore the whole virtual machine to a temporary location, unpause it without letting it connect to the live network, pull the files you want, and then destroy it.>Now what I''m doing is backing up the files of domU with rdiff-backup >but I''d like a system that allow to restore a domU quickly (now I''ve >to reinstall the whole SO and restore the files and applications)I use Rsync from within each guest to back their entire filesystems to a shared space. From there I use StoreBackup to make multiple generations. In a recovery situation, I can restore any individual machine to it''s last backup with the following steps : In Dom0, create the LVM volumes, create filesystems, mount them. Rsync the files back from the shared backup space. Unmount the volumes from Dom0 Startup the guest. If I''ve had to move the guests (eg host gone tits-up), then I can pull the guest config files from the backup space (hosts are backed up here as well) to another host. That''s only one wya of doing it. If I need to access an old version of a file, then that can be pulled from the StoreBackup archives. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books.
2012/3/9 Simon Hobson <linux@thehobsons.co.uk>> Todd H. Foster wrote: > > Yes, It has worked fine for me. There may be some danger though. Centos >> handles it fine. >> > > Not "may be some", but "there is a definite" danger. > > If you make a backup of the virtual disk (by whatever means) from teh Som0 > while a DomU is running then what you copy will be the equivalent of what > you''d have on a real machine if you just yanked the power cord. Ie, you''d > not include anything cached in the machine''s memory and not yet written to > disk. > >> That''s not a theoretical risk, it is a most definite and demonstrable > problem. > >I think that taking a backup of a virtual disk while it is used is even worse: if you yank the power cord and you are using a journaled file system, the file system will probably recover by itself. If you backup a virtual disk while it is used you are taking an inconsistent image of the journal and an inconsistent (corrupted) image of the files!> > At one extreme, if you have a guest with little disk I/O (particularly > writes), uses journalled filesystems, and you trigger the guest to flush > it''s write cache* before making a snapshot - then the risk would be low. > >I think that the risk will always be high: you cannot be guaranteed that no process will write any file during the backup, and you have no way to check if the backup is working (the os will see the filesystem as ok, but application data could still be corrupted) Andrea _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Andrea Monti wrote:>I think that taking a backup of a virtual disk while it is used is >even worse: if you yank the power cord and you are using a journaled >file system, the file system will probably recover by itself. >If you backup a virtual disk while it is used you are taking an >inconsistent image of the journal and an inconsistent (corrupted) >image of the files!If you do a "file copy" then yes, the whole thing will be inconsistent. If you use something like an LVM snapshot then it would be a "point in time" copy and equivalent to yanking the power cord.>I think that the risk will always be high: you cannot be guaranteed >that no process will write any file during the backupSee above about snapshots.>and you have no way to check if the backup is working (the os will >see the filesystem as ok, but application data could still be >corrupted)Indeed. Which is why I specifically said it depends heavily on what the guest is doing. Databases in particular would be a problem as most DB engines do a lot of their own caching. Simplest advice on such backups is ... DON''T ! Less simple is ... DON''T unless you understand the nuances enough to determine that it''s safe for **your** situation. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books.
Todd H. Foster wrote: Yes. Read my previous reply (hint: snapshot) I don''t have LVM on this partition, I''ve a big RAID 5 (I''ll try to migrate to LVM) Simon Hobson wrote: In a recovery situation, I can restore any individual machine to it''s last> backup with the following steps :In Dom0, create the LVM volumes, create filesystems, mount them.> Rsync the files back from the shared backup space. > Unmount the volumes from Dom0 > Startup the guest.It''d perfect but I can''t pause certains guests it''s a server of high availability... I think my best options is : 1) Migrate to LVM and I''ll try with snapshot 2) But I''ll also keep the backup of files of each domU. With the option 2 (more security), I''ll keep safe from a posible power supplies interruption (I''ve a UPS and a powervault redundance) Thank you guys! *- - - Alberto Asuero Arroyo* Software Engineer (+34) 645 816 025 On Fri, Mar 9, 2012 at 11:13 AM, Andrea Monti <ilsuonogiallo@gmail.com>wrote:> > > 2012/3/9 Simon Hobson <linux@thehobsons.co.uk> > >> Todd H. Foster wrote: >> >> Yes, It has worked fine for me. There may be some danger though. Centos >>> handles it fine. >>> >> >> Not "may be some", but "there is a definite" danger. >> >> If you make a backup of the virtual disk (by whatever means) from teh >> Som0 while a DomU is running then what you copy will be the equivalent of >> what you''d have on a real machine if you just yanked the power cord. Ie, >> you''d not include anything cached in the machine''s memory and not yet >> written to disk. >> >> > >> That''s not a theoretical risk, it is a most definite and demonstrable >> problem. >> >> > > I think that taking a backup of a virtual disk while it is used is even > worse: if you yank the power cord and you are using a journaled file > system, the file system will probably recover by itself. > If you backup a virtual disk while it is used you are taking an > inconsistent image of the journal and an inconsistent (corrupted) image of > the files! > > > >> >> At one extreme, if you have a guest with little disk I/O (particularly >> writes), uses journalled filesystems, and you trigger the guest to flush >> it''s write cache* before making a snapshot - then the risk would be low. >> >> > I think that the risk will always be high: you cannot be guaranteed that > no process will write any file during the backup, and you have no way to > check if the backup is working (the os will see the filesystem as ok, but > application data could still be corrupted) > > Andrea > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hello Here what we have ben doing here for a few years. We have about 20 VMs on several physical servers. Some of those VMs have a heavy traffic with large disks (>100Gb) ("big machines"), while other machines ("little machines") only provide important - but not too much sollicited - services with little system disks (10Gb). For the former we installed locally a client backup software on the VM, but for the "little machines" we snapshot through lvm, mount the snapshot, then the backup software running on the phyical server saves the snapshot. If the traffic is heavy you''ll have a lot of opened files and the snapshot may be difficult, if not impossible, to mount (a fsck of the snapshot may help), so this solution may not be used (but as we get an error message from the backup software, we know it''s time to switch to a local backup solution). If the machine is just quiet, this solution may be used without any problem, and this avoids paying for too many software license (our backup solution is not free). So to summarize, saving through a backup software running on the VM is the best solution, but saving through an lvm snapshot can help. -- Emmanuel COURCELLE emmanuel.courcelle@toulouse.inra.fr L.I.P.M. (UMR CNRS-INRA 2594/441) tel (33) 5-61-28-54-50 B.P.52627 - 31326 CASTANET TOLOSAN Cedex - FRANCE
Hi anyone already experienced with btrfs and its snapshots ? (performance / stability) ? Philippe From: xen-users-bounces@lists.xen.org [mailto:xen-users-bounces@lists.xen.org] On Behalf Of Alberto Asuero Sent: Friday, March 09, 2012 12:15 PM To: Andrea Monti Cc: xen-users@lists.xen.org Subject: Re: [Xen-users] Backup domU image at runtime Todd H. Foster wrote: Yes. Read my previous reply (hint: snapshot) I don''t have LVM on this partition, I''ve a big RAID 5 (I''ll try to migrate to LVM) Simon Hobson wrote: In a recovery situation, I can restore any individual machine to it''s last backup with the following steps : In Dom0, create the LVM volumes, create filesystems, mount them. Rsync the files back from the shared backup space. Unmount the volumes from Dom0 Startup the guest. It''d perfect but I can''t pause certains guests it''s a server of high availability... I think my best options is : 1) Migrate to LVM and I''ll try with snapshot 2) But I''ll also keep the backup of files of each domU. With the option 2 (more security), I''ll keep safe from a posible power supplies interruption (I''ve a UPS and a powervault redundance) Thank you guys! - - - Alberto Asuero Arroyo Software Engineer (+34) 645 816 025 On Fri, Mar 9, 2012 at 11:13 AM, Andrea Monti <ilsuonogiallo@gmail.com<mailto:ilsuonogiallo@gmail.com>> wrote: 2012/3/9 Simon Hobson <linux@thehobsons.co.uk<mailto:linux@thehobsons.co.uk>> Todd H. Foster wrote: Yes, It has worked fine for me. There may be some danger though. Centos handles it fine. Not "may be some", but "there is a definite" danger. If you make a backup of the virtual disk (by whatever means) from teh Som0 while a DomU is running then what you copy will be the equivalent of what you''d have on a real machine if you just yanked the power cord. Ie, you''d not include anything cached in the machine''s memory and not yet written to disk. That''s not a theoretical risk, it is a most definite and demonstrable problem. I think that taking a backup of a virtual disk while it is used is even worse: if you yank the power cord and you are using a journaled file system, the file system will probably recover by itself. If you backup a virtual disk while it is used you are taking an inconsistent image of the journal and an inconsistent (corrupted) image of the files! At one extreme, if you have a guest with little disk I/O (particularly writes), uses journalled filesystems, and you trigger the guest to flush it''s write cache* before making a snapshot - then the risk would be low. I think that the risk will always be high: you cannot be guaranteed that no process will write any file during the backup, and you have no way to check if the backup is working (the os will see the filesystem as ok, but application data could still be corrupted) Andrea _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org<mailto:Xen-users@lists.xen.org> http://lists.xen.org/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Fri, Mar 9, 2012 at 9:39 PM, <Philippe.Simonet@swisscom.com> wrote:> Hi > > > > anyone already experienced with btrfs and its snapshots ? > > (performance / stability) ?Is this a trick question? :D Btrfs is still under heavy development, so don''t be surprised if something suddenly goes wrong. The list occasionally still got "I suddenly can mount my fs" post (though to be fair, it''s usually followed by "after a power failure" or "after I yank my USB disk"). So there''s really not much point in asking about stability, is there? As for performance, even under optimum situation, btrfs still has to write the metadata twice (since the metadata is mirrored by default). Add to that the "still under heavy development" part, so the performance question is not really fair, is it? Having said that, no, I don''t use btrfs for xen. But I do use it in my notebook (mainly for its compression and snapshot feature), and would recommend it for development/backup environment. -- Fajar
On Fri, Mar 9, 2012 at 10:05 AM, Fajar A. Nugraha <list@fajar.net> wrote:>> anyone already experienced with btrfs and its snapshots ? >> >> (performance / stability) ? > > > Is this a trick question? :Dalso, btrfs is _not_ a cluster filesystem, so i would be very (pleasantly) surprised if you could mount a snapshot on a different machine from the one that 'owns' the main filesystem. so, it would only facilitate 'from inside' backups, no advantage for 'from outside' access. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
I agree totally. This is why we are migrating to xcp. The backup/snapshot is much better and easier to deal with as well as having little or no risk. Be aware that snapshots on xcp are a little messy, but you can suspend a domain and do a full backup, which is a clean way to go. That being said, just copying a running DomU is a high risk maneuver, but I''ve been doing this for over 4 or 5 years and have yet to have a restore fail to start. Yes you are going to lose something. If you are running a database, this is probably a bad idea. But if you are running an app server or web server, which I am, copying the DomU file has always worked for me. -Todd Foster From: xen-users-bounces@lists.xen.org [mailto:xen-users-bounces@lists.xen.org] On Behalf Of Alberto Asuero Sent: Friday, March 09, 2012 3:15 AM To: Andrea Monti Cc: xen-users@lists.xen.org Subject: Re: [Xen-users] Backup domU image at runtime Todd H. Foster wrote: Yes. Read my previous reply (hint: snapshot) I don''t have LVM on this partition, I''ve a big RAID 5 (I''ll try to migrate to LVM) Simon Hobson wrote: In a recovery situation, I can restore any individual machine to it''s last backup with the following steps : In Dom0, create the LVM volumes, create filesystems, mount them. Rsync the files back from the shared backup space. Unmount the volumes from Dom0 Startup the guest. It''d perfect but I can''t pause certains guests it''s a server of high availability... I think my best options is : 1) Migrate to LVM and I''ll try with snapshot 2) But I''ll also keep the backup of files of each domU. With the option 2 (more security), I''ll keep safe from a posible power supplies interruption (I''ve a UPS and a powervault redundance) Thank you guys! - - - Alberto Asuero Arroyo Software Engineer (+34) 645 816 025 On Fri, Mar 9, 2012 at 11:13 AM, Andrea Monti <ilsuonogiallo@gmail.com> wrote: 2012/3/9 Simon Hobson <linux@thehobsons.co.uk> Todd H. Foster wrote: Yes, It has worked fine for me. There may be some danger though. Centos handles it fine. Not "may be some", but "there is a definite" danger. If you make a backup of the virtual disk (by whatever means) from teh Som0 while a DomU is running then what you copy will be the equivalent of what you''d have on a real machine if you just yanked the power cord. Ie, you''d not include anything cached in the machine''s memory and not yet written to disk. That''s not a theoretical risk, it is a most definite and demonstrable problem. I think that taking a backup of a virtual disk while it is used is even worse: if you yank the power cord and you are using a journaled file system, the file system will probably recover by itself. If you backup a virtual disk while it is used you are taking an inconsistent image of the journal and an inconsistent (corrupted) image of the files! At one extreme, if you have a guest with little disk I/O (particularly writes), uses journalled filesystems, and you trigger the guest to flush it''s write cache* before making a snapshot - then the risk would be low. I think that the risk will always be high: you cannot be guaranteed that no process will write any file during the backup, and you have no way to check if the backup is working (the os will see the filesystem as ok, but application data could still be corrupted) Andrea _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Fri, Mar 9, 2012 at 10:30 PM, Javier Guerra Giraldez <javier@guerrag.com> wrote:> On Fri, Mar 9, 2012 at 10:05 AM, Fajar A. Nugraha <list@fajar.net> wrote: >>> anyone already experienced with btrfs and its snapshots ? >>> >>> (performance / stability) ? >> >> >> Is this a trick question? :D > > also, btrfs is _not_ a cluster filesystem, so i would be very > (pleasantly) surprised if you could mount a snapshot on a different > machine from the one that ''owns'' the main filesystem. > > so, it would only facilitate ''from inside'' backups, no advantage for > ''from outside'' access.Not necessarily. Phillppe MIGHT want the following scenarios: - all domU''s disk image are stored as files - the files are stored on a btrfs fs on dom0 - backups will be done by copying domU''s disk image In that case btrfs'' snapshot can replace LVM''s snapshot, with the additional benefit that you don''t have to worry about the snapshot LV being full. Also, if you implement some continious snapshot mechanism (like snapper), you could actually roll-back a domU''s disk to a certain point-in-time in past. Very useful if (for example) a user accidently delete an important file, realize it immediately, and want it back. -- Fajar