Hi, I recently set-up a Xen Server that''s hosting 3 Development Systems. It works quite well, so it''s time to put this into Production -- and to perform regular Backups. We use Bacula to Backup the dom0. My Idea was, to mount all domU Filesystems Read-Only to some Directories and then back these up from the Bacula-FD on the dom0. I found some warnings that Filesystems may not be mounted RW by more than one domain / system (makes sense when they don''t "know" about each other). However, are there any issues mounting a Filesystem Read-Only, then have it being backed-up by Bacula and then unmount it? Thanks for any hints in advance! Cheers, _ralf_ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, Dec 18, 2005 at 09:26:12PM +0100, Ralf Folkerts wrote:> I found some warnings that Filesystems may not be mounted RW by more > than one domain / system (makes sense when they don''t "know" about each > other). However, are there any issues mounting a Filesystem Read-Only, > then have it being backed-up by Bacula and then unmount it?The short answer is yes, it''s almost as bad as mounting it RW twice. Your dom0 kernel will make some assumptions about the read-only nature of the filesystem which will be invalidated when the domU changes it underneath. You need a cluster filesystem susch as GFS or OCFS2 to do this. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andy Smith wrote:> On Sun, Dec 18, 2005 at 09:26:12PM +0100, Ralf Folkerts wrote: > >>I found some warnings that Filesystems may not be mounted RW by more >>than one domain / system (makes sense when they don''t "know" about each >>other). However, are there any issues mounting a Filesystem Read-Only, >>then have it being backed-up by Bacula and then unmount it? > > > The short answer is yes, it''s almost as bad as mounting it RW twice. > Your dom0 kernel will make some assumptions about the read-only > nature of the filesystem which will be invalidated when the domU > changes it underneath. > > You need a cluster filesystem susch as GFS or OCFS2 to do this.Or if you use LVM you can create snapshots from which you can backup your data. Nathan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sun, Dec 18, 2005 at 09:59:59PM +0000, cornet@sheepy.org wrote:> Andy Smith wrote: > >On Sun, Dec 18, 2005 at 09:26:12PM +0100, Ralf Folkerts wrote: > > > >>I found some warnings that Filesystems may not be mounted RW by more > >>than one domain / system (makes sense when they don''t "know" about each > >>other). However, are there any issues mounting a Filesystem Read-Only, > >>then have it being backed-up by Bacula and then unmount it?[...]> >You need a cluster filesystem susch as GFS or OCFS2 to do this. > > Or if you use LVM you can create snapshots from which you can backup > your data.Unfortunately LVM snapshots are still considered too buggy (by Red Hat themselves) for production use. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andy Smith wrote:> On Sun, Dec 18, 2005 at 09:59:59PM +0000, cornet@sheepy.org wrote: >>Or if you use LVM you can create snapshots from which you can backup >>your data. > > > Unfortunately LVM snapshots are still considered too buggy (by Red > Hat themselves) for production use.Not tested them enough yet tbh. Use them on home server here and they appear fine thou :) Nathan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ralf Folkerts wrote:>Hi, > >I recently set-up a Xen Server that''s hosting 3 Development Systems. It >works quite well, so it''s time to put this into Production -- and to >perform regular Backups. > >We use Bacula to Backup the dom0. My Idea was, to mount all domU >Filesystems Read-Only to some Directories and then back these up from >the Bacula-FD on the dom0. > >I found some warnings that Filesystems may not be mounted RW by more >than one domain / system (makes sense when they don''t "know" about each >other). However, are there any issues mounting a Filesystem Read-Only, >then have it being backed-up by Bacula and then unmount it? > >When you mount a filesystem (even read-only) that''s already mounted read-write, the filesystem will contain meta-data that says "I''m dirty and currently in use." Best case scenario it just refuses to mount. Worst case scenario fsck gets automagically invoked somehow in an attempt to repair the partition and you risk serious corruption. Besides, you really don''t want to back up a raw partition. There''s no guarentee any application has the data in a consistent state. You really want to run a backup application with the domU. You could attach/detach a device to be the target of the backup from within dom0 at the appropriate scheduled time of course. Regards, Anthony Liguori>Thanks for any hints in advance! >Cheers, >_ralf_ > > > >------------------------------------------------------------------------ > >_______________________________________________ >Xen-users mailing list >Xen-users@lists.xensource.com >http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, On Sun, Dec 18, 2005 at 09:26:12PM +0100, Ralf Folkerts wrote:> Hi, > > I recently set-up a Xen Server that''s hosting 3 Development Systems. It > works quite well, so it''s time to put this into Production -- and to > perform regular Backups. > > We use Bacula to Backup the dom0. My Idea was, to mount all domU > Filesystems Read-Only to some Directories and then back these up from > the Bacula-FD on the dom0. > > I found some warnings that Filesystems may not be mounted RW by more > than one domain / system (makes sense when they don''t "know" about each > other). However, are there any issues mounting a Filesystem Read-Only, > then have it being backed-up by Bacula and then unmount it?Very bad idea. If you mount the FS read-only, the kernel will assume that certain things won''t change and it is going to get some nasty surprises when they _do_ change. In short: Don''t do this. A better approach is to use LVM and snapshots. Use LVM logical volumes as "disks" for your DomU (if you aren''t already doing this), create snapshots for your backups, mount the snapshots and backup _those_. After the backup, umount the snapshots and destroy them. Don''t forget to allocate enough storage to your snapshots to cover any changes in the volume. I''m currently using this to backup our production DomU systems from the Dom0 and it works very well. Regards, Alex. -- "Opportunity is missed by most people because it is dressed in overalls and looks like work." -- Thomas A. Edison _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Dec 19, 2005 at 08:04:50AM +0100, als@thangorodrim.de wrote: [...]> A better approach is to use LVM and snapshots. Use LVM logical volumes > as "disks" for your DomU (if you aren''t already doing this), create > snapshots for your backups, mount the snapshots and backup _those_.[...]> > I''m currently using this to backup our production DomU systems from the > Dom0 and it works very well.How do you assure that domU''s filesystem is in a consistent state when the snapshot is being taken in dom0? Also, which distro+kernel do you use? Marcin -- Marcin Owsiany porridge@expro.pl _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Dec 19, 2005 at 08:04:50AM +0100, als@thangorodrim.de wrote:> A better approach is to use LVM and snapshots. Use LVM logical volumes > as "disks" for your DomU (if you aren''t already doing this), create > snapshots for your backups, mount the snapshots and backup _those_. > After the backup, umount the snapshots and destroy them. > Don''t forget to allocate enough storage to your snapshots to cover any > changes in the volume. > > I''m currently using this to backup our production DomU systems from the > Dom0 and it works very well.So was I, but as the number of snapshots increased my LVM devices began to experience deadlocks leading to the whole machine having to be reported. When reported to Red Hat the word is that LVM snapshots are not yet ready for heavy production use. This is after discounting lack of memory in dom0 or too-small snapshot LVs. If you look in the archives of the LVM list you will see multiple people reporting the same issues, it is not peculiar to Xen. I am eagerly awaiting the time when I can start doing this again, but until then I prefer my block devices to work. Andy _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Dec 19, 2005 at 08:04:50AM +0100, als@thangorodrim.de wrote: [...]> A better approach is to use LVM and snapshots. Use LVM logical volumes > as "disks" for your DomU (if you aren''t already doing this), create > snapshots for your backups, mount the snapshots and backup _those_.[...]> > I''m currently using this to backup our production DomU systems from > the Dom0 and it works very well.How do you assure that domU''s filesystem is in a consistent state when the snapshot is being taken in dom0? Also, which distro+kernel do you use? Marcin You won''t know, but I bet it works most of the time. While taking a snapshot the LVM system will lock the VFS to assure consistency. In the above scenario the dom0 has no way of locking the domU''s VFS, therefore consistency is NOT assured. The LVM user above should be creating the snapshot from the same domU that has that logical volume mounted... At this point you''re probably better off with another solution. Dan. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Dec 19, 2005 at 01:18:00PM -0600, Daniel Goertzen wrote:> On Mon, Dec 19, 2005 at 08:04:50AM +0100, als@thangorodrim.de wrote: > [...] > > A better approach is to use LVM and snapshots. Use LVM logical volumes > > as "disks" for your DomU (if you aren''t already doing this), create > > snapshots for your backups, mount the snapshots and backup _those_. > [...] > > > > I''m currently using this to backup our production DomU systems from > > the Dom0 and it works very well. > > How do you assure that domU''s filesystem is in a consistent state when the > snapshot is being taken in dom0?Hmm, reading fs/ext3/super.c turned out to be enlightening.> Also, which distro+kernel do you use?On both Dom0 and DomU I use SLES9 (SUSE Linux Enterprise Server 9) and kernel 2.6.11 with Xen 2.0.7.> You won''t know, but I bet it works most of the time. While taking a > snapshot the LVM system will lock the VFS to assure consistency.If the VFS is aware of the FS being mounted, which in the Dom0 it isn''t.> In the > above scenario the dom0 has no way of locking the domU''s VFS, therefore > consistency is NOT assured. The LVM user above should be creating the > snapshot from the same domU that has that logical volume mounted...This is hardly possible as the DomU isn''t even aware that there is LVM involved. The Dom0 sees the blockdevice as a LVM logical volume, but the DomU sees it as a SCSI disk partition - as sda1. For some of my current virtual systems this is not much of a problem since the system itself is pretty stable and only some logfiles and transient files change - those would not matter upon restore anyway. But there will be other virtuals system where I''m not going to be this lucky. Time to investigate ...> At this > point you''re probably better off with another solution.Seems like I have to take a look at XFS and xfs_freeze. Regards, Alex. -- "Opportunity is missed by most people because it is dressed in overalls and looks like work." -- Thomas A. Edison _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 12/19/05, Ralf Folkerts <ralf.folkerts@gmx.de> wrote:> Hi, > > I recently set-up a Xen Server that''s hosting 3 Development Systems. It > works quite well, so it''s time to put this into Production -- and to > perform regular Backups.What about: 1. xm save <domU> <domU>-<timestamp>.xen 2. compress/copy etc. <domU>-<timestamp>.xen 3. xm restore <domU>-<timestamp>.xen Pros: saving also the state of the domU (runing processes etc.) Cons: stopping the domain during the backup _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 12/20/05, Stoyan Zhekov <zhekov@gmail.com> wrote:> On 12/19/05, Ralf Folkerts <ralf.folkerts@gmx.de> wrote:> What about: > 1. xm save <domU> <domU>-<timestamp>.xen > 2. compress/copy etc. <domU>-<timestamp>.xen > 3. xm restore <domU>-<timestamp>.xenHm ,there was a reply to me from Tom Brown. Seems I missunderstood the documentation. save/resrore deals only with the domain memory, not with the filesystem. btw we are using LVM2 snapshots (debian sarge). Haven''t problems with the snapshoting, but during the creating of the tar files, tar complain about sockets: tar: dev/log: socket ignored tar: var/spool/postfix/public/cleanup: socket ignored ... _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Dec 19, 2005 at 11:27:57PM +0100, als@thangorodrim.de wrote:> On both Dom0 and DomU I use SLES9 (SUSE Linux Enterprise Server 9) and > kernel 2.6.11 with Xen 2.0.7.I was asking, because LVM snapshots are known to have problems on certain systems, and I would like to use something close to what works for someone :) Does the kernel you use have any SuSE-specific patches applied, or is it the vanilla one? And the last thing: what disk controller you use? regards, Marcin -- Marcin Owsiany porridge@expro.pl _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users