Anyone ? -nb --- On Sun, 1/3/10, J Nb <j_nwb@yahoo.com> wrote:> From: J Nb <j_nwb@yahoo.com> > Subject: coordinating VM snapshot with LVM snapshot. > To: xen-devel@lists.xensource.com > Date: Sunday, January 3, 2010, 6:17 PM > Hi > How to co-ordinate VM snapshots and disk > snapshots ? Here is a sequence I am thinking of > automating... > > 1.Tell VM to sync (say via ssh or some special > agent) > * This will do "sync" and any > application specific hook to make it quiescent. > > 2. Tell VM/Hypervisor to flush all disk buferes > ? (HOW ??? ) > * Anything at this level? > > 3. Pause the VM > > 4. xm save -c (To save the VMs state in to > checkpoint file.) > (This may take more time !) > > 5. Take LVM snapshot for each disk. > > 6. Unpause the VM > > 7. Application level resume if any. > > 7. Use the snapshot to copy each disks ?!!!! (either > dd or mount followed by cp/rsync) (This can be big! > Any optimization suggestions ) > > 8. discard snapshot. > > > Would the above sequence work ? > > Any suggestions on individual steps ? > > Anything specific for PV and HVM virtual machines? > > Is there a backdoor to qemu-dm console ? > > Does anyone know how similar functionality is implemented > in XenServer product or Virtual Iron or anyone else? > > -nb > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-Jan-13 07:11 UTC
Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.
On Tue, Jan 12, 2010 at 05:44:18PM -0800, J Nb wrote:> Anyone ? >Now that XenServer is opensourced as XCP (Xen Cloud Platform), you could take a look at the source.. I believe XenServer supports Windows VSS coordination for snapshots. "xe vm-snapshot-with-quiesce" -- Pasi> -nb > > --- On Sun, 1/3/10, J Nb <j_nwb@yahoo.com> wrote: > > > From: J Nb <j_nwb@yahoo.com> > > Subject: coordinating VM snapshot with LVM snapshot. > > To: xen-devel@lists.xensource.com > > Date: Sunday, January 3, 2010, 6:17 PM > > Hi > > How to co-ordinate VM snapshots and disk > > snapshots ? Here is a sequence I am thinking of > > automating... > > > > 1.Tell VM to sync (say via ssh or some special > > agent) > > * This will do "sync" and any > > application specific hook to make it quiescent. > > > > 2. Tell VM/Hypervisor to flush all disk buferes > > ? (HOW ??? ) > > * Anything at this level? > > > > 3. Pause the VM > > > > 4. xm save -c (To save the VMs state in to > > checkpoint file.) > > (This may take more time !) > > > > 5. Take LVM snapshot for each disk. > > > > 6. Unpause the VM > > > > 7. Application level resume if any. > > > > 7. Use the snapshot to copy each disks ?!!!! (either > > dd or mount followed by cp/rsync) (This can be big! > > Any optimization suggestions ) > > > > 8. discard snapshot. > > > > > > Would the above sequence work ? > > > > Any suggestions on individual steps ? > > > > Anything specific for PV and HVM virtual machines? > > > > Is there a backdoor to qemu-dm console ? > > > > Does anyone know how similar functionality is implemented > > in XenServer product or Virtual Iron or anyone else? > > > > -nb > > > > > > > > > > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi guys, do you use/recommend any management tool to provision/admin your windows/linux DomUs? Thanks Cris _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-Jan-14 17:15 UTC
Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.
On Wed, Jan 13, 2010 at 09:11:47AM +0200, Pasi Kärkkäinen wrote:> On Tue, Jan 12, 2010 at 05:44:18PM -0800, J Nb wrote: > > Anyone ? > > > > Now that XenServer is opensourced as XCP (Xen Cloud Platform), you could > take a look at the source.. > > I believe XenServer supports Windows VSS coordination for snapshots. > "xe vm-snapshot-with-quiesce" >Did you get anywhere with this? For a long time I''ve been thinking of writing some kind of "VSS" library/daemon for Linux.. similar to Microsoft Windows Volume Snapshot Services. It could (and should) be called something different than VSS, but I''ll just use that name for now.. The interface/usage could be something like: 1. backup begins 2. execute in the guest: "vss_control pre-snapshot xvda" 3. take snapshot of the lvm volume having guest disk xvda 4. execute in the guest: "vss_control post-snapshot xvda" 5. do the actual backup from the lvm snapshot 6. backup ends This would have minimal downtime for the applications running in the guest. pre-snapshot action would go through all the application specific pre-scripts, execute them, preparing the applications for taking the snapshot, and then it would flush the filesystem, kernel and disk caches as needed to prepare for a consistent snapshot. Preparing the application for taking the snapshot might mean something like placing locks to database tables preventing disk writes from happening, or shutting down the application, or whatever the specific application requires. post-snapshot action would again go through and execute all the application specific post-scripts releasing the applications back to normal running state. The application specific pre/post scripts could be stored like this: /etc/vss/scripts/xvda/001-mysql.pre /etc/vss/scripts/xvda/001-mysql.post /etc/vss/scripts/xvda/002-pgsql.pre /etc/vss/scripts/xvda/002-pgsql.post /etc/vss/scripts/xvda/003-my_custom_app.pre /etc/vss/scripts/xvda/003-my_custom_app.post Or you could have "scripts/xvda2/*" if you have lvm volumes directly as partitions to the guest. You get the idea. Then the next step would be to create a daemon ("vssd") that listens for requests from the network, and/or possibly from Xen dom0 through the hypervisor. Or then the backup server might issue those pre-snapshot and post-snapshot commands through ssh.. many options how to implement it. Does this make any sense? Comments? -- Pasi> > > > > --- On Sun, 1/3/10, J Nb <j_nwb@yahoo.com> wrote: > > > > > From: J Nb <j_nwb@yahoo.com> > > > Subject: coordinating VM snapshot with LVM snapshot. > > > To: xen-devel@lists.xensource.com > > > Date: Sunday, January 3, 2010, 6:17 PM > > > Hi > > > How to co-ordinate VM snapshots and disk > > > snapshots ? Here is a sequence I am thinking of > > > automating... > > > > > > 1.Tell VM to sync (say via ssh or some special > > > agent) > > > * This will do "sync" and any > > > application specific hook to make it quiescent. > > > > > > 2. Tell VM/Hypervisor to flush all disk buferes > > > ? (HOW ??? ) > > > * Anything at this level? > > > > > > 3. Pause the VM > > > > > > 4. xm save -c (To save the VMs state in to > > > checkpoint file.) > > > (This may take more time !) > > > > > > 5. Take LVM snapshot for each disk. > > > > > > 6. Unpause the VM > > > > > > 7. Application level resume if any. > > > > > > 7. Use the snapshot to copy each disks ?!!!! (either > > > dd or mount followed by cp/rsync) (This can be big! > > > Any optimization suggestions ) > > > > > > 8. discard snapshot. > > > > > > > > > Would the above sequence work ? > > > > > > Any suggestions on individual steps ? > > > > > > Anything specific for PV and HVM virtual machines? > > > > > > Is there a backdoor to qemu-dm console ? > > > > > > Does anyone know how similar functionality is implemented > > > in XenServer product or Virtual Iron or anyone else? > > > > > > -nb > > > > > > > > > > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
J Nb
2010-Jan-14 18:34 UTC
Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.
I will look in to XCP.. but my guess is that they use their pv drivers to communicate via xenstore..etc. I am looking for something that would work with vanilla open source xen. (well non XCP :) ) But my plan is same as I described in skeleton and organization very similar to yours. but to have something that works with ssh/wmi. This way scripts etc can be centrally managed. What I was looking for was -- For xen is there a flush command for PV and HVM vms. (hypervisor level) -- Is sync sufficient as a starting point at OS level. -- Any repository of application specific scripts. (Mysql,oracle..etc) -- And anything to control cpu/io consumption of the backup process. (Probably start with cstream / nice) -nb --- On Thu, 1/14/10, Pasi Kärkkäinen <pasik@iki.fi> wrote:> From: Pasi Kärkkäinen <pasik@iki.fi> > Subject: Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot. > To: "J Nb" <j_nwb@yahoo.com> > Cc: "xen-users" <xen-users@lists.xensource.com> > Date: Thursday, January 14, 2010, 9:15 AM > On Wed, Jan 13, 2010 at 09:11:47AM > +0200, Pasi Kärkkäinen wrote: > > On Tue, Jan 12, 2010 at 05:44:18PM -0800, J Nb wrote: > > > Anyone ? > > > > > > > Now that XenServer is opensourced as XCP (Xen Cloud > Platform), you could > > take a look at the source.. > > > > I believe XenServer supports Windows VSS coordination > for snapshots. > > "xe vm-snapshot-with-quiesce" > > > > Did you get anywhere with this? > > For a long time I''ve been thinking of writing some kind of > > "VSS" library/daemon for Linux.. similar to Microsoft > Windows > Volume Snapshot Services. > > It could (and should) be called something different than > VSS, > but I''ll just use that name for now.. > > The interface/usage could be something like: > > 1. backup begins > 2. execute in the guest: "vss_control > pre-snapshot xvda" > 3. take snapshot of the lvm volume > having guest disk xvda > 4. execute in the guest: "vss_control > post-snapshot xvda" > 5. do the actual backup from the lvm > snapshot > 6. backup ends > > This would have minimal downtime for the applications > running in the guest. > > pre-snapshot action would go through all the application > specific pre-scripts, > execute them, preparing the applications for taking the > snapshot, and then > it would flush the filesystem, kernel and disk caches as > needed to prepare for > a consistent snapshot. > > Preparing the application for taking the snapshot might > mean something > like placing locks to database tables preventing disk > writes from happening, > or shutting down the application, or whatever the specific > application requires. > > post-snapshot action would again go through and execute all > the application specific > post-scripts releasing the applications back to normal > running state. > > The application specific pre/post scripts could be stored > like this: > > > /etc/vss/scripts/xvda/001-mysql.pre > /etc/vss/scripts/xvda/001-mysql.post > > /etc/vss/scripts/xvda/002-pgsql.pre > /etc/vss/scripts/xvda/002-pgsql.post > > > /etc/vss/scripts/xvda/003-my_custom_app.pre > > /etc/vss/scripts/xvda/003-my_custom_app.post > > Or you could have "scripts/xvda2/*" if you have lvm volumes > directly as > partitions to the guest. You get the idea. > > Then the next step would be to create a daemon ("vssd") > that listens for > requests from the network, and/or possibly from Xen dom0 > through the hypervisor. > > Or then the backup server might issue those pre-snapshot > and post-snapshot > commands through ssh.. many options how to implement it. > > Does this make any sense? Comments? > > -- Pasi > > > > > > > > > --- On Sun, 1/3/10, J Nb <j_nwb@yahoo.com> > wrote: > > > > > > > From: J Nb <j_nwb@yahoo.com> > > > > Subject: coordinating VM snapshot with LVM > snapshot. > > > > To: xen-devel@lists.xensource.com > > > > Date: Sunday, January 3, 2010, 6:17 PM > > > > Hi > > > > How to co-ordinate VM snapshots and > disk > > > > snapshots ? Here is a sequence I am thinking > of > > > > automating... > > > > > > > > 1.Tell VM to sync (say via ssh or some > special > > > > agent) > > > > * This will do "sync" and any > > > > application specific hook to make it > quiescent. > > > > > > > > 2. Tell VM/Hypervisor to flush all disk > buferes > > > > ? (HOW ??? ) > > > > * Anything at this level? > > > > > > > > 3. Pause the VM > > > > > > > > 4. xm save -c (To save the VMs state in > to > > > > checkpoint file.) > > > > (This may take more time !) > > > > > > > > 5. Take LVM snapshot for each disk. > > > > > > > > 6. Unpause the VM > > > > > > > > 7. Application level resume if any. > > > > > > > > 7. Use the snapshot to copy each disks > ?!!!! (either > > > > dd or mount followed by cp/rsync) (This > can be big! > > > > Any optimization suggestions ) > > > > > > > > 8. discard snapshot. > > > > > > > > > > > > Would the above sequence work ? > > > > > > > > Any suggestions on individual steps ? > > > > > > > > Anything specific for PV and HVM virtual > machines? > > > > > > > > Is there a backdoor to qemu-dm console ? > > > > > > > > Does anyone know how similar functionality > is implemented > > > > in XenServer product or Virtual Iron or > anyone else? > > > > > > > > -nb > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-Jan-14 20:06 UTC
Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.
On Thu, Jan 14, 2010 at 10:34:30AM -0800, J Nb wrote:> I will look in to XCP.. but my guess is that they use their pv drivers to communicate via xenstore..etc. I am looking for something that would work with vanilla open source xen. (well non XCP :) ) >Yeah.. most probably.> But my plan is same as I described in skeleton and organization very similar to yours. but to have something that works with ssh/wmi. This way scripts etc can be centrally managed. >Indeed.> What I was looking for was > -- For xen is there a flush command for PV and HVM vms. (hypervisor level) >Not that I know of. If you use phy: backend for LVM volumes, then dom0 doesn''t do any caching.. then it''s enough to flush the caches and disks in the guest.> -- Is sync sufficient as a starting point at OS level. >I''m not sure.. Linux kernel v2.6.29 added filesystem "freeze" feature (http://lwn.net/Articles/287435/) that can possibly help here. I remember reading that "sync" might not really flush everything.. but yeah, I guess it''s good enough as a starting point.> -- Any repository of application specific scripts. (Mysql,oracle..etc) >Do you mean a website with scripts to share with others? Ideally application specific scripts would be packaged together with the apps, assuming this kind of "vssd" was included as a default in the distros :)> -- And anything to control cpu/io consumption of the backup process. (Probably start with cstream / nice) >This pretty much depends where you do the snapshots and from where you run the backups.. and also on the used storage solution. -- Pasi> -nb > > --- On Thu, 1/14/10, Pasi Kärkkäinen <pasik@iki.fi> wrote: > > > From: Pasi Kärkkäinen <pasik@iki.fi> > > Subject: Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot. > > To: "J Nb" <j_nwb@yahoo.com> > > Cc: "xen-users" <xen-users@lists.xensource.com> > > Date: Thursday, January 14, 2010, 9:15 AM > > On Wed, Jan 13, 2010 at 09:11:47AM > > +0200, Pasi Kärkkäinen wrote: > > > On Tue, Jan 12, 2010 at 05:44:18PM -0800, J Nb wrote: > > > > Anyone ? > > > > > > > > > > Now that XenServer is opensourced as XCP (Xen Cloud > > Platform), you could > > > take a look at the source.. > > > > > > I believe XenServer supports Windows VSS coordination > > for snapshots. > > > "xe vm-snapshot-with-quiesce" > > > > > > > Did you get anywhere with this? > > > > For a long time I''ve been thinking of writing some kind of > > > > "VSS" library/daemon for Linux.. similar to Microsoft > > Windows > > Volume Snapshot Services. > > > > It could (and should) be called something different than > > VSS, > > but I''ll just use that name for now.. > > > > The interface/usage could be something like: > > > > 1. backup begins > > 2. execute in the guest: "vss_control > > pre-snapshot xvda" > > 3. take snapshot of the lvm volume > > having guest disk xvda > > 4. execute in the guest: "vss_control > > post-snapshot xvda" > > 5. do the actual backup from the lvm > > snapshot > > 6. backup ends > > > > This would have minimal downtime for the applications > > running in the guest. > > > > pre-snapshot action would go through all the application > > specific pre-scripts, > > execute them, preparing the applications for taking the > > snapshot, and then > > it would flush the filesystem, kernel and disk caches as > > needed to prepare for > > a consistent snapshot. > > > > Preparing the application for taking the snapshot might > > mean something > > like placing locks to database tables preventing disk > > writes from happening, > > or shutting down the application, or whatever the specific > > application requires. > > > > post-snapshot action would again go through and execute all > > the application specific > > post-scripts releasing the applications back to normal > > running state. > > > > The application specific pre/post scripts could be stored > > like this: > > > > > > /etc/vss/scripts/xvda/001-mysql.pre > > /etc/vss/scripts/xvda/001-mysql.post > > > > /etc/vss/scripts/xvda/002-pgsql.pre > > /etc/vss/scripts/xvda/002-pgsql.post > > > > > > /etc/vss/scripts/xvda/003-my_custom_app.pre > > > > /etc/vss/scripts/xvda/003-my_custom_app.post > > > > Or you could have "scripts/xvda2/*" if you have lvm volumes > > directly as > > partitions to the guest. You get the idea. > > > > Then the next step would be to create a daemon ("vssd") > > that listens for > > requests from the network, and/or possibly from Xen dom0 > > through the hypervisor. > > > > Or then the backup server might issue those pre-snapshot > > and post-snapshot > > commands through ssh.. many options how to implement it. > > > > Does this make any sense? Comments? > > > > -- Pasi > > > > > > > > > > > > > --- On Sun, 1/3/10, J Nb <j_nwb@yahoo.com> > > wrote: > > > > > > > > > From: J Nb <j_nwb@yahoo.com> > > > > > Subject: coordinating VM snapshot with LVM > > snapshot. > > > > > To: xen-devel@lists.xensource.com > > > > > Date: Sunday, January 3, 2010, 6:17 PM > > > > > Hi > > > > > How to co-ordinate VM snapshots and > > disk > > > > > snapshots ? Here is a sequence I am thinking > > of > > > > > automating... > > > > > > > > > > 1.Tell VM to sync (say via ssh or some > > special > > > > > agent) > > > > > * This will do "sync" and any > > > > > application specific hook to make it > > quiescent. > > > > > > > > > > 2. Tell VM/Hypervisor to flush all disk > > buferes > > > > > ? (HOW ??? ) > > > > > * Anything at this level? > > > > > > > > > > 3. Pause the VM > > > > > > > > > > 4. xm save -c (To save the VMs state in > > to > > > > > checkpoint file.) > > > > > (This may take more time !) > > > > > > > > > > 5. Take LVM snapshot for each disk. > > > > > > > > > > 6. Unpause the VM > > > > > > > > > > 7. Application level resume if any. > > > > > > > > > > 7. Use the snapshot to copy each disks > > ?!!!! (either > > > > > dd or mount followed by cp/rsync) (This > > can be big! > > > > > Any optimization suggestions ) > > > > > > > > > > 8. discard snapshot. > > > > > > > > > > > > > > > Would the above sequence work ? > > > > > > > > > > Any suggestions on individual steps ? > > > > > > > > > > Anything specific for PV and HVM virtual > > machines? > > > > > > > > > > Is there a backdoor to qemu-dm console ? > > > > > > > > > > Does anyone know how similar functionality > > is implemented > > > > > in XenServer product or Virtual Iron or > > anyone else? > > > > > > > > > > -nb > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Dirk H. Schulz
2010-Jan-21 10:49 UTC
Re: [Xen-users] Fw: coordinating VM snapshot with LVM snapshot.
Pasi Kärkkäinen schrieb:> > Do you mean a website with scripts to share with others? Ideally > application specific scripts would be packaged together with the apps, > assuming this kind of "vssd" was included as a default in the distros :) >This sounds altogether interesting and promising. What about making this a community effort and founding a project? I am working on something like this as well, and we could throw together ideas and share code parts. Dirk _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users