I''m tossing around the idea of creating a sort of "VDI" setup with Xen. One thing that I would like to do in this scenario is "checkpoint" the user''s VM at a certain interval so that if something bad happens they can go right back to the latest point time - maybe a half-hour earlier or something like that. I was trying to use "xm save -c <Domain> <File>" to do this, but the domain pauses while the contents of memory are saved off to disk. I was wondering if anyone knows of a way to "checkpoint" without pausing the domain? I can''t do something like an hourly checkpoint if the domain pauses every hour - the users on the domUs would kill me. Is "seamless checkpointing" something that''s being developed, something that can be done another way, or something I should just give up on now? Thanks - Nick This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, Sep 27, 2008 at 2:48 PM, Nick Couchman <Nick.Couchman@seakr.com> wrote:> I''m tossing around the idea of creating a sort of "VDI" setup with Xen. One > thing that I would like to do in this scenario is "checkpoint" the user''s VM > at a certain interval so that if something bad happens they can go right > back to the latest point time - maybe a half-hour earlier or something like > that. I was trying to use "xm save -c <Domain> <File>" to do this, but the > domain pauses while the contents of memory are saved off to disk. I was > wondering if anyone knows of a way to "checkpoint" without pausing the > domain? I can''t do something like an hourly checkpoint if the domain pauses > every hour - the users on the domUs would kill me. Is "seamless > checkpointing" something that''s being developed, something that can be done > another way, or something I should just give up on now? >It is on the TODO list (last updated Jan 08): http://wiki.xensource.com/xenwiki/XenTodoList I think the LVM snapshots should be a good way. Another idea is to use a ZFS domain. I''ll look into this more soon. A quick google search and remembering back to a previous Xen summit, there have been attempts. I don''t know the status of them though. Feel free to float around ideas and discussion about this type of thing on the xen-research list. We should also do a better search of the summits and mailing lists to see what has been done and also check roadmaps and ask xen- devel if anybody is working on it. Cheers, Todd -- Todd Deshane http://todddeshane.net check out our book: http://runningxen.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thanks, Todd, I''m less concerned about the disk and more about Xen being able to do the memory save "seamlessly" so that there isn''t any pause for the user while the save is completed (or at least not near as long as it is now). The disk would be nice, too, although I''m using OCFS2 and not LVM or ZFS for my domUs, so I''m not sure how much that would help me. Any chance it could be implemented with Qcow, as well? VMware does something like this and then quickly changes out the disk that the VM sees from the original to the qcow image. -Nick>>> On 2008/09/30 at 22:32, "Todd Deshane" <deshantm@gmail.com> wrote:On Sat, Sep 27, 2008 at 2:48 PM, Nick Couchman <Nick.Couchman@seakr.com> wrote:> I''m tossing around the idea of creating a sort of "VDI" setup with Xen. One > thing that I would like to do in this scenario is "checkpoint" the user''s VM > at a certain interval so that if something bad happens they can go right > back to the latest point time - maybe a half-hour earlier or something like > that. I was trying to use "xm save -c <Domain> <File>" to do this, but the > domain pauses while the contents of memory are saved off to disk. I was > wondering if anyone knows of a way to "checkpoint" without pausing the > domain? I can''t do something like an hourly checkpoint if the domain pauses > every hour - the users on the domUs would kill me. Is "seamless > checkpointing" something that''s being developed, something that can be done > another way, or something I should just give up on now? >It is on the TODO list (last updated Jan 08): http://wiki.xensource.com/xenwiki/XenTodoList I think the LVM snapshots should be a good way. Another idea is to use a ZFS domain. I''ll look into this more soon. A quick google search and remembering back to a previous Xen summit, there have been attempts. I don''t know the status of them though. Feel free to float around ideas and discussion about this type of thing on the xen-research list. We should also do a better search of the summits and mailing lists to see what has been done and also check roadmaps and ask xen- devel if anybody is working on it. Cheers, Todd -- Todd Deshane http://todddeshane.net check out our book: http://runningxen.com This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
It _should_ work with qcow, but does not. At least not for me. I use Ubuntu 8.04 LTS with Xen 3.2.1 and 2.6.24-41 and it does not work to attach a qcow image to a vm. And therefore the vm sees no disk while booting. If you find a solution for this, please let me know. Regards Fabian 2008/10/1 Nick Couchman <Nick.Couchman@seakr.com>:> Thanks, Todd, > I''m less concerned about the disk and more about Xen being able to do the memory save "seamlessly" so that there isn''t any pause for the user while the save is completed (or at least not near as long as it is now). The disk would be nice, too, although I''m using OCFS2 and not LVM or ZFS for my domUs, so I''m not sure how much that would help me. Any chance it could be implemented with Qcow, as well? VMware does something like this and then quickly changes out the disk that the VM sees from the original to the qcow image. > > -Nick > >>>> On 2008/09/30 at 22:32, "Todd Deshane" <deshantm@gmail.com> wrote: > On Sat, Sep 27, 2008 at 2:48 PM, Nick Couchman <Nick.Couchman@seakr.com> wrote: >> I''m tossing around the idea of creating a sort of "VDI" setup with Xen. One >> thing that I would like to do in this scenario is "checkpoint" the user''s VM >> at a certain interval so that if something bad happens they can go right >> back to the latest point time - maybe a half-hour earlier or something like >> that. I was trying to use "xm save -c <Domain> <File>" to do this, but the >> domain pauses while the contents of memory are saved off to disk. I was >> wondering if anyone knows of a way to "checkpoint" without pausing the >> domain? I can''t do something like an hourly checkpoint if the domain pauses >> every hour - the users on the domUs would kill me. Is "seamless >> checkpointing" something that''s being developed, something that can be done >> another way, or something I should just give up on now? >> > > It is on the TODO list (last updated Jan 08): > http://wiki.xensource.com/xenwiki/XenTodoList > > I think the LVM snapshots should be a good way. > > Another idea is to use a ZFS domain. > > I''ll look into this more soon. > > A quick google search and remembering back to a previous Xen summit, > there have been attempts. I don''t know the status of them though. > > Feel free to float around ideas and discussion about this type of thing > on the xen-research list. > > We should also do a better search of the summits and mailing lists > to see what has been done and also check roadmaps and ask xen- > devel if anybody is working on it. > > Cheers, > Todd > > -- > Todd Deshane > http://todddeshane.net > check out our book: http://runningxen.com > > > > This e-mail may contain confidential and privileged material for the sole use of the intended recipient. If this email is not intended for you, or you are not responsible for the delivery of this message to the intended recipient, please note that this message may contain SEAKR Engineering (SEAKR) Privileged/Proprietary Information. In such a case, you are strictly prohibited from downloading, photocopying, distributing or otherwise using this message, its contents or attachments in any way. If you have received this message in error, please notify us immediately by replying to this e-mail and delete the message from your mailbox. Information contained in this message that does not relate to the business of SEAKR is neither endorsed by nor attributable to SEAKR. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Nick,> I was wondering if anyone knows of a way to "checkpoint" without pausing the > domain? I can''t do something like an hourly checkpoint if the domain pauses > every hour - the users on the domUs would kill me. Is "seamless > checkpointing" something that''s being developed, something that can be done > another way, or something I should just give up on now?I''ve been working on something for my research that probably addresses what you''re looking for: low latency/downtime checkpointing. There are two ways I know this can be done. You can modify the live-migration mechanism in Xen to save a checkpoint instead of migrating to another host. I don''t know why this hasn''t been provided in the xm save tool, but it''s a pretty simple hack to get working since the xm migrate and xm save codebase is basically the same. If you''re interested, I can send patches to allow you to do that. Then basically on a checkpoint, pre-copying would be used to cut the actual time the domU has to be paused. The other way that I''ve implemented for my research project is to use copy-on-write. I''ve implemented a prototype and in the middle of writing paper for publication. It''s not suited for production use, but I''d be glad to contribute the code if there''s interest. Mike _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Mike, i''d like to see your code. :) Regards Fabian 2008/11/14 Mike Sun <msun@gatech.edu>> Hi Nick, > > > I was wondering if anyone knows of a way to "checkpoint" without pausing > the > > domain? I can''t do something like an hourly checkpoint if the domain > pauses > > every hour - the users on the domUs would kill me. Is "seamless > > checkpointing" something that''s being developed, something that can be > done > > another way, or something I should just give up on now? > > I''ve been working on something for my research that probably addresses > what you''re looking for: low latency/downtime checkpointing. There > are two ways I know this can be done. > > You can modify the live-migration mechanism in Xen to save a > checkpoint instead of migrating to another host. I don''t know why > this hasn''t been provided in the xm save tool, but it''s a pretty > simple hack to get working since the xm migrate and xm save codebase > is basically the same. If you''re interested, I can send patches to > allow you to do that. Then basically on a checkpoint, pre-copying > would be used to cut the actual time the domU has to be paused. > > The other way that I''ve implemented for my research project is to use > copy-on-write. I''ve implemented a prototype and in the middle of > writing paper for publication. It''s not suited for production use, > but I''d be glad to contribute the code if there''s interest. > > Mike > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Fabian, We''re still working to finish the paper first, but I''ll certainly cleanup and provide the prototype code after we''re done. Thanks, Mike On Fri, Nov 14, 2008 at 4:42 AM, Fabian Flägel <fabigant@googlemail.com> wrote:> Hi Mike, > > i''d like to see your code. :) > > Regards > Fabian > > 2008/11/14 Mike Sun <msun@gatech.edu> >> >> Hi Nick, >> >> > I was wondering if anyone knows of a way to "checkpoint" without pausing >> > the >> > domain? I can''t do something like an hourly checkpoint if the domain >> > pauses >> > every hour - the users on the domUs would kill me. Is "seamless >> > checkpointing" something that''s being developed, something that can be >> > done >> > another way, or something I should just give up on now? >> >> I''ve been working on something for my research that probably addresses >> what you''re looking for: low latency/downtime checkpointing. There >> are two ways I know this can be done. >> >> You can modify the live-migration mechanism in Xen to save a >> checkpoint instead of migrating to another host. I don''t know why >> this hasn''t been provided in the xm save tool, but it''s a pretty >> simple hack to get working since the xm migrate and xm save codebase >> is basically the same. If you''re interested, I can send patches to >> allow you to do that. Then basically on a checkpoint, pre-copying >> would be used to cut the actual time the domU has to be paused. >> >> The other way that I''ve implemented for my research project is to use >> copy-on-write. I''ve implemented a prototype and in the middle of >> writing paper for publication. It''s not suited for production use, >> but I''d be glad to contribute the code if there''s interest. >> >> Mike >> >> _______________________________________________ >> Xen-users mailing list >> Xen-users@lists.xensource.com >> http://lists.xensource.com/xen-users > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi Mike, ok, i''ll have to wait... :) Even if your paper is not ready yet, may i ask which COW do you use? I tried qcow but this doesn''t seem to be supported under Ubuntu 8.04 with Xen 3.2.1. Regards, Fabian 2008/11/14 Mike Sun <msun@gatech.edu>> Hi Fabian, > > We''re still working to finish the paper first, but I''ll certainly > cleanup and provide the prototype code after we''re done. > > Thanks, > Mike > > On Fri, Nov 14, 2008 at 4:42 AM, Fabian Flägel <fabigant@googlemail.com> > wrote: > > Hi Mike, > > > > i''d like to see your code. :) > > > > Regards > > Fabian > > > > 2008/11/14 Mike Sun <msun@gatech.edu> > >> > >> Hi Nick, > >> > >> > I was wondering if anyone knows of a way to "checkpoint" without > pausing > >> > the > >> > domain? I can''t do something like an hourly checkpoint if the domain > >> > pauses > >> > every hour - the users on the domUs would kill me. Is "seamless > >> > checkpointing" something that''s being developed, something that can be > >> > done > >> > another way, or something I should just give up on now? > >> > >> I''ve been working on something for my research that probably addresses > >> what you''re looking for: low latency/downtime checkpointing. There > >> are two ways I know this can be done. > >> > >> You can modify the live-migration mechanism in Xen to save a > >> checkpoint instead of migrating to another host. I don''t know why > >> this hasn''t been provided in the xm save tool, but it''s a pretty > >> simple hack to get working since the xm migrate and xm save codebase > >> is basically the same. If you''re interested, I can send patches to > >> allow you to do that. Then basically on a checkpoint, pre-copying > >> would be used to cut the actual time the domU has to be paused. > >> > >> The other way that I''ve implemented for my research project is to use > >> copy-on-write. I''ve implemented a prototype and in the middle of > >> writing paper for publication. It''s not suited for production use, > >> but I''d be glad to contribute the code if there''s interest. > >> > >> Mike > >> > >> _______________________________________________ > >> Xen-users mailing list > >> Xen-users@lists.xensource.com > >> http://lists.xensource.com/xen-users > > > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> Even if your paper is not ready yet, may i ask which COW do you use? I tried > qcow but this doesn''t seem to be supported under Ubuntu 8.04 with Xen 3.2.1.I think there''s some confusion. Are you interested in CoW for the disk or for the memory checkpoint? QCOW is a disk format for disk checkpointing. The earlier question by Nick seemed more interested in doing a seamless memory checkpoint of a running VM. _______________________________________________ Xen-research mailing list Xen-research@lists.xensource.com http://lists.xensource.com/mailman/listinfo/xen-research
Hi, i missed your answer, sorry for the delay. 2008/11/14 Mike Sun <msun@gatech.edu>> > Even if your paper is not ready yet, may i ask which COW do you use? I > tried > > qcow but this doesn''t seem to be supported under Ubuntu 8.04 with Xen > 3.2.1. > > I think there''s some confusion. Are you interested in CoW for the > disk or for the memory checkpoint? QCOW is a disk format for disk > checkpointing. The earlier question by Nick seemed more interested in > doing a seamless memory checkpoint of a running VM. >I meant qcow. It just don''t work for me. Maybe you could give me a hand on that. Regards, Fabian _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users