Hi, I have a machine running several DomU''s, PVM''s als well as HVM''s. Each DomU resides in its own LV. Also, I managed drbd to replicate the LV''s to a second machine. Now, I want to try out Xen''s ability to live migrate a running DomU, that means, from machine1''s LV to machine2''s LV. If I understood this correct, during the process of migration, both LV''s need to be writable. In a default installation, drbd only supports one primary node, but I need primary/primary, correct? Here it comes to a cluster filesystem like OCFS - and I am stuck. How can OCFS help me? Or am I totally wrong and what I want is impossible? The main goal, of course, is to have a redundant fail over solution. If my question is more related to drbd-users, I''ll switch over to the other list. Thanks, Rainer _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> Hi, > > I have a machine running several DomU''s, PVM''s als well as HVM''s. Each > DomU resides in its own LV. > Also, I managed drbd to replicate the LV''s to a second machine. Now, I > want to try out Xen''s ability to live migrate a running DomU, that > means, from machine1''s LV to machine2''s LV. If I understood this > correct, during the process of migration, both LV''s need to be writable. > In a default installation, drbd only supports one primary node, but I > need primary/primary, correct? > Here it comes to a cluster filesystem like OCFS - and I am stuck. How > can OCFS help me? Or am I totally wrong and what I want is impossible? > The main goal, of course, is to have a redundant fail over solution. > If my question is more related to drbd-users, I''ll switch over to the > other list. > > Thanks, > RainerHi Rainer, Basically the only prerequisite is that you run DRBD8, which gives you primary/primary, to get live migration working. You don''t have to bother with cluster aware FS. Both nodes will not have exclusive access at the same time so running ext3 is perfectly fine. You can have a look at this guide to get some additional information: http://www.asplund.nu/xencluster/xen-cluster-howto.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
No no, that''s not right!! You definetly need an network-filesystem for drbd primary/primary! There are TWO systems, that access files for writing and both are not running the same kernel ergo doesn''t "know", that the other side is actually locking/writing a file! So you need OCFS & co. For doing the file-locking.... Regards, Florian -----Ursprüngliche Nachricht----- Von: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von Daniel Asplund Gesendet: Montag, 10. November 2008 09:46 An: Rainer Sokoll Cc: xen-users@lists.xensource.com Betreff: Re: [Xen-users] Live migration and drbd> Hi, > > I have a machine running several DomU''s, PVM''s als well as HVM''s. Each > DomU resides in its own LV. > Also, I managed drbd to replicate the LV''s to a second machine. Now, I > want to try out Xen''s ability to live migrate a running DomU, that > means, from machine1''s LV to machine2''s LV. If I understood this > correct, during the process of migration, both LV''s need to be writable. > In a default installation, drbd only supports one primary node, but I > need primary/primary, correct? > Here it comes to a cluster filesystem like OCFS - and I am stuck. How > can OCFS help me? Or am I totally wrong and what I want is impossible? > The main goal, of course, is to have a redundant fail over solution. > If my question is more related to drbd-users, I''ll switch over to the > other list. > > Thanks, > RainerHi Rainer, Basically the only prerequisite is that you run DRBD8, which gives you primary/primary, to get live migration working. You don''t have to bother with cluster aware FS. Both nodes will not have exclusive access at the same time so running ext3 is perfectly fine. You can have a look at this guide to get some additional information: http://www.asplund.nu/xencluster/xen-cluster-howto.html _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ********************************************************************************************** IMPORTANT: The contents of this email and any attachments are confidential. They are intended for the named recipient(s) only. If you have received this email in error, please notify the system manager or the sender immediately and do not disclose the contents to anyone or make copies thereof. *** eSafe scanned this email for viruses, vandals, and malicious content. *** ********************************************************************************************** _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Nov 10, 2008 at 10:07:48AM +0100, Rustedt, Florian wrote: Hello,> No no, > > that''s not right!! > You definetly need an network-filesystem for drbd primary/primary!Not for Xen live migration.> There are TWO systems, that access files for writing and both are not > running the same kernel ergo doesn''t "know", that the other side is > actually locking/writing a file!It is explained in http://www.drbd.org/users-guide/ch-xen.html. Xen needs primary/primary for block device write test during live migration only.>From that manual:"Enabling dual-primary mode is necessary because Xen, before initiating live migration, checks for write access on all VBDs a resource is configured to use on both the source and the destination host for the migration."> So you need OCFS & co. For doing the file-locking....If you want concurrent file access from two DomUs -- yes, otherwise no. Regards, Kupson -- Great software without the knowledge to run it is pretty useless. (Linux Gazette #1) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Well, ok, if you''re only running images with one side and have no shared virtual disks, then you only need write access from one side... But the cute advantage of live-migration & drbd & paravirtualization is, that you can do load-balancing with two active xen-hosts, so running several machines on both sides with several shared partitions, p.e. "/usr" or "/lib/modules", "/boot", "/usr/src", "/var/log" etc.. And in that moment, you''ve got concurrent access to the same file(system) and need an additional secutiry layer between. So, as far as i can see, we''re both right, depending on the eye of the beholder ;) Regards, Florian -----Ursprüngliche Nachricht----- Von: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von Rafal Kupka Gesendet: Montag, 10. November 2008 10:41 An: xen-users@lists.xensource.com Betreff: Re: [Xen-users] Live migration and drbd On Mon, Nov 10, 2008 at 10:07:48AM +0100, Rustedt, Florian wrote: Hello,> No no, > > that''s not right!! > You definetly need an network-filesystem for drbd primary/primary!Not for Xen live migration.> There are TWO systems, that access files for writing and both are not > running the same kernel ergo doesn''t "know", that the other side is > actually locking/writing a file!It is explained in http://www.drbd.org/users-guide/ch-xen.html. Xen needs primary/primary for block device write test during live migration only.>From that manual:"Enabling dual-primary mode is necessary because Xen, before initiating live migration, checks for write access on all VBDs a resource is configured to use on both the source and the destination host for the migration."> So you need OCFS & co. For doing the file-locking....If you want concurrent file access from two DomUs -- yes, otherwise no. Regards, Kupson -- Great software without the knowledge to run it is pretty useless. (Linux Gazette #1) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users ********************************************************************************************** IMPORTANT: The contents of this email and any attachments are confidential. They are intended for the named recipient(s) only. If you have received this email in error, please notify the system manager or the sender immediately and do not disclose the contents to anyone or make copies thereof. *** eSafe scanned this email for viruses, vandals, and malicious content. *** ********************************************************************************************** _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Nov 10, 2008 at 5:07 AM, Rustedt, Florian <Florian.Rustedt@smartnet.de> wrote:> But the cute advantage of live-migration & drbd & paravirtualization is, that you can do load-balancing with two active xen-hosts, so running several machines on both sides with several shared partitions, p.e. "/usr" or "/lib/modules", "/boot", "/usr/src", "/var/log" etc..that''s a totally different usecase. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Nov 10, 2008 at 09:46:29AM +0100, Daniel Asplund wrote:> Basically the only prerequisite is that you run DRBD8, which gives you > primary/primary, to get live migration working. You don''t have to > bother with cluster aware FS. Both nodes will not have exclusive > access at the same time so running ext3 is perfectly fine.Now I feel better :-)> You can have a look at this guide to get some additional information: > http://www.asplund.nu/xencluster/xen-cluster-howto.htmlThank you for the great howto - however, there is one question remaining open to me: What is the difference between disk=[ ''drbd:adrbdresource,w'', ] and disk=[ ''phy:/dev/drbdn,xvda,w'', ] ? Rainer _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Nov 10, 2008 at 8:46 AM, Daniel Asplund <danielsaori@gmail.com> wrote:>> Hi, >> >> I have a machine running several DomU''s, PVM''s als well as HVM''s. Each >> DomU resides in its own LV. >> Also, I managed drbd to replicate the LV''s to a second machine. Now, I >> want to try out Xen''s ability to live migrate a running DomU, that >> means, from machine1''s LV to machine2''s LV. If I understood this >> correct, during the process of migration, both LV''s need to be writable. >> In a default installation, drbd only supports one primary node, but I >> need primary/primary, correct? >> Here it comes to a cluster filesystem like OCFS - and I am stuck. How >> can OCFS help me? Or am I totally wrong and what I want is impossible? >> The main goal, of course, is to have a redundant fail over solution. >> If my question is more related to drbd-users, I''ll switch over to the >> other list. >> >> Thanks, >> Rainer > > Hi Rainer, > > Basically the only prerequisite is that you run DRBD8, which gives you > primary/primary, to get live migration working. You don''t have to > bother with cluster aware FS. Both nodes will not have exclusive > access at the same time so running ext3 is perfectly fine. > > You can have a look at this guide to get some additional information: > http://www.asplund.nu/xencluster/xen-cluster-howto.htmlI plan to try this soon, I''ve read the guide and it all makes sense to me, but could somebody explain why using drbd on top of lvm is preferable to using drbd with physical devices and lvm on the drbd devices? Thinking of the layers of devices: disk partition lvm pv lvm vg lvm lv (which is a physical disk to the VM) am I correct in thinking that drbd could be inserted anywhere in that list? where is best? why? Thanks Andy _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, 10 Nov 2008, Andrew Lyon wrote:>> http://www.asplund.nu/xencluster/xen-cluster-howto.html > > I plan to try this soon, I''ve read the guide and it all makes sense to > me, but could somebody explain why using drbd on top of lvm is > preferable to using drbd with physical devices and lvm on the drbd > devices?I would also love to know the answer to this, currently I am running; Disk > DRBD > LVM2 > GFS > xen tap:aio and: Disk > DRBD > LVM2 > xen phy:{lvm lv} The 2nd is obliviously faster, then the first, but I have not moved everything over yet. Would moving DRBD on LVM2 make things faster or slower? The downside I can see is that you need to build a DRBD config for every domu, what makes this overhead worth it?><>Nathan Stratton CTO, BlinkMind, Inc. nathan at robotics.net nathan at blinkmind.com http://www.robotics.net http://www.blinkmind.com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Nov 10, 2008 at 7:48 PM, Nathan Stratton <nathan@robotics.net> wrote:> On Mon, 10 Nov 2008, Andrew Lyon wrote: > >>> http://www.asplund.nu/xencluster/xen-cluster-howto.html >> >> I plan to try this soon, I''ve read the guide and it all makes sense to >> me, but could somebody explain why using drbd on top of lvm is >> preferable to using drbd with physical devices and lvm on the drbd >> devices? > > I would also love to know the answer to this, currently I am running; > > Disk > DRBD > LVM2 > GFS > xen tap:aio > > and: > > Disk > DRBD > LVM2 > xen phy:{lvm lv}What happens if you make a lvm snapshot in that setup (the 2nd)? I assume lvm on the second node would need to re-read the lvm metadata? And if you extend the logical volume? I''ve spent a lot of time googling this and while I can find plenty of documentation I am unable to find anything that explains the reasoning and theory behind it.> > The 2nd is obliviously faster, then the first, but I have not moved > everything over yet. > > Would moving DRBD on LVM2 make things faster or slower? The downside I can > see is that you need to build a DRBD config for every domu, what makes this > overhead worth it? > > >> <> > > Nathan Stratton CTO, BlinkMind, Inc. > nathan at robotics.net nathan at blinkmind.com > http://www.robotics.net http://www.blinkmind.com >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>... but could somebody explain why using drbd on top of >lvm is preferable to using drbd with physical devices and lvm on thedrbd devices? I could only imagine, that this was born on the difficulty running drbd as primary|slave so the slave node couldn''t find the LVMs'' because it couldn''t access the device. In that case, you can start LVM not prior to setting the primary state on drbd. So are we running our Nagios, two nodes, primary/slave, heartbeat for LVM-startover on the change... As you can see, LVM over DRBD is no special trick.. Now, with primary|primary today, there''s no sense in keeping LVM "down under" in my eyes. However, there was no real sence for this before.. Seems to me, as it is just a textual skeleton from previous doku-versions that''s not prooved again or born in the ease of usage because the system can find the LVMs independent of the drbd ..? Regards, Florian ********************************************************************************************** IMPORTANT: The contents of this email and any attachments are confidential. They are intended for the named recipient(s) only. If you have received this email in error, please notify the system manager or the sender immediately and do not disclose the contents to anyone or make copies thereof. *** eSafe scanned this email for viruses, vandals, and malicious content. *** ********************************************************************************************** _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users