Hi - Is there someone around who successfully got cLVM on Debian/Lenny working? I was wondering if I was the only one facing problems with... Thanks in anticipation, -- Olivier Le Cam Département des Technologies de l''Information et de la Communication CRDP de l''académie de Versailles _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thiago Camargo Martins Cordeiro
2009-Apr-15 18:05 UTC
Re: [Xen-users] cLVM on Debian/Lenny
Hi! I would also like to obtain this information too, if anyone knows how to configure and use CLVM on Debian Lenny, please share it! The Xen Live CD v2.0 will be enhanced with the CLVM! ;-) - Thiago 2009/4/15 Olivier Le Cam <Olivier.LeCam@crdp.ac-versailles.fr>> Hi - > > Is there someone around who successfully got cLVM on Debian/Lenny > working? I was wondering if I was the only one facing problems with... > > Thanks in anticipation, > > -- > Olivier Le Cam > Département des Technologies de l''Information et de la Communication > CRDP de l''académie de Versailles > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Olivier Le Cam <Olivier.LeCam@crdp.ac-versailles.fr> writes:> Is there someone around who successfully got cLVM on Debian/Lenny > working?Yes. There isn''t anything to set up on clvm, once you set up the cluster itself. There are usable guides on setting up RedHat clusters (it doesn''t matter the least you''re running Debian), and it''s totally unrelated to Xen. -- Regards, Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, Apr 16, 2009 at 10:58 AM, Ferenc Wagner <wferi@niif.hu> wrote:> Olivier Le Cam <Olivier.LeCam@crdp.ac-versailles.fr> writes: > >> Is there someone around who successfully got cLVM on Debian/Lenny >> working? > > Yes. There isn''t anything to set up on clvm, once you set up the > cluster itself. There are usable guides on setting up RedHat clusters > (it doesn''t matter the least you''re running Debian), and it''s totally > unrelated to Xen.now that RedHat''s cman is based on openAIS, i''ve seen that SLES11 clvm skips cman, so you can simply install openais and clvm. the result is a much simpler stack than RedHat''s. would Debian packages work similarly? -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Javier Guerra <javier@guerrag.com> writes:> On Thu, Apr 16, 2009 at 10:58 AM, Ferenc Wagner <wferi@niif.hu> wrote: > >> Olivier Le Cam <Olivier.LeCam@crdp.ac-versailles.fr> writes: >> >>> Is there someone around who successfully got cLVM on Debian/Lenny >>> working? >> >> Yes. There isn''t anything to set up on clvm, once you set up the >> cluster itself. There are usable guides on setting up RedHat clusters >> (it doesn''t matter the least you''re running Debian), and it''s totally >> unrelated to Xen. > > now that RedHat''s cman is based on openAIS, i''ve seen that SLES11 clvm > skips cman, so you can simply install openais and clvm. the result is > a much simpler stack than RedHat''s. > > would Debian packages work similarly?Currently they don''t, clvm depends on cman (which in turn depends on openais). But as there isn''t anything Debian specific around here, in principle surely they could. But I haven''t heard about the possibility of leaving out cman, even though I loosely follow the RedHat cluster mailing list. Have you got a reference? -- Cheers, Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, Apr 16, 2009 at 11:22 AM, Ferenc Wagner <wferi@niif.hu> wrote:> Currently they don''t, clvm depends on cman (which in turn depends on > openais). But as there isn''t anything Debian specific around here, in > principle surely they could. > > But I haven''t heard about the possibility of leaving out cman, even > though I loosely follow the RedHat cluster mailing list. Have you got > a reference?i just tried the SLES11 packages: clvm just depends on openais. install both and it just works. great if you don''t want to use GFS (Novell likes OCFS) -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Javier Guerra <javier@guerrag.com> writes:> On Thu, Apr 16, 2009 at 11:22 AM, Ferenc Wagner <wferi@niif.hu> wrote: >> Currently they don''t, clvm depends on cman (which in turn depends on >> openais). But as there isn''t anything Debian specific around here, in >> principle surely they could. >> >> But I haven''t heard about the possibility of leaving out cman, even >> though I loosely follow the RedHat cluster mailing list. Have you got >> a reference? > > i just tried the SLES11 packages: clvm just depends on openais. > install both and it just works. great if you don''t want to use GFS > (Novell likes OCFS)Just works?! How does it know about the cluster? What do you configure? The clvmd in Debian Lenny (2.02.39) links to libcman. -- Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, Apr 16, 2009 at 11:41 AM, Ferenc Wagner <wferi@niif.hu> wrote:> Just works?! How does it know about the cluster? What do you > configure? The clvmd in Debian Lenny (2.02.39) links to libcman.you configure openAIS. i haven''t checked the contents of either package (yet); but reading RedHat docs, it seems that cman is now mostly a compatibility layer, to let existing users keep the same config files. maybe libcman is still there, but the redhat-style configs are gone -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Javier Guerra wrote:> On Thu, Apr 16, 2009 at 10:58 AM, Ferenc Wagner <wferi@niif.hu> wrote: >> Olivier Le Cam <Olivier.LeCam@crdp.ac-versailles.fr> writes: >> >>> Is there someone around who successfully got cLVM on Debian/Lenny >>> working? >> Yes. There isn''t anything to set up on clvm, once you set up the >> cluster itself. There are usable guides on setting up RedHat clusters >> (it doesn''t matter the least you''re running Debian), and it''s totally >> unrelated to Xen. > > now that RedHat''s cman is based on openAIS, i''ve seen that SLES11 clvm > skips cman, so you can simply install openais and clvm. the result is > a much simpler stack than RedHat''s. > > would Debian packages work similarly?Hi - Actually not, the Debian Lenny version is still based on cman. But thanks to your great informations, I''ve been able to compile the debian''s clvm package in order to use openais. I''ve been impressed how it is easy to get both to work in opposition to cman and manual fencing I always suffered to deal with! Exactly what I was looking for from always for my xen servers sharing the same FC storage unit! Feel free to contact me if you need instructions or want to get a copy of the clvm deb package with openais for lenny on amd64. Enjoy your week end! -- Olivier Le Cam _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Apr 17, 2009 at 8:22 AM, Olivier Le Cam <Olivier.LeCam@crdp.ac-versailles.fr> wrote:> Feel free to contact me if you need instructions or want to get a copy of > the clvm deb package with openais for lenny on amd64.sure, please tell your procedure. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Anno domini 2009 Olivier Le Cam scripsit: Hi!> Actually not, the Debian Lenny version is still based on cman. But > thanks to your great informations, I''ve been able to compile the > debian''s clvm package in order to use openais. I''ve been impressed how > it is easy to get both to work in opposition to cman and manual fencing > I always suffered to deal with! Exactly what I was looking for from > always for my xen servers sharing the same FC storage unit!> Feel free to contact me if you need instructions or want to get a copy > of the clvm deb package with openais for lenny on amd64.Could you share your debian/ directory from the updates packages? Thanks Ciao Max -- Follow the white penguin. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thiago Camargo Martins Cordeiro
2009-Apr-17 19:01 UTC
Re: [Xen-users] Re: cLVM on Debian/Lenny
Yes! Please! Share your entire procedure and conf files! - Thiago 2009/4/17 Javier Guerra <javier@guerrag.com>> On Fri, Apr 17, 2009 at 8:22 AM, Olivier Le Cam > <Olivier.LeCam@crdp.ac-versailles.fr> wrote: > > Feel free to contact me if you need instructions or want to get a copy of > > the clvm deb package with openais for lenny on amd64. > > sure, please tell your procedure. > > > > -- > Javier > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi - Thiago Camargo Martins Cordeiro wrote:> Yes! Please! Share your entire procedure and conf files! > > - > Thiago > > 2009/4/17 Javier Guerra <javier@guerrag.com <mailto:javier@guerrag.com>> > > On Fri, Apr 17, 2009 at 8:22 AM, Olivier Le Cam > <Olivier.LeCam@crdp.ac-versailles.fr > <mailto:Olivier.LeCam@crdp.ac-versailles.fr>> wrote: > > Feel free to contact me if you need instructions or want to get a > copy of > > the clvm deb package with openais for lenny on amd64. > > sure, please tell your procedure.There you are: http://h2o.glou.fr/post/2009/04/20/clvm-openais-on-Debian/Lenny HTH 73, -- Olivier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Olivier Le Cam wrote:> Hi - > > Thiago Camargo Martins Cordeiro wrote: >> Yes! Please! Share your entire procedure and conf files! >> >> - >> Thiago >> >> 2009/4/17 Javier Guerra <javier@guerrag.com <mailto:javier@guerrag.com>> >> >> On Fri, Apr 17, 2009 at 8:22 AM, Olivier Le Cam >> <Olivier.LeCam@crdp.ac-versailles.fr >> <mailto:Olivier.LeCam@crdp.ac-versailles.fr>> wrote: >> > Feel free to contact me if you need instructions or want to get a >> copy of >> > the clvm deb package with openais for lenny on amd64. >> >> sure, please tell your procedure. > > There you are: > http://h2o.glou.fr/post/2009/04/20/clvm-openais-on-Debian/Lenny >Thanks, that''s very interesting. A stupid question just to better understand how cLVM works. Forgive me. I think of cLVM as the LVM with the cluster extension. Any setting on a shared storage is visible by all the nodes of the cluster. But what exactly does this mean? At which level (VG or LV) does the locking library work? Can I enable (active) the same VG on different nodes? If yes, how cLVM prevent split-brain? I would eventually be able to load a LV only on one node? Thanks, Jan Can I balance the load running the VMs on several nodes? _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I think of cLVM as the LVM with the cluster extension. Any setting on > a > shared storage is visible by all the nodes of the cluster. > But what exactly does this mean? At which level (VG or LV) does the > locking library work? Can I enable (active) the same VG on different > nodes? If yes, how cLVM prevent split-brain? I would eventually be > able > to load a LV only on one node?cLVM just keeps LVM changes consistent across the cluster. It''s a system of locks (via dlm) that prevents one node from making LVM changes while another node is making them. You can mount a LV on multiple nodes simultaneously and completely trash your data if you like -- clvm doesn''t prevent this -- and in fact this behavior is necessary if you''re going to use a cluster filesystem such as OCFS or GFS. John -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
John Madden wrote:>> I think of cLVM as the LVM with the cluster extension. Any setting on >> a >> shared storage is visible by all the nodes of the cluster. >> But what exactly does this mean? At which level (VG or LV) does the >> locking library work? Can I enable (active) the same VG on different >> nodes? If yes, how cLVM prevent split-brain? I would eventually be >> able >> to load a LV only on one node? >> > > cLVM just keeps LVM changes consistent across the cluster. It''s a > system of locks (via dlm) that prevents one node from making LVM changes > while another node is making them. You can mount a LV on multiple nodes > simultaneously and completely trash your data if you like -- clvm > doesn''t prevent this -- and in fact this behavior is necessary if you''re > going to use a cluster filesystem such as OCFS or GFS. > > John > > > > >Clear. cLVM does not provide a lock manager which manages *access* (I know EVMS2 does for instance) to a LV or VG from the nodes of a cluster. As you said, It just keeps LVM changes consistent. Let me say it provides a DLM at volume level. To prevent split-brain I need a DLM at file system level, of course, so a cluster-aware file system like OCFS2 is needed. But this does not make sense to me. Where are the pros of using LVM for my VMs if they eventually reside as file on a (cluster) file system? I loose all the features I want from LVM (snapshot and resizing). I think of a cluster where the VMs are using a physical block device (shared) based on LVM which provides me the LVM features as above and manage access to the volume it self. Does it make sense or I am completely "out"? Thanks, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Apr 20, 2009 at 9:19 PM, Jan Kalcic <jandot@googlemail.com> wrote:> But this does not make sense to me. Where are the pros of using LVM for > my VMs if they eventually reside as file on a (cluster) file system? I > loose all the features I want from LVM (snapshot and resizing). > > I think of a cluster where the VMs are using a physical block device > (shared) based on LVM which provides me the LVM features as above and > manage access to the volume it self. > > Does it make sense or I am completely "out"?I think you''re looking at it the wrong way. You still need a separate locking mechanism if you want to prevent a domU backend (be it LVM or file) from being used by two or more dom0 at the same time. Cluster file system (like GFS) can create a shared filesystem that can be mounted on two or more nodes at the same time, but it does NOT prevent you from (lets say) having a file-backed vm, located on GFS, that is used by two (or more) dom0 to as domU backend. End result: domU corruption. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> But this does not make sense to me. Where are the pros of using LVM for > my VMs if they eventually reside as file on a (cluster) file system? I > loose all the features I want from LVM (snapshot and resizing).You can use LVM inside LVM (although I think it''s said that you shouldn''t) to provide these features inside the domU. I wouldn''t do snapshotting and such from dom0 anyway. LVM isn''t designed to support virtualization, it''s just handy that it works well in a cluster. Use virt management tools (or at least good documentation) to keep LVs from getting mounted in the wrong places at the wrong times. Shoot, up until this point, all I''ve been doing is passing raw SAN LUNs around -- I''m only starting to look at using LVM to make space usage more efficient. =) John -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha wrote:> On Mon, Apr 20, 2009 at 9:19 PM, Jan Kalcic <jandot@googlemail.com> wrote: > >> But this does not make sense to me. Where are the pros of using LVM for >> my VMs if they eventually reside as file on a (cluster) file system? I >> loose all the features I want from LVM (snapshot and resizing). >> >> I think of a cluster where the VMs are using a physical block device >> (shared) based on LVM which provides me the LVM features as above and >> manage access to the volume it self. >> >> Does it make sense or I am completely "out"? >> > > I think you''re looking at it the wrong way. You still need a separate > locking mechanism if you want to prevent a domU backend (be it LVM or > file) from being used by two or more dom0 at the same time. > > Cluster file system (like GFS) can create a shared filesystem that can > be mounted on two or more nodes at the same time, but it does NOT > prevent you from (lets say) having a file-backed vm, located on GFS, > that is used by two (or more) dom0 to as domU backend. End result: > domU corruption. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > >Right, I think I almost get the point. cLVM can''t say node1 is accessing this LV so node2 can''t and vice versa. There is not a cluster lock mechanism of this type (what is called Cluster Segment Manager in EVMS2) Moreover, if there was this kind of lock mechanism, it would not work for VMs live migration just because multiple node can access to the same LV simultaneously. Instead, cLVM can notifies all the nodes whenever a LVM change occurs. That''s it. Thanks, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
John Madden wrote:> LVM isn''t designed to support virtualization, it''s just handy that it > works well in a cluster. Use virt management tools (or at least good > documentation) to keep LVs from getting mounted in the wrong places at > the wrong times.I see. Virt manager tools allow me to prevent this? How? Can you please point me to any document about it?> Shoot, up until this point, all I''ve been doing is > passing raw SAN LUNs around -- I''m only starting to look at using LVM to > make space usage more efficient. =) > > > >It''s more clear now. That''s still the only advantage of using LVM in a cluster. Efficiently and rationally use of disk space. Honestly, I was expecting more from cLVM :-( Thanks, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> Virt manager tools allow me to prevent this? How? Can you please point > me to any document about it?No, they don''t (can''t) prevent you from doing it as it is in fact necessary as part of live migration, they just make it more difficult to bork up. John -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Mon, Apr 20, 2009 at 9:19 AM, Jan Kalcic <jandot@googlemail.com> wrote:> cLVM does not provide a lock manager which manages *access* (I know > EVMS2 does for instance) to a LV or VG from the nodes of a cluster. As > you said, It just keeps LVM changes consistent. Let me say it provides a > DLM at volume level. To prevent split-brain I need a DLM at file system > level, of course, so a cluster-aware file system like OCFS2 is needed.you''re mixing levels a bit. - dm: (kernelspace) device mapper, creates ''virtual block devices''. any access on these is mapped to some access on a real device. - LVM: (mostly userspace) uses dm to create LVs. it manages some metadata blocks on the block devices, and loads the mapping tables into the kernel. when running on a single node it can do changes while online because dm supports a ''suspend'' operation; the sequence is: change metadata blocks, suspend logical devices, change mapping tables in the kernel, unsuspend. in theory, you can use LVM as is with shared block devices and several nodes; with two precautions: never mount a single LV on more than one node, and never do any change to the VG while other nodes are running. for that, you have to disconnect the VG from all nodes, do the change on the only node still connected, and reconnect all the rest. - lock manager: there are several, the one from the GFS stack is now based on openAIS. locks are an abstract service in the sense that can be used for several different things. - cLVM: (userspace) a small extension to LVM utilities, and a daemon (clvmd). any operation that would change a shared VG first acquires a lock. for that lock operation to succeed, the clvmd of all other nodes do a ''suspend''. when the lock is acquired, the first node can be sure that no write operation will happen on the device. it changes the metadata blocks and notifies all other nodes. the other nodes reload the mapping tables from the changed metadata blocks, and unsuspend the volume group to release the lock. in short, the only thing cLVM adds to ''bare'' LVM is to extend the online management capabilities of LVM to the whole cluster. on top of these, there can be several uses of these LVs. you can put non-cluster filesystems, as long as you never do a double mount; or you can use cluster filesystems, which will use a lock manager (may be the same as cLVM, or may be another one) to assure consistency at that level, or you can use them for Xen, with similar limitations, depending on the filesystems used by the DomUs. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Javier Guerra wrote:> On Mon, Apr 20, 2009 at 9:19 AM, Jan Kalcic <jandot@googlemail.com> wrote: > >> cLVM does not provide a lock manager which manages *access* (I know >> EVMS2 does for instance) to a LV or VG from the nodes of a cluster. As >> you said, It just keeps LVM changes consistent. Let me say it provides a >> DLM at volume level. To prevent split-brain I need a DLM at file system >> level, of course, so a cluster-aware file system like OCFS2 is needed. >> > > you''re mixing levels a bit. > > - dm: (kernelspace) device mapper, creates ''virtual block devices''. > any access on these is mapped to some access on a real device. > > - LVM: (mostly userspace) uses dm to create LVs. it manages some > metadata blocks on the block devices, and loads the mapping tables > into the kernel. when running on a single node it can do changes > while online because dm supports a ''suspend'' operation; the sequence > is: change metadata blocks, suspend logical devices, change mapping > tables in the kernel, unsuspend. > > in theory, you can use LVM as is with shared block devices and > several nodes; with two precautions: never mount a single LV on more > than one node, and never do any change to the VG while other nodes are > running. for that, you have to disconnect the VG from all nodes, do > the change on the only node still connected, and reconnect all the > rest. > > - lock manager: there are several, the one from the GFS stack is now > based on openAIS. locks are an abstract service in the sense that can > be used for several different things. > > - cLVM: (userspace) a small extension to LVM utilities, and a daemon > (clvmd). any operation that would change a shared VG first acquires > a lock. for that lock operation to succeed, the clvmd of all other > nodes do a ''suspend''. when the lock is acquired, the first node can > be sure that no write operation will happen on the device. it changes > the metadata blocks and notifies all other nodes. the other nodes > reload the mapping tables from the changed metadata blocks, and > unsuspend the volume group to release the lock. > > in short, the only thing cLVM adds to ''bare'' LVM is to extend the > online management capabilities of LVM to the whole cluster. > >That''s really clear now. Very interesting.> on top of these, there can be several uses of these LVs. you can put > non-cluster filesystems, as long as you never do a double mount; or > you can use cluster filesystems, which will use a lock manager (may be > the same as cLVM, or may be another one) to assure consistency at that > level, or you can use them for Xen, with similar limitations, > depending on the filesystems used by the DomUs. > >Now I think I can figure out that there is no difference between a domU based on a file block device on top of a cluster file system, let''s suppose OCFS2, and a domU based on physical block device on top of LVM (cLVM). In both case, I am eventually able to run the domU on both nodes having data corruption. Neither OCFS2 nor cLVM with their different lock managers can provide that mechanism to assure consistency. Right? If yes, I would say that the best solution should always be using LVM, at least it provides more features. Do you agree? _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> Now I think I can figure out that there is no difference between a > domU > based on a file block device on top of a cluster file system, let''s > suppose OCFS2, and a domU based on physical block device on top of LVM > (cLVM). In both case, I am eventually able to run the domU on both > nodes > having data corruption. Neither OCFS2 nor cLVM with their different > lock > managers can provide that mechanism to assure consistency. Right? If > yes, I would say that the best solution should always be using LVM, at > least it provides more features.OCFS2 is a filesystem, LVM is a volume manager. Don''t use the two products in the same sentence unless it''s to say "I formatted an LVM LV with OCFS2 instead of ext3." OCFS2 will allow you to run two domU''s (or dom0''s, or any host for that matter) sharing storage and writing to it at the same time without corruption because it is a cluster filesystem. You don''t need LVM to accomplish this, either, although you should use it too to manage the volumes. John -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jan Kalcic wrote:> Now I think I can figure out that there is no difference between a domU> based on a file block device on top of a cluster file system, let''s > suppose OCFS2, and a domU based on physical block device on top of LVM > (cLVM). In both case, I am eventually able to run the domU on both nodes > having data corruption. Neither OCFS2 nor cLVM with their different lock > managers can provide that mechanism to assure consistency. Right? If > yes, I would say that the best solution should always be using LVM, at > least it provides more features.right. they manage very different things (cLVM gives you blockdevices, while OCFS gives you files); but Xen can use either one for the same purpose (storage for DomUs), and neither of them will prevent several nodes clashing each other when using the same ''thing'' (file or blockdevice) at the same time. just like a non-cluster filesystem can''t prevent two different processes from corrupting a datafile by writing at the same time. for that there are some scripts for HA (hearbeat2) that manage Xen instances as ''cluster resources'', and make sure that each one is running on exactly one node. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> Jan Kalcic wrote:> Now I think I can figure out that there is no difference between a domU > > based on a file block device on top of a cluster file system, let''s > > suppose OCFS2, and a domU based on physical block device on top of LVM > > (cLVM). In both case, I am eventually able to run the domU on both nodes > > having data corruption. Neither OCFS2 nor cLVM with their different lock > > managers can provide that mechanism to assure consistency. Right? If > > yes, I would say that the best solution should always be using LVM, at > > least it provides more features.Wrong -- sort of. OCFS2, GFS, Lustre, etc., are CLUSTER filesystems, this "clash" you''re talking about is actually supposed to be happening. And it isn''t a "clash," it allows multiple writers. With a cluster filesystem, you should, for example, be able to have two domU''s on different dom0''s writing to the same tap:aio file-based filesystem residing on OCFS2. John -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
John Madden wrote:> > Jan Kalcic wrote:> Now I think I can figure out that there is no difference between a domU > > > based on a file block device on top of a cluster file system, let''s > > > suppose OCFS2, and a domU based on physical block device on top of LVM > > > (cLVM). In both case, I am eventually able to run the domU on both nodes > > > having data corruption. Neither OCFS2 nor cLVM with their different lock > > > managers can provide that mechanism to assure consistency. Right? If > > > yes, I would say that the best solution should always be using LVM, at > > > least it provides more features. > > Wrong -- sort of. OCFS2, GFS, Lustre, etc., are CLUSTER filesystems, > this "clash" you''re talking about is actually supposed to be happening. > And it isn''t a "clash," it allows multiple writers. With a cluster > filesystem, you should, for example, be able to have two domU''s on > different dom0''s writing to the same tap:aio file-based filesystem > residing on OCFS2.yes, multiple nodes can use the same (cluster) filesystem; but if you store file-based DomUs on that filesystem, you could start the same DomU on two nodes, with the same nefarious results. that''s the non-protection he''s talking about. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Javier Guerra wrote:> > > right. they manage very different things (cLVM gives you blockdevices, while OCFS gives you files); but Xen can use either one for the same purpose (storage for DomUs), and neither of them will prevent several nodes clashing each other when using the same ''thing'' (file or blockdevice) at the same time. just like a non-cluster filesystem can''t prevent two different processes from corrupting a datafile by writing at the same time. > > for that there are some scripts for HA (hearbeat2) that manage Xen instances as ''cluster resources'', and make sure that each one is running on exactly one node. > > >That''s very clear and you got it, HA. With Xen in HA, like any other kind of resource, you can obviously start manually a domU from console or virt-manager and so you are not sure it is running on one node only. Of course, you don''t manage clustered resources (usually services or whatever) without hearbeat, otherwise you end up to face strange and unpredictable behavior. What would be really useful here, is a kind of lock which prevent a domU already running on a dom0 to be started on another domU.I wrongly expected this was managed by cLVM and now which everything is explained I see the reason why it cannot be done and I look forward to see such a kind of mechanism implemented in Xen. Thanks, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
John Madden wrote:>> Jan Kalcic wrote:> Now I think I can figure out that there is no difference between a domU >> >>> based on a file block device on top of a cluster file system, let''s >>> suppose OCFS2, and a domU based on physical block device on top of LVM >>> (cLVM). In both case, I am eventually able to run the domU on both nodes >>> having data corruption. Neither OCFS2 nor cLVM with their different lock >>> managers can provide that mechanism to assure consistency. Right? If >>> yes, I would say that the best solution should always be using LVM, at >>> least it provides more features. >>> > > Wrong -- sort of. OCFS2, GFS, Lustre, etc., are CLUSTER filesystems, > this "clash" you''re talking about is actually supposed to be happening. > And it isn''t a "clash," it allows multiple writers. With a cluster > filesystem, you should, for example, be able to have two domU''s on > different dom0''s writing to the same tap:aio file-based filesystem > residing on OCFS2. > > John > > > > > >I think you can do the same with cLVM. You run different domU on different dom0 writing on the same VG. So you can do load balancing in the same way. Thanks, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jan Kalcic wrote:> Javier Guerra wrote: > > > > > > right. they manage very different things (cLVM gives you blockdevices, while OCFS gives you files); but Xen can use either one for the same purpose (storage for DomUs), and neither of them will prevent several nodes clashing each other when using the same ''thing'' (file or blockdevice) at the same time. just like a non-cluster filesystem can''t prevent two different processes from corrupting a datafile by writing at the same time. > > > > for that there are some scripts for HA (hearbeat2) that manage Xen instances as ''cluster resources'', and make sure that each one is running on exactly one node. > > > > > > > That''s very clear and you got it, HA. With Xen in HA, like any other > kind of resource, you can obviously start manually a domU from console > or virt-manager and so you are not sure it is running on one node only. > Of course, you don''t manage clustered resources (usually services or > whatever) without hearbeat, otherwise you end up to face strange and > unpredictable behavior. What would be really useful here, is a kind of > lock which prevent a domU already running on a dom0 to be started on > another domU.I wrongly expected this was managed by cLVM and now which > everything is explained I see the reason why it cannot be done and I > look forward to see such a kind of mechanism implemented in Xen.that''s exactly what heartbeat2 does. -- Javier _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> I think you can do the same with cLVM. You run different domU on > different dom0 writing on the same VG. So you can do load balancing in > the same way.No. -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> yes, multiple nodes can use the same (cluster) filesystem; but if you > store file-based DomUs on that filesystem, you could start the same > DomU on two nodes, with the same nefarious results. that''s the > non-protection he''s talking about.What would those results be? I imagine the second node would just fail to boot given the delays and whatnot in locking, but it shouldn''t corrupt anything, should it? John -- John Madden Sr UNIX Systems Engineer Ivy Tech Community College of Indiana jmadden@ivytech.edu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
John Madden wrote:>> I think you can do the same with cLVM. You run different domU on >> different dom0 writing on the same VG. So you can do load balancing in >> the same way. >> > > No. > > > > >What do you mean? Explain your sentence please. Thanks, Jan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users