Hi Folks, Anybody have any recent experience building a small cluster that combines GlusterFS and Xen (with bonus points for a Debian environment)? We''re getting ready to shift from a 2-node Xen/Pacemaker/DRBD environment to a 4-node environment - and I''d really like to be able to create/migrate/failover/delete VMs transparently across all 4 nodes. Seems like Gluster might be a good storage platform. Anybody have any recent experience they could share along these lines? Thanks very much, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
Hi, 2012/2/15 Miles Fidelman <mfidelman@meetinghouse.net>:> Anybody have any recent experience they could share along these lines?If nobody reponds, look in the list archives. I doubt you will find many people recommend it. florian -- the purpose of libvirt is to provide an abstraction layer hiding all xen features added since 2006 until they were finally understood and copied by the kvm devs.
Miles Fidelman
2012-Feb-15 21:30 UTC
Re: any recent experience combining GlusterFS w/ Xen?
Florian Heigl wrote:> Hi, > > 2012/2/15 Miles Fidelman<mfidelman@meetinghouse.net>: >> Anybody have any recent experience they could share along these lines? > If nobody reponds, look in the list archives. I doubt you will find > many people recommend it. >I''ve seen some comments in the archives relating to previous versions of Gluster, particularly around resynchronization delays that can effect VM migration - things that purport to be fixed in later versions. And... RedHat seems to be pushing some Gluster-based appliances for use in a virtualized environment. Hence my interest in RECENT experience :-) -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
I asked the same thing less than a year ago and the answer was not very encouraging. I gave it a try anyway and it was a miserable failure. I did make my attempt using XCP 1.0 though, so you might want to give it a try again with Xen or XCP 1.1. I don''t have the particulars available on the failure, but I can look around and find my notes if you would like. On Wed, Feb 15, 2012 at 3:30 PM, Miles Fidelman <mfidelman@meetinghouse.net> wrote:> Florian Heigl wrote: >> >> Hi, >> >> 2012/2/15 Miles Fidelman<mfidelman@meetinghouse.net>: >>> >>> Anybody have any recent experience they could share along these lines? >> >> If nobody reponds, look in the list archives. I doubt you will find >> many people recommend it. >> > I''ve seen some comments in the archives relating to previous versions of > Gluster, particularly around resynchronization delays that can effect VM > migration - things that purport to be fixed in later versions. And... > RedHat seems to be pushing some Gluster-based appliances for use in a > virtualized environment. > > Hence my interest in RECENT experience :-) > > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users
dwight at supercomputer.org
2012-Feb-15 22:06 UTC
Re: any recent experience combining GlusterFS w/ Xen?
On Wednesday 15 February 2012 01:30:58 pm Miles Fidelman wrote:> Florian Heigl wrote: > > Hi, > > > > 2012/2/15 Miles Fidelman<mfidelman@meetinghouse.net>: > >> Anybody have any recent experience they could share along these > >> lines? > > > > If nobody reponds, look in the list archives. I doubt you will > > find many people recommend it. > > I''ve seen some comments in the archives relating to previous > versions of Gluster, particularly around resynchronization delays > that can effect VM migration - things that purport to be fixed in > later versions. And... RedHat seems to be pushing some > Gluster-based appliances for use in a virtualized environment. > > Hence my interest in RECENT experience :-)That''s interesting. FWIW, I''m evaluating Nimbula right now, and they claim that they are using Gluster along with DRBD. They are CentOS/RH based, so that fits in with your comment about RedHat. Can''t say yet how well it works though. -dwight-
Miles Fidelman
2012-Feb-15 22:09 UTC
Re: any recent experience combining GlusterFS w/ Xen?
Scott Damron wrote:> I asked the same thing less than a year ago and the answer was not > very encouraging. I gave it a try anyway and it was a miserable > failure. I did make my attempt using XCP 1.0 though, so you might > want to give it a try again with Xen or XCP 1.1. I don''t have the > particulars available on the failure, but I can look around and find > my notes if you would like. >actually, I''m a bit more curious as to which version of GlusterFS you were using - I expect any improvements would be coming from that end of things. thanks! Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
I looked at them a while back. The first "Preview" release was based on Debain, then when I went to upgrade, it was based on CentOS...what a PITA! the 40 CPU free license was good though. On Wed, Feb 15, 2012 at 4:06 PM, dwight at supercomputer.org <dwight@supercomputer.org> wrote:> On Wednesday 15 February 2012 01:30:58 pm Miles Fidelman wrote: >> Florian Heigl wrote: >> > Hi, >> > >> > 2012/2/15 Miles Fidelman<mfidelman@meetinghouse.net>: >> >> Anybody have any recent experience they could share along these >> >> lines? >> > >> > If nobody reponds, look in the list archives. I doubt you will >> > find many people recommend it. >> >> I''ve seen some comments in the archives relating to previous >> versions of Gluster, particularly around resynchronization delays >> that can effect VM migration - things that purport to be fixed in >> later versions. And... RedHat seems to be pushing some >> Gluster-based appliances for use in a virtualized environment. >> >> Hence my interest in RECENT experience :-) > > That''s interesting. FWIW, I''m evaluating Nimbula right now, and they > claim that they are using Gluster along with DRBD. They are > CentOS/RH based, so that fits in with your comment about RedHat. > Can''t say yet how well it works though. > > -dwight- > > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users
dwight at supercomputer.org
2012-Feb-15 22:32 UTC
Re: any recent experience combining GlusterFS w/ Xen?
That little bit of information was valuable. Thank you. On Wednesday 15 February 2012 02:14:55 pm Scott Damron wrote:> I looked at them a while back. The first "Preview" release was > based on Debain, then when I went to upgrade, it was based on > CentOS...what a PITA! the 40 CPU free license was good though. > > On Wed, Feb 15, 2012 at 4:06 PM, dwight at supercomputer.org > > <dwight@supercomputer.org> wrote:[snip]> > > > That''s interesting. FWIW, I''m evaluating Nimbula right now, and > > they claim that they are using Gluster along with DRBD. They are > > CentOS/RH based, so that fits in with your comment about RedHat. > > Can''t say yet how well it works though. > > > > -dwight- > > > > > > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users
Miles Fidelman
2012-Feb-16 18:50 UTC
Re: any recent experience combining GlusterFS w/ Xen?
dwight at supercomputer.org wrote:> That''s interesting. FWIW, I''m evaluating Nimbula right now, and they > claim that they are using Gluster along with DRBD. They are CentOS/RH > based, so that fits in with your comment about RedHat. Can''t say yet > how well it works though.Gluster + DRBD seems an odd combination. Seems like DRBD would be extraneous. -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
dwight at supercomputer.org
2012-Feb-16 20:44 UTC
Re: any recent experience combining GlusterFS w/ Xen?
On Thursday 16 February 2012 10:50:00 am Miles Fidelman wrote:> dwight at supercomputer.org wrote: > > That''s interesting. FWIW, I''m evaluating Nimbula right now, and > > they claim that they are using Gluster along with DRBD. They are > > CentOS/RH based, so that fits in with your comment about RedHat. > > Can''t say yet how well it works though. > > Gluster + DRBD seems an odd combination. Seems like DRBD would be > extraneous.I honestly can''t even say if it has Gluster. I was just going from the marketing lit and high level claims. I heard late yesterday (from someone who was at a conference here) that they were using something else. I''ve just gotten it installed this morning, and so I''m going to play around with it. I see it also claims to support NFS. I''m especially curious if it has the same fatal flaw that XCP has, with regards to NFS (which I asked recently about on this list, and got no response). This should be amusing. -dwight-
Oh I did the reply thing again.... I am wondering how GlusterFS has performed providing VDI storage? On Feb 22, 2012 2:06 PM, "dwight at supercomputer.org" < dwight@supercomputer.org> wrote:> Sure, which part are you interested in? DRBD appears to be used for > one of the internal subsystems (a database). Nimbula uses Gluster, > along with Hadoop for the rest of their Cloud. > > So far, I like what I see, mostly. It''s clearly done by a bunch of > experienced systems guys. E.g. they have a sane CLI, unlike XCP. In > short, they''ve put together a bunch of sophisticated technology, > using people who really understand what they are doing. > > The biggest question in my mind is scalability. Right now, the > networking setup is questionable. Your weakest link appears to be > the backplane of the router. That is, they use one single ethernet > segment for everything. That has clearly got to change. But still, > for a young product, I''m liking what I''m seeing. > > Note that I haven''t installed their recent upgrade yet, so things may > have changed with their 2.x product. > > -dwight- > > On Wednesday 22 February 2012 05:44:17 am you wrote: > > This thread has become relevant to my interests. > > > > Let me know how it goes. > > > > On Thu, Feb 16, 2012 at 3:44 PM, dwight at supercomputer.org < > > > > dwight@supercomputer.org> wrote: > > > On Thursday 16 February 2012 10:50:00 am Miles Fidelman wrote: > > > > dwight at supercomputer.org wrote: > > > > > That''s interesting. FWIW, I''m evaluating Nimbula right now, > > > > > and they claim that they are using Gluster along with DRBD. > > > > > They are CentOS/RH based, so that fits in with your comment > > > > > about RedHat. Can''t say yet how well it works though. > > > > > > > > Gluster + DRBD seems an odd combination. Seems like DRBD > > > > would be extraneous. > > > > > > I honestly can''t even say if it has Gluster. I was just going > > > from the marketing lit and high level claims. I heard late > > > yesterday (from someone who was at a conference here) that they > > > were using something else. > > > > > > I''ve just gotten it installed this morning, and so I''m going to > > > play around with it. I see it also claims to support NFS. I''m > > > especially curious if it has the same fatal flaw that XCP has, > > > with regards to NFS (which I asked recently about on this list, > > > and got no response). > > > > > > This should be amusing. > > > > > > -dwight- > > > > > > > > > _______________________________________________ > > > Xen-users mailing list > > > Xen-users@lists.xensource.com > > > http://lists.xensource.com/xen-users > > >_______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users