Gluster users - Jul 2012 - Gluster v 3.3 with KVM and High Availability

I am looking to build a proof of concept cluster using Gluster as the
storage back-end.

I have looked through the mailing list archives and have seen that
many others before have done this but what I can find is what
technologies were used to complete the task. Also there have been many
reports on poor performance with running KVM images on Gluster has
version 3.3 fixed many of these "problems"?

Would anyone care to share what they are using for their technology
stack and any comments on how it works?

Thanks,

-- 
 _
/-\ ndrew Niemantsverdriet
Linux System Administrator
Academic Computing
(406) 238-7360
Rocky Mountain College
1511 Poly Dr.
Billings MT, 59102

3.3 brought granular locking, which is very useful with VMs.  There's 
been talk on the list about running VMs on Gluster that you can search for.

I tried it in 3.2.6 and gave up, I haven't tried it on 3.3 yet.

Jeff White - GNU+Linux Systems Engineer
University of Pittsburgh - CSSD


On 07/10/2012 05:32 PM, Andrew Niemantsverdriet wrote:
> I am looking to build a proof of concept cluster using Gluster as the
> storage back-end.
>
> I have looked through the mailing list archives and have seen that
> many others before have done this but what I can find is what
> technologies were used to complete the task. Also there have been many
> reports on poor performance with running KVM images on Gluster has
> version 3.3 fixed many of these "problems"?
>
> Would anyone care to share what they are using for their technology
> stack and any comments on how it works?
>
> Thanks,
>

On Tue, Jul 10, 2012 at 03:32:20PM -0600, Andrew Niemantsverdriet
wrote:
> I have looked through the mailing list archives and have seen that
> many others before have done this but what I can find is what
> technologies were used to complete the task.
I tested glusterfs 3.3, KVM/libvirt, ubuntu 12.04. Nothing else required.
> Also there have been many
> reports on poor performance with running KVM images on Gluster has
> version 3.3 fixed many of these "problems"?
Write performance remains poor. It would probably be OK to use Gluster as a
backend for a mostly-static VM, which mounts a separate gluster volume for
large data (which also has the advantage that this data is shareable to
other VMs and for easy backup)

We store KVM images on Gluster 3.3 for our OpenStack cloud. It performs ok
what I've seen so far (we use cache=writeback).
>From my short experience with Gluster 3.3 the granular locking stuff is not
working as expected. We made some tests with a replicate+distributed
volume, the VM was locked during the self-heal. I have no clue why.

Cheers,
Christian


2012/7/10 Andrew Niemantsverdriet <andrew at rocky.edu>
> I am looking to build a proof of concept cluster using Gluster as the
> storage back-end.
>
> I have looked through the mailing list archives and have seen that
> many others before have done this but what I can find is what
> technologies were used to complete the task. Also there have been many
> reports on poor performance with running KVM images on Gluster has
> version 3.3 fixed many of these "problems"?
>
> Would anyone care to share what they are using for their technology
> stack and any comments on how it works?
>
> Thanks,
>
> --
>  _
> /-\ ndrew Niemantsverdriet
> Linux System Administrator
> Academic Computing
> (406) 238-7360
> Rocky Mountain College
> 1511 Poly Dr.
> Billings MT, 59102
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120711/ed92c79d/attachment.html>

On Thu, Jul 12, 2012 at 09:01:48AM +0300, Samuli Heinonen
wrote:
> On 11.07.2012 21:35, Brian Candler wrote:
> >On Wed, Jul 11, 2012 at 12:55:50PM -0500, Mark Nipper wrote:
> >>	Would that be using something like O_DIRECT which FUSE
> >>doesn't support at the moment?
> >
> >Yes. FUSE does support it in recent kernels (3.4), and I tried it.
> >Nothing
> >happened until I also mounted with -o direct-io-mode=enable; with
> >that and
> >cache=none, the VM was unable to start up at all.
> 
> This FUSE patch has been backported into RHEL 6.3 and should also
> work with latest 6.2 kernels.
> 
> Iirc, it also worked with cache=none without any special mount
> options, but unfortunately I don't have possibility to test if
> that's true :(
When I tested it, it did work with cache=none without any special mount
options and a kernel which supports FUSE+O_DIRECT, but the performance was
no better than without.

With -o direct-io-mode=enable the VM wouldn't start at all.