Gluster users - Nov 2016 - Comparison with other SDS

I did a very simple and stupid LizardFS installation this weekend.
Same configuration as gluster, same nodes, same disks. Both set with
replica 2, same ZFS filesystem on each disks/bricks

LizardFS installation took 10 minutes on all servers (1 client that
i've also used as master and 2 chunkservers), Gluster took less than 5
minutes from 0 to a working cluster. (just apt-get, gluster peer probe
and volume create)

Performances:
extracting this:
https://cdn.kernel.org/pub/linux/kernel/v4.x/testing/linux-4.9-rc5.tar.xz
took 45 minutes (forty-five minutes) on Gluster, 4 minutes (four
minutes) on LizardFS. It's not a typo. 45 minutes vs 4.

removing the whole directory tree: in Lizard less than 4 minutes, in
gluster i've stopped the process after about 20 minutes.

Both were configured with sharding (64M). LizardFS/MooseFS has this hardcoded.

Can this be related to the metadata server? I don't think so. Gluster
is able to know where a file is without asking to the brick servers.
In fact, gluster should be faster, as there isn't any query to make to
a metadata server when reading/writing.

Failures: LizardFS detect properly a missing/corrupted (like bitrot)
chunk but I was unable to understand it's recovery process. I've not
tried the bit-rot feature in gluster.

Can someone explain me why Lizard is 10 times faster than gluster?
This is not a flame, I would only like to know the technical
differences between these two software

Hi Gandalf,

Can you provide more information about your setup?
How many nodes? What disk sizes? Are they VMs or physical machines? What is the
speed of the network?
What OS are you running Lizard on , and finally how are the disks setup?

We use MooseFS, Nexenta, Gluster and Ceph here, and in our tests we see very
little difference in speeds.
Some of these setups have advantages with many clients writing at once.

Thanks,

----- Mensagem original -----
De: "Gandalf Corvotempesta" <gandalf.corvotempesta at gmail.com>
Para: "gluster-users" <Gluster-users at gluster.org>
Enviadas: Segunda-feira, 14 de novembro de 2016 9:00:00
Assunto: [Gluster-users] Comparison with other SDS

I did a very simple and stupid LizardFS installation this weekend.
Same configuration as gluster, same nodes, same disks. Both set with
replica 2, same ZFS filesystem on each disks/bricks

LizardFS installation took 10 minutes on all servers (1 client that
i've also used as master and 2 chunkservers), Gluster took less than 5
minutes from 0 to a working cluster. (just apt-get, gluster peer probe
and volume create)

Performances:
extracting this:
https://cdn.kernel.org/pub/linux/kernel/v4.x/testing/linux-4.9-rc5.tar.xz
took 45 minutes (forty-five minutes) on Gluster, 4 minutes (four
minutes) on LizardFS. It's not a typo. 45 minutes vs 4.

removing the whole directory tree: in Lizard less than 4 minutes, in
gluster i've stopped the process after about 20 minutes.

Both were configured with sharding (64M). LizardFS/MooseFS has this hardcoded.

Can this be related to the metadata server? I don't think so. Gluster
is able to know where a file is without asking to the brick servers.
In fact, gluster should be faster, as there isn't any query to make to
a metadata server when reading/writing.

Failures: LizardFS detect properly a missing/corrupted (like bitrot)
chunk but I was unable to understand it's recovery process. I've not
tried the bit-rot feature in gluster.

Can someone explain me why Lizard is 10 times faster than gluster?
This is not a flame, I would only like to know the technical
differences between these two software
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users

On 14/11/2016 9:00 PM, Gandalf Corvotempesta wrote:
> Can someone explain me why Lizard is 10 times faster than gluster?
> This is not a flame, I would only like to know the technical
> differences between these two software
Its my understanding that with many/small file operations involving 
directory lookup etc on disperse volumes gluster has to check each 
directory on each brick set, which leads to very high latencies. This is 
where a metadata server is an advantage I guess.


Of course if you're running a replica <n> volume, non-dispersed you 
should only need to do lookups locally. It would be interesting to know 
if thats a optimization gluster does.

-- 
Lindsay Mathieson