Gluster users - Jul 2016 - New cluster - first experience

On 7/07/2016 8:30 PM, Gandalf Corvotempesta wrote:
> 1) Can I use the same cluster for both, maildir hosting and VM images
hosting?
Yes. However maildir involves many tens of thousands of small & large 
files, I *think* that glusters performances isn't the best with very 
large numbers of files in a dir, but hopefully someone else with more 
experience can chime in on that.
>
> 2) a simple kernel extract (with replica 3) is
> sloooooooooooooooooooow. Direct extract in a brick took 12 seconds.
> Extract through gluster (replica 3) took 10m2.214s. In a perfect
> world, it should be 3 times the direct extraction (due to the replica)
> plus a little bit overhead for the network. Can I optimize this ?
That does sound slow - how big was the tar file? what is your network 
speed and setup?
>
> 3) I'm thinking creating a volume for Maildir, with no sharding, and 1
> volume for VM hosting in the future (with sharding enabled). Is this
> OK?
Yes, sounds like a good idea.
>
> 4) I would like to get replica 3 for all, but also the ability to add
> 1 brick/server per time and not in multiple (in our case, 3 server
> every time), like Ceph does. Is this a supported configuration?
Not sure. Disperse Replicated vol maybe?
>
> 5) currently i'm testing gluster on a gigabit network (not bonded,
> it's just a test). When running in production, i'll move to 10GbE
but
> currently I would like to get the best performance from the gigabit
> network.
If you're not using a dual or better bonded connection on replica 3 then 
your write speeds will be limited to 1Gb/3 max.
>
> some hardware info: currently 2x E5-2603, 16GB RAM on each storage node.
Are your clients on the storage nodes or are they dedicated?


-- 
Lindsay Mathieson

2016-07-07 13:22 GMT+02:00 Lindsay Mathieson <lindsay.mathieson at
gmail.com>:
> Yes. However maildir involves many tens of thousands of small & large
files,
> I *think* that glusters performances isn't the best with very large
numbers
> of files in a dir, but hopefully someone else with more experience can
chime
> in on that.
Performance for maildir shoul not be much important, I think.
I can also create a VM with the mail server, in this case, on gluster
i'll put the VM image and not the plain maildir, but I prefere the
first solution.
> That does sound slow - how big was the tar file? what is your network speed
> and setup?
I've used the 4.7-rc6 from here: https://www.kernel.org/
Gigabit network

# gluster volume info

Volume Name: gv0
Type: Replicate
Volume ID: 2a36dc0f-1d9b-469c-82de-9d8d98321b83
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 77.95.175.112:/export/sdb1/brick
Brick2: 77.95.175.113:/export/sdb1/brick
Brick3: 77.95.175.114:/export/sdb1/brick
Options Reconfigured:
features.shard: on
transport.address-family: inet
performance.readdir-ahead: on
nfs.disable: on
> Not sure. Disperse Replicated vol maybe?
Is disperse based on erasure code ?
I've read that erasure code store the encoded file, I don't like
to store encoded file, in case of issue, the encoding could lead
to a mess.
> If you're not using a dual or better bonded connection on replica 3
then
> your write speeds will be limited to 1Gb/3 max.
Ok.
> Are your clients on the storage nodes or are they dedicated?
I'm using a dedicated client.