Gluster users - Apr 2016 - Question about the number of nodes

Hi,

As stated in another thread, we currently have a 3 nodes cluster with sharding
enabled used for storing VM disks.
I am migrating that to a new 3.7.11 cluster to hopefully fix the problems with
had friday, but since those 3
nodes are nearly full we'd like to expand.

We have 3 nodes with a replica 3. What would be better, go to 5 nodes and use a
replica 2 (so "wasting" one node),
or go to 6 nodes with still a replica 3 ? Seems like having 3 replicas is better
for safety, but can someone confirm
that whats important for quorum is the number of bricks in a replica set, not
the number of nodes total ?
Would hate to get into a split brain because we upgraded to an even number of
node.

Thanks,

-- 
Kevin Lemonnier
PGP Fingerprint : 89A5 2283 04A0 E6E9 0111
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160419/4e604d7b/attachment.sig>

On Tue, Apr 19, 2016 at 5:25 PM, Kevin Lemonnier <lemonnierk at ulrar.net>
wrote:
> Hi,
>
> As stated in another thread, we currently have a 3 nodes cluster with
> sharding enabled used for storing VM disks.
> I am migrating that to a new 3.7.11 cluster to hopefully fix the problems
> with had friday, but since those 3
> nodes are nearly full we'd like to expand.
>
> We have 3 nodes with a replica 3. What would be better, go to 5 nodes and
> use a replica 2 (so "wasting" one node),
> or go to 6 nodes with still a replica 3 ? Seems like having 3 replicas is
> better for safety, but can someone confirm
> that whats important for quorum is the number of bricks in a replica set,
> not the number of nodes total ?
> Would hate to get into a split brain because we upgraded to an even number
> of node.
>
You need to go for 3 more nodes with still a replica 3 to guard against
split-brains.

-Krutika

>
> Thanks,
>
> --
> Kevin Lemonnier
> PGP Fingerprint : 89A5 2283 04A0 E6E9 0111
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160419/d244552f/attachment.html>

On 19/04/2016 9:55 PM, Kevin Lemonnier wrote:
> As stated in another thread, we currently have a 3 nodes cluster with
sharding enabled used for storing VM disks.
> I am migrating that to a new 3.7.11 cluster to hopefully fix the problems
with had friday, but since those 3
> nodes are nearly full we'd like to expand.
don't forget to update the opversion for the cluster:

     gluster volume set all cluster.op-version 30710

(its not 30711 as there are no new features from 3.7.10 to 3.7.11)

>
> We have 3 nodes with a replica 3. What would be better, go to 5 nodes and
use a replica 2 (so "wasting" one node),
> or go to 6 nodes with still a replica 3 ? Seems like having 3 replicas is
better for safety, but can someone confirm
> that whats important for quorum is the number of bricks in a replica set,
not the number of nodes total ?
> Would hate to get into a split brain because we upgraded to an even number
of node.
Definitely need 3 nodes with replica 3.

You can have a 5 proxmox node cluster with only three nodes being used 
for gluster.

What sort of network setup do you have? mine is relatively low end. 
2*1GB Eth on each node, LACP bonding.

-- 
Lindsay Mathieson

On 04/19/2016 07:55 AM, Kevin Lemonnier wrote:
> Hi,
> 
> As stated in another thread, we currently have a 3 nodes cluster with
sharding enabled used for storing VM disks.
> I am migrating that to a new 3.7.11 cluster to hopefully fix the problems
with had friday, but since those 3
> nodes are nearly full we'd like to expand.
> 
> We have 3 nodes with a replica 3. What would be better, go to 5 nodes and
use a replica 2 (so "wasting" one node),
> or go to 6 nodes with still a replica 3 ? Seems like having 3 replicas is
better for safety, but can someone confirm
> that whats important for quorum is the number of bricks in a replica set,
not the number of nodes total ?
> Would hate to get into a split brain because we upgraded to an even number
of node.
I believe you could set up a `2x2 replica 2` cluster and use the fifth
node as an arbiter node to prevent/minimize split brain.

--

Kaleb