Jeff Darcy
2016-Mar-04 15:40 UTC
[Gluster-users] [Gluster-devel] Default quorum for 2 way replication
> I like the default to be 'none'. Reason: If we have 'auto' as quorum for > 2-way replication and first brick dies, there is no HA. If users are > fine with it, it is better to use plain distribute volume"Availability" is a tricky word. Does it mean access to data now, or later despite failure? Taking a volume down due to loss of quorum might be equivalent to having no replication in the first sense, but certainly not in the second. When the possibility (likelihood?) of split brain is considered, enforcing quorum actually does a *better* job of preserving availability in the second sense. I believe this second sense is most often what users care about, and therefore quorum enforcement should be the default. I think we all agree that quorum is a bit slippery when N=2. That's where there really is a tradeoff between (immediate) availability and (highest levels of) data integrity. That's why arbiters showed up first in the NSR specs, and later in AFR. We should definitely try to push people toward N>=3 as much as we can. However, the ability to "scale down" is one of the things that differentiate us vs. both our Ceph cousins and our true competitors. Many of our users will stop at N=2 no matter what we say. However unwise that might be, we must still do what we can to minimize harm when things go awry.
Diego Remolina
2016-Mar-04 16:35 UTC
[Gluster-users] [Gluster-devel] Default quorum for 2 way replication
I run a few two node glusterfs instances, but always have a third machine acting as an arbiter. I am with Jeff on this one, better safe than sorry. Setting up a 3rd system without bricks to achieve quorum is very easy. Diego On Fri, Mar 4, 2016 at 10:40 AM, Jeff Darcy <jdarcy at redhat.com> wrote:>> I like the default to be 'none'. Reason: If we have 'auto' as quorum for >> 2-way replication and first brick dies, there is no HA. If users are >> fine with it, it is better to use plain distribute volume > > "Availability" is a tricky word. Does it mean access to data now, or > later despite failure? Taking a volume down due to loss of quorum might > be equivalent to having no replication in the first sense, but certainly > not in the second. When the possibility (likelihood?) of split brain is > considered, enforcing quorum actually does a *better* job of preserving > availability in the second sense. I believe this second sense is most > often what users care about, and therefore quorum enforcement should be > the default. > > I think we all agree that quorum is a bit slippery when N=2. That's > where there really is a tradeoff between (immediate) availability and > (highest levels of) data integrity. That's why arbiters showed up first > in the NSR specs, and later in AFR. We should definitely try to push > people toward N>=3 as much as we can. However, the ability to "scale > down" is one of the things that differentiate us vs. both our Ceph > cousins and our true competitors. Many of our users will stop at N=2 no > matter what we say. However unwise that might be, we must still do what > we can to minimize harm when things go awry. > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users
Pranith Kumar Karampuri
2016-Mar-05 10:00 UTC
[Gluster-users] [Gluster-devel] Default quorum for 2 way replication
On 03/04/2016 09:10 PM, Jeff Darcy wrote:>> I like the default to be 'none'. Reason: If we have 'auto' as quorum for >> 2-way replication and first brick dies, there is no HA. If users are >> fine with it, it is better to use plain distribute volume > "Availability" is a tricky word. Does it mean access to data now, or > later despite failure? Taking a volume down due to loss of quorum might > be equivalent to having no replication in the first sense, but certainly > not in the second. When the possibility (likelihood?) of split brain is > considered, enforcing quorum actually does a *better* job of preserving > availability in the second sense. I believe this second sense is most > often what users care about, and therefore quorum enforcement should be > the default. > > I think we all agree that quorum is a bit slippery when N=2. That's > where there really is a tradeoff between (immediate) availability and > (highest levels of) data integrity. That's why arbiters showed up first > in the NSR specs, and later in AFR. We should definitely try to push > people toward N>=3 as much as we can. However, the ability to "scale > down" is one of the things that differentiate us vs. both our Ceph > cousins and our true competitors. Many of our users will stop at N=2 no > matter what we say. However unwise that might be, we must still do what > we can to minimize harm when things go awry.I always felt 2-way replication, 3-way replication analogy is similar to 2-wheeler(motor-bikes) and 4-wheeler vehicles(cars). You have more fatal accidents with 2-wheelers than 4-wheelers. But it has its place. Arbiter volumes is like a 3-wheeler(auto rickshaw) :-). I feel users should be given the power to choose what they want based on what they are looking for and how much hardware they want to buy (affordability). We should educate them about the risks but the final decision should be theirs. So in that sense I don't like to *push* them to N>=3. "Many of our users will stop at N=2 no matter what we say". That right there is what I had to realize, some years back. I naively thought that people will rush to replica-3 with client quorum, but it didn't happen. That is the reason for investing time in arbiter volumes as a solution. Because we wanted to reduce the cost. People didn't want to spend so much money for consistency(based on what we are still seeing). Fact of the matter is, even after arbiter volumes I am sure some people will stick with replica-2 with unsplit-brain patch from facebook (For people who don't know: it resolves split-brain based on policies automatically without human intervention, it will be available soon in gluster). You do have a very good point though. I think it makes sense to make more people aware of what they are getting into with 2-way replication. So may be an interactive question at the time of 2-way replica volume creation about the possibility of split-brains and availability of other options(like arbiter/unsplit-brain in 2-way replication) could be helpful, keeping the default still as 'none'. I think it would be better if we educate users about value of arbiter volumes, so that users naturally progress towards that and embrace it. We are seeing more and more questions on the IRC and mailing list about arbiter volumes, so there is a +ve trend. Pranith