----- Original Message -----> From: "Gandalf Corvotempesta" <gandalf.corvotempesta at gmail.com> > To: "Pranith Kumar Karampuri" <pkarampu at redhat.com> > Cc: "Anuradha Talur" <atalur at redhat.com>, "gluster-users" <Gluster-users at gluster.org> > Sent: Tuesday, July 12, 2016 4:32:57 PM > Subject: Re: [Gluster-users] Expand distributed replicated volume > > 2016-07-12 12:56 GMT+02:00 Pranith Kumar Karampuri <pkarampu at redhat.com>: > > For adding new bricks into a replica set you need each brick in the replica > > set to be from different machine. So you can't add all bricks directly from > > just one machine. So how do you get the extra bricks that can be combined > > with the bricks on new machine? From old servers. How do you get those? > > Using replace-brick. I hope with this info if you can go through the > > example > > once again, it probably may clarify your doubts. Please feel free to ask > > any > > questions you may have. > > I'm new to gluster, so let me try to explain and sorry for my dump question. > > If I have a replica made with: > > (s1b1, s2b1, s3b1) > (s1b2, s2b2, s3b2) > (s1b3, s2b3, s3b3) > > Why I cant add a new server getting this result: > > (s1b1, s2b1, s3b1, s4b1) > (s1b2, s2b2, s3b2, s4b2) > (s1b3, s2b3, s3b3, s4b3) > > ?? In this case , I've added a new server with 3 bricks. >Some clarification here : In Pranith's example, each replica set is represented as comma separated values. Meaning, (s1b1, s2b1, s3b1) will all contain the same data replicated thrice. If you add new server with 3 bricks like you've represented : (s1b1, s2b1, s3b1, s4b1) as per Pranith's representation it means there are 4 bricks which are replica of each other. Meaning they all have the same data, so there is 4 fold replication here.>From my understanding, you say that your job should only be to add a node with 3bricks and gluster should internally take care of distributing the data. This is a perfectly valid requirement, but right now, gluster makes its replica set based on the order of bricks specified in the cli. When you perform add-brick, and mention 3 new bricks, they will become replica of each other automatically. Which is why steps mentioned by Pranith will help in solving your issue. I'm repeating the same example that Pranith gave. Skip the next section if you have already understood. What Pranith is trying to say is, at the end you would have technically added 3 bricks to your cluster (all residing in the same node) but, the distribution will not happen automatically. It will be your responsibility (until heketi is usable to do the same) to make sure that the newly added bricks are arranged such that you don't lose redundancy in case of node failures. Taking your example here: Step 1) You have : (s1b1, s2b1, s3b1) (s1b2, s2b2, s3b2) (s1b3, s2b3, s3b3) Step 2) You want to add 3 more disks/bricks to increase volume size. Condition is that you want to add only one new node. As you don't want to lose redundancy in case of node failure, you have to ensure that no 2 bricks which contain the same data reside in the same node. As this arrangement is not taken care by gluster yet, using the same resources, there is only one way to do it: Empty one brick from each existing node so that we have gained distribution of our bricks to all nodes. And now substitute each of these bricks with the 3 new bricks from the latest node. This is done using replace-brick option. At this point, your cluster will look like : (s4b1, s2b1, s3b1) (s1b2, s4b2, s3b2) (s1b3, s2b3, s4b3) So instead of s4b1, s4b2 and s4b3 being free, you have s1b1, s2b2 and s3b3 free. Step 3) You can now add the three free bricks to form a new 3 way replica set. Do this, using add-brick command. This way, you have essentially added only one node containing 3 bricks, and also achieved redundancy. I realize I just repeated the same example in different words but I hope this answers your questions. So do let us know if something is not clear. -- Thanks, Anuradha.
Gandalf Corvotempesta
2016-Jul-12 14:54 UTC
[Gluster-users] Expand distributed replicated volume
2016-07-12 14:06 GMT+02:00 Anuradha Talur <atalur at redhat.com>:> I realize I just repeated the same example in different words but I hope this answers > your questions. So do let us know if something is not clear.Is unclear, probably because i don't understand how gluster works and because I don't understand why was designed like this. Having to replace bricks just to add a single server and not multiple servers due to the replica count, is absolutely unclear. Any other scale-out storage allows this natively.