Taylor Lewick
2015-Mar-19 16:14 UTC
[Gluster-users] Distributed replicated Volume questions
I'm new to gluster administration, and am looking at a current gluster cluster... We have a 6 node gluster cluster and a volume called cdparchive2 that is set to distributed replicated. My assumption was that it is set to distribute across the first 3 bricks, and that group of 3 bricks would replicate to the next three bricks. Here is the output from gluster volume info cdparchive2 gluster volume info cdparchive2 Volume Name: cdparchive2 Type: Distributed-Replicate Volume ID: 1e3fdd58-14fc-459b-817f-6c6612090ecd Status: Started Number of Bricks: 3 x 2 = 6 Transport-type: tcp Bricks: Brick1: gluster07:/data/cdparchive2 Brick2: gluster08:/data/cdparchive2 Brick3: gluster09:/data/cdparchive2 Brick4: gluster10:/data/cdparchive2 Brick5: gluster11:/data/cdparchive2 Brick6: gluster12:/data/cdparchive2 So based on my assumptions/understanding, I thought it would distribute across nodes 7 8 and 9, and then replicate to 10, 11, 12. But when I login to each gluster data node and du a du -sm on /data/cdparchive2 I see the following: Nodes 7 and 8 are the same size, nodes 9 and 10 are the same size, and nodes 11 and 12 are the same size. So I guess that means its distributing to 7, 9 and 11, and replicating to 8, 10, and 12? Going forward, how do I create a new distributed replicated volume that acts the way I assume, that is distributes across first 3 nodes, and then replicates to the next 3 nodes? Would that be gluster volume create test-volume replica 2 transport tcp gluster071:/test1 gluster08:/test2 gluster09:/test3 gluster10:/test4 gluster11:/test5 gluster12:/test6? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150319/89fa8e33/attachment.html>
JF Le Fillâtre
2015-Mar-19 17:07 UTC
[Gluster-users] Distributed replicated Volume questions
Hello, I would advise that you take some time to read the following document carefully: https://github.com/gluster/glusterfs/blob/master/doc/admin-guide/en-US/markdown/admin_setting_volumes.md Distribution is always on top of replication. You have the option "replica 2", so gluster writes the same file to two bricks (the order in which they're specified on the command line matters). Then, because you have 6 bricks, it has three 2-brick replicas over which it distributes the files. Basically, the behaviour you are seeing is perfectly normal. I guess that you may be confusing distribution with striping. Please read the above document to see if one of the creation options matches what you are looking for. Thanks, JF On 19/03/15 17:14, Taylor Lewick wrote:> I?m new to gluster administration, and am looking at a current gluster > cluster? > > > > We have a 6 node gluster cluster and a volume called cdparchive2 that is > set to distributed replicated. > > My assumption was that it is set to distribute across the first 3 > bricks, and that group of 3 bricks would replicate to the next three bricks. > > > > Here is the output from gluster volume info cdparchive2 > > > > gluster volume info cdparchive2 > > Volume Name: cdparchive2 > > Type: Distributed-Replicate > > Volume ID: 1e3fdd58-14fc-459b-817f-6c6612090ecd > > Status: Started > > Number of Bricks: 3 x 2 = 6 > > Transport-type: tcp > > Bricks: > > Brick1: gluster07:/data/cdparchive2 > > Brick2: gluster08:/data/cdparchive2 > > Brick3: gluster09:/data/cdparchive2 > > Brick4: gluster10:/data/cdparchive2 > > Brick5: gluster11:/data/cdparchive2 > > Brick6: gluster12:/data/cdparchive2 > > > > So based on my assumptions/understanding, I thought it would distribute > across nodes 7 8 and 9, and then replicate to 10, 11, 12. > > > > But when I login to each gluster data node and du a du ?sm on > /data/cdparchive2 I see the following: > > Nodes 7 and 8 are the same size, nodes 9 and 10 are the same size, and > nodes 11 and 12 are the same size. > > > > So I guess that means its distributing to 7, 9 and 11, and replicating > to 8, 10, and 12? > > > > Going forward, how do I create a new distributed replicated volume that > acts the way I assume, that is distributes across first 3 nodes, and > then replicates to the next 3 nodes? > > > > Would that be gluster volume create test-volume replica 2 transport tcp > gluster071:/test1 gluster08:/test2 gluster09:/test3 gluster10:/test4 > gluster11:/test5 gluster12:/test6? > > > > > > > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-- Jean-Fran?ois Le Fill?tre ------------------------------- HPC Systems Administrator LCSB - University of Luxembourg ------------------------------- PGP KeyID 0x134657C6