DUCARROZ Birgit
2019-Dec-27 12:00 UTC
[Gluster-users] remove arbiter to add new brick - possible?
Hi Xavi, Thank you a lot for your response. That's the information I needed. However, I run actually into an other error message: root at nas22:# gluster volume add-brick vol-cdimages nas28:/bigdisk/brick7 nas29:/bigdisk/brick8 nas23:/bigdisk/brick9 volume add-brick: failed: Pre Validation failed on nas28. Please check log file for details. Pre Validation failed on nas29. Please check log file for details. The logfile says: [2019-12-27 11:37:41.439048] E [MSGID: 106121] [glusterd-mgmt.c:1148:glusterd_mgmt_v3_pre_validate] 0-management: Pre Validation failed on peers All peers are in Cluster and Connected. How can I solve this problem? Kind regards, Birgit On 19/12/19 08:43, Xavi Hernandez wrote:> Hi Brigit, > > On Wed, Dec 18, 2019 at 3:16 PM DUCARROZ Birgit > <birgit.ducarroz at unifr.ch <mailto:birgit.ducarroz at unifr.ch>> wrote: > > Hi list, > > I have a replica 3 volume with arbiter. > Now I would like to add a new brick. > How can I do this? > > > When you expand a volume you need to add as many bricks as necessary to > create a new replica set. In your case you need to add 3 more bricks (2 > data bricks and 1 arbiter) to expand the volume. > > You can check the documentation [1] for more information. > > Xavi > > [1] > https://docs.gluster.org/en/latest/Administrator%20Guide/Managing%20Volumes/#expanding-volumes > > > > gluster volume add-brick ... gives me output: > volume add-brick: failed: Increasing replica count for arbiter volumes > is not supported. > > Thank you for any help. > Regards, > Birgit > ________ > > Community Meeting Calendar: > > APAC Schedule - > Every 2nd and 4th Tuesday at 11:30 AM IST > Bridge: https://bluejeans.com/441850968 > > NA/EMEA Schedule - > Every 1st and 3rd Tuesday at 01:00 PM EDT > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> > https://lists.gluster.org/mailman/listinfo/gluster-users >
DUCARROZ Birgit
2019-Dec-27 13:46 UTC
[Gluster-users] remove arbiter to add new brick - possible?
Hi, me again. I think I managed it, anyway, it works, I hope this is the right way to do it and not a hack. I tried the version using.. setfattr -x trusted.glusterfs.volume-id /bigdisk/brick7/vol-cdimages setfattr -x trusted.gfid /bigdisk/brick7/vol-cdimages but this did not work (error message). Copied from a working server the vol informations to the new servers ... scp -r /var/lib/glusterd/vols/* sysadmin at nas29:/var/lib/glusterd/vols/ changed the rights on the new server: chown -R root.root /var/lib/glusterd/vols/* Then, on the new server: gluster volume add-brick vol-cdimages nas28:/bigdisk/brick7/vol-cdimages nas29:/bigdisk/brick8/vol-cdimages nas22:/bigdisk/brick9/vol-cdimages force --> success. #gluster volume info vol-cdimages Volume Name: vol-cdimages Type: Distributed-Replicate Volume ID: xxxxxxxxx Status: Started Snapshot Count: 0 Number of Bricks: 2 x (2 + 1) = 6 Transport-type: tcp Bricks: Brick1: nas20:/bigdisk/brick1/vol-cdimages Brick2: nas21:/bigdisk/brick2/vol-cdimages Brick3: nas22:/bigdisk/brick3/vol-cdimages (arbiter) Brick4: nas28:/bigdisk/brick7/vol-cdimages Brick5: nas29:/bigdisk/brick8/vol-cdimages Brick6: nas22:/bigdisk/brick9/vol-cdimages (arbiter) Hoping this info might be useful for other users. Regards, Birgit On 27/12/19 13:00, DUCARROZ Birgit wrote:> Hi Xavi, > > Thank you a lot for your response. That's the information I needed. > > However, I run actually into an other error message: > > > root at nas22:# gluster volume add-brick vol-cdimages nas28:/bigdisk/brick7 > nas29:/bigdisk/brick8 nas23:/bigdisk/brick9 > > volume add-brick: failed: Pre Validation failed on nas28. Please check > log file for details. > Pre Validation failed on nas29. Please check log file for details. > > > The logfile says: > > [2019-12-27 11:37:41.439048] E [MSGID: 106121] > [glusterd-mgmt.c:1148:glusterd_mgmt_v3_pre_validate] 0-management: Pre > Validation failed on peers > > All peers are in Cluster and Connected. > > How can I solve this problem? > Kind regards, > Birgit > > On 19/12/19 08:43, Xavi Hernandez wrote: >> Hi Brigit, >> >> On Wed, Dec 18, 2019 at 3:16 PM DUCARROZ Birgit >> <birgit.ducarroz at unifr.ch <mailto:birgit.ducarroz at unifr.ch>> wrote: >> >> ??? Hi list, >> >> ??? I have a replica 3 volume with arbiter. >> ??? Now I would like to add a new brick. >> ??? How can I do this? >> >> >> When you expand a volume you need to add as many bricks as necessary >> to create a new replica set. In your case you need to add 3 more >> bricks (2 data bricks and 1 arbiter) to expand the volume. >> >> You can check the documentation [1] for more information. >> >> Xavi >> >> [1] >> https://docs.gluster.org/en/latest/Administrator%20Guide/Managing%20Volumes/#expanding-volumes >> >> >> >> >> ??? gluster volume add-brick ... gives me output: >> ??? volume add-brick: failed: Increasing replica count for arbiter >> volumes >> ??? is not supported. >> >> ??? Thank you for any help. >> ??? Regards, >> ??? Birgit >> ??? ________ >> >> ??? Community Meeting Calendar: >> >> ??? APAC Schedule - >> ??? Every 2nd and 4th Tuesday at 11:30 AM IST >> ??? Bridge: https://bluejeans.com/441850968 >> >> ??? NA/EMEA Schedule - >> ??? Every 1st and 3rd Tuesday at 01:00 PM EDT >> ??? Bridge: https://bluejeans.com/441850968 >> >> ??? Gluster-users mailing list >> ??? Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> >> ??? https://lists.gluster.org/mailman/listinfo/gluster-users >> > ________ > > Community Meeting Calendar: > > APAC Schedule - > Every 2nd and 4th Tuesday at 11:30 AM IST > Bridge: https://bluejeans.com/441850968 > > NA/EMEA Schedule - > Every 1st and 3rd Tuesday at 01:00 PM EDT > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users at gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users
Xavi Hernandez
2020-Jan-08 07:58 UTC
[Gluster-users] remove arbiter to add new brick - possible?
Hi Brigit, sorry for the late reply. I've been on holidays and I haven't checked the email for a while. On Fri, Dec 27, 2019 at 2:46 PM DUCARROZ Birgit <birgit.ducarroz at unifr.ch> wrote:> Hi, me again. > > I think I managed it, anyway, it works, I hope this is the right way to > do it and not a hack. I tried the version using.. > > setfattr -x trusted.glusterfs.volume-id /bigdisk/brick7/vol-cdimages > setfattr -x trusted.gfid /bigdisk/brick7/vol-cdimages > > but this did not work (error message). >If the directory was completely empty and no xattrs were present on it, it should have worked. Probably the error was not on the directory itself, but on gluster configuration based on the way you solved it.> > > Copied from a working server the vol informations to the new servers ... > scp -r /var/lib/glusterd/vols/* sysadmin at nas29:/var/lib/glusterd/vols/ > > changed the rights on the new server: > chown -R root.root /var/lib/glusterd/vols/* > > Then, on the new server: > gluster volume add-brick vol-cdimages nas28:/bigdisk/brick7/vol-cdimages > nas29:/bigdisk/brick8/vol-cdimages nas22:/bigdisk/brick9/vol-cdimages force > > --> success. >Not sure what this fixed, but it was the problem. The possible issue with this change is that sometimes the volume information is not exactly the same on all servers. Now that it seems to work, one way to force a rewrite of the configuration of a volume is to change any option using 'gluster volume set <volname> <option> <value>'. This will regenerate volume configuration with the new option and, in the process, any specific difference between servers will be reapplied. Regards, Xavi> > #gluster volume info vol-cdimages > > Volume Name: vol-cdimages > Type: Distributed-Replicate > Volume ID: xxxxxxxxx > Status: Started > Snapshot Count: 0 > Number of Bricks: 2 x (2 + 1) = 6 > Transport-type: tcp > Bricks: > Brick1: nas20:/bigdisk/brick1/vol-cdimages > Brick2: nas21:/bigdisk/brick2/vol-cdimages > Brick3: nas22:/bigdisk/brick3/vol-cdimages (arbiter) > Brick4: nas28:/bigdisk/brick7/vol-cdimages > Brick5: nas29:/bigdisk/brick8/vol-cdimages > Brick6: nas22:/bigdisk/brick9/vol-cdimages (arbiter) > > Hoping this info might be useful for other users. > > Regards, > Birgit > > > > > On 27/12/19 13:00, DUCARROZ Birgit wrote: > > Hi Xavi, > > > > Thank you a lot for your response. That's the information I needed. > > > > However, I run actually into an other error message: > > > > > > root at nas22:# gluster volume add-brick vol-cdimages > nas28:/bigdisk/brick7 > > nas29:/bigdisk/brick8 nas23:/bigdisk/brick9 > > > > volume add-brick: failed: Pre Validation failed on nas28. Please check > > log file for details. > > Pre Validation failed on nas29. Please check log file for details. > > > > > > The logfile says: > > > > [2019-12-27 11:37:41.439048] E [MSGID: 106121] > > [glusterd-mgmt.c:1148:glusterd_mgmt_v3_pre_validate] 0-management: Pre > > Validation failed on peers > > > > All peers are in Cluster and Connected. > > > > How can I solve this problem? > > Kind regards, > > Birgit > > > > On 19/12/19 08:43, Xavi Hernandez wrote: > >> Hi Brigit, > >> > >> On Wed, Dec 18, 2019 at 3:16 PM DUCARROZ Birgit > >> <birgit.ducarroz at unifr.ch <mailto:birgit.ducarroz at unifr.ch>> wrote: > >> > >> Hi list, > >> > >> I have a replica 3 volume with arbiter. > >> Now I would like to add a new brick. > >> How can I do this? > >> > >> > >> When you expand a volume you need to add as many bricks as necessary > >> to create a new replica set. In your case you need to add 3 more > >> bricks (2 data bricks and 1 arbiter) to expand the volume. > >> > >> You can check the documentation [1] for more information. > >> > >> Xavi > >> > >> [1] > >> > https://docs.gluster.org/en/latest/Administrator%20Guide/Managing%20Volumes/#expanding-volumes > >> > >> > >> > >> > >> gluster volume add-brick ... gives me output: > >> volume add-brick: failed: Increasing replica count for arbiter > >> volumes > >> is not supported. > >> > >> Thank you for any help. > >> Regards, > >> Birgit > >> ________ > >> > >> Community Meeting Calendar: > >> > >> APAC Schedule - > >> Every 2nd and 4th Tuesday at 11:30 AM IST > >> Bridge: https://bluejeans.com/441850968 > >> > >> NA/EMEA Schedule - > >> Every 1st and 3rd Tuesday at 01:00 PM EDT > >> Bridge: https://bluejeans.com/441850968 > >> > >> Gluster-users mailing list > >> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org> > >> https://lists.gluster.org/mailman/listinfo/gluster-users > >> > > ________ > > > > Community Meeting Calendar: > > > > APAC Schedule - > > Every 2nd and 4th Tuesday at 11:30 AM IST > > Bridge: https://bluejeans.com/441850968 > > > > NA/EMEA Schedule - > > Every 1st and 3rd Tuesday at 01:00 PM EDT > > Bridge: https://bluejeans.com/441850968 > > > > Gluster-users mailing list > > Gluster-users at gluster.org > > https://lists.gluster.org/mailman/listinfo/gluster-users > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200108/09e756af/attachment.html>