Atin Mukherjee
2016-Mar-21 08:34 UTC
[Gluster-users] How to recover after one node breakdown
On 03/21/2016 01:30 PM, songxin wrote:> Hi? > Thank you for your reply. > Could you help me to answer my questions as below. > > Now C node has replace the B node in replicate volume. > And then I want to use B node to replace C node. > > 13.shutdown C node > 14.may continue create files on mount point > 15.power up B node (the old files in /var/lib/gluster/ are still there) > 16.B disk is mount on B node(the disk are non-volatile) > > at this point my question: > 1.Must I remove all files in /var/lib/gluster/ and create a glusterd.info in dirctory /var/lib/glusterd/ based on peers of A node on B node? > 2.how to sync the two bricks?Can heal full work? > 3."heal full" must be run on the node whose uuid is biggest in volume?The answer is same, if you want to now replace C with B, you'd need to cleanup /var/lib/glusterd on B, and follow the same steps of procedure of peer replacement. I was just wondering what use case does have this requirement of frequent peer replacement procedures. Can you elaborate?> > Thanks, > Xin > > > > > At 2016-03-21 14:04:25, "Atin Mukherjee" <amukherj at redhat.com> wrote: >> >> >>On 03/19/2016 06:50 AM, songxin wrote: >>> Hi Gaurav Garg, >>> >>> Thank you for you reply.It is very helpful for me. >>> And I have a little more questions as below. >>> >>> precondition: >>> 1.I have three node, A node, B node and C node. >>> 2.A node is always runing. >>> 3.B node and C node has same ip,so just only one can be running. >>> 4.there are two external hard disks, A disk and B disk >>> 5.At first the rootfs are emtpy on all nodes >>> >>> >>> reprodue steps: >>> 1.A disk is mounted on A node as A brick >>> 2.B disk is mounted on B node as B brick >>> 3.start glusterd on A node >>> 4.start glusterd on B node >>> 5.create a replicate volume use A brich and B brick >>> 6.start volume >>> 7.a directory is mounted on volume >>> 8.create some files on mount point >>> 9.shutdown B node >>> 10.may continue create files on mount point >>> 11.power up C node (rootfs is empty on C node) >>> 12.Bdisk is mount on C node (the disk are non-volatile ) >>> >>> at this point my question: >>> 1.Must I create a glusterd.info in dirctory /var/lib/glusterd/ based on >>> peers of A node on C node, if I want to use the C node replace the B node? >>> I found that the glusterd will create a new glusterd.info after I start >>> glusterd on C node if I can't create a glusterd.info.Is that ok? >>This is where all your problem starts as per the multiple mails I have >>seen from you and now it makes sense why you/Abhishek see multiple UUIDs >>for the same host. >> >>You'd need to copy glusterd.info file from Node B and place it on Node C >>to indicate Node C is just a replacement server and for that You'd need >>to follow the steps which Gaurav had mentioned in the mail thread. >> >>Post that if gluster cluster is operational (peers are in sync) I don't >>see any reason why heal full won't work. >>> 2.how to sync the two bricks?Can heal full work? >>> >>> Now C node has replace the B node in replicate volume. >>> And then I want to use B node to replace C node. >>> >>> 13.shutdown C node >>> 14.may continue create files on mount point >>> 15.power up B node (the old files in /var/lib/gluster/ are still there) >>> 12.Bdisk is mount on B node(the disk are non-volatile) >>> >>> at this point my question: >>> 1.Must I remove all files in /var/lib/gluster/ and create a >>> glusterd.info in dirctory /var/lib/glusterd/ based on peers of A node on >>> B node? >>> 2.how to sync the two bricks?Can heal full work? >>> 3."heal full" must be run on the node whose uuid is biggest in volume? >>> >>> >>> >>> Thanks, >>> Xin >>> >>> >>> >>> >>> At 2016-03-19 02:25:58, "Gaurav Garg" <ggarg at redhat.com> wrote: >>>>Hi songxin, >>>> >>>>both method are almost same for recovering the replicated volume. i forgot to mentioned one steps: >>>> >>>> #gluster volume heal $vol full >>>> >>>>IMO this solution should also apply for gluster 3.4 >>>> >>>>Thanks, >>>> >>>>Regards, >>>>Gaurav >>>> >>>>----- Original Message ----- >>>>From: "songxin" <songxin_1980 at 126.com> >>>>To: "Gaurav Garg" <ggarg at redhat.com> >>>>Cc: "gluster-user" <gluster-users at gluster.org>, "Alastair Neil" <ajneil.tech at gmail.com> >>>>Sent: Friday, March 18, 2016 7:26:50 AM >>>>Subject: Re:Re: [Gluster-users] How to recover after one node breakdown >>>> >>>>Hi, >>>> >>>>Thank you for your reply. >>>>And I found a link that is about how to use a server to replace a old one as below. >>>> >>>> >>>>http://www.gluster.org/community/documentation/index.php/Gluster_3.4:_Brick_Restoration_-_Replace_Crashed_Server >>>> >>>> >>>>My question is following. >>>>1.I found that your recovery step is a little different with the link about sync the volume.I want to know if both methods are right for recovering the replicate volume? >>>>2.Is this link is just fit for glusterfs 3.4?My glusterfs version is 3.7.6.Is the the link fit the glusterfs 3.7.6? >>>> >>>> >>>>Thanks, >>>>Xin >>>> >>>> >>>> >>>> >>>> >>>> >>>>At 2016-03-17 14:51:34, "Gaurav Garg" <ggarg at redhat.com> wrote: >>>>>>> Could I run some glusterfs command on good node to recover the replicate volume? if I don't copy the files ,including glusterd.info and other files,from good node to new node. >>>>> >>>>> >>>>>running glusterfs command is not enough to recover the replicate volume. for recovery you need to follow following steps. >>>>> >>>>>1) remove /var/lib/glusterd/* data from new node (if in any case it present) then start glusterd on new node >>>>>1) kill glusterd on new node >>>>>2) from 1st node (which is in good condition) execute #gluster peer status command and copy the uuid from the peer status (you will see one failed node entry with hostname and uuid) and replace this UUID in node file /var/lib/glusterd/glusterd.info >>>>>from the 1st node you can also get the uuid of failed node by doing #cat /var/lib/glusterd/peer/* it will show uuid of failed node along with hostname/ip-address of failed node. >>>>>3) copy /var/lib/glusterd/peers/* to new node. >>>>>4) you need to rename one of the /var/lib/glusterd/peers/* file (you can find that file in new node by just matching the uuid of new node (/var/lib/glusterd/glusterd.info) with /var/lib/glusterd/peers/* file name) with the uuid of 1st node (/var/lib/glusterd/glusterd.info) and modify the content of the same file with having uuid of 1st node and hostname of 1st node. >>>>>5) now start glusterd on new node. >>>>>6) your volume will recover. >>>>>> >>>>>above steps are mandatory steps to recover failed node. >>>>> >>>>>Thanks, >>>>> >>>>>Regards, >>>>>Gaurav >>>>> >>>>>----- Original Message ----- >>>>>From: "songxin" <songxin_1980 at 126.com> >>>>>To: "Alastair Neil" <ajneil.tech at gmail.com> >>>>>Cc: gluster-users at gluster.org >>>>>Sent: Thursday, March 17, 2016 8:56:58 AM >>>>>Subject: Re: [Gluster-users] How to recover after one node breakdown >>>>> >>>>>Thank you very much for your reply. >>>>> >>>>>In fact it is that I use a new node ,of which rootfs is new , to replace the failed node. >>>>>And the new node has same IP address with the failed one. >>>>> >>>>>The brick is on a external hard disk.Because the hard disk is mounted on the node ?so the data on the brick of failed node will not be loss but may be async with the brick of good node.And the brick of failed node will be mounted on the new node. >>>>> >>>>>Now my recovery steps is run some glusterfs command on good node as below, after starting the glusterd on new node. >>>>>1.remove brick of new node from volume(the volume type is changed from replicate to distribute) >>>>>2.peer detach the new node ip(the new node ip is same as failed node) >>>>>3.peer probe the new node ip >>>>>3.add brick of new node to volume(the volume type is change to replicate) >>>>> >>>>>But many problem,like data async or peer state is error etc, will happen. >>>>> >>>>>My question is below. >>>>> >>>>>Could I run some glusterfs command on good node to recover the replicate volume? if I don't copy the files ,including glusterd.info and other files,from good node to new node. >>>>> >>>>>Thanks >>>>>Xin >>>>> >>>>> >>>>> >>>>> >>>>>???? iPhone >>>>> >>>>>? 2016?3?17??04:54?Alastair Neil < ajneil.tech at gmail.com > ??? >>>>> >>>>> >>>>> >>>>> >>>>>hopefully you have a back up of /var/lib/glusterd/ glusterd.info and /var/lib/glusterd/peers, if so I think you can copy them back to and restart glusterd and the volume info should get populated from the other node. If not you can probably reconstruct these from these files on the other node. >>>>> >>>>>i.e: >>>>>On the unaffected node the peers directory should have an entry for the failed node containing the uuid of the failed node. The glusterd.info file should enable you to recreate the peer file on the failed node. >>>>> >>>>> >>>>>On 16 March 2016 at 09:25, songxin < songxin_1980 at 126.com > wrote: >>>>> >>>>> >>>>> >>>>>Hi, >>>>>Now I face a problem. >>>>>Reproduc step is as below. >>>>>1.I create a replicate volume using two brick on two board >>>>>2.start the volume >>>>>3.one board is breakdown and all >>>>>files in the rootfs ,including /var/lib/glusterd/*,are lost. >>>>>4.reboot the board and ip is not change. >>>>> >>>>>My question: >>>>>How to recovery the replicate volume? >>>>> >>>>>Thanks, >>>>>Xin >>>>> >>>>> >>>>> >>>>> >>>>>_______________________________________________ >>>>>Gluster-users mailing list >>>>>Gluster-users at gluster.org >>>>>http://www.gluster.org/mailman/listinfo/gluster-users >>>>> >>>>> >>>>>_______________________________________________ >>>>>Gluster-users mailing list >>>>>Gluster-users at gluster.org >>>>>http://www.gluster.org/mailman/listinfo/gluster-users >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Gluster-users mailing list >>> Gluster-users at gluster.org >>> http://www.gluster.org/mailman/listinfo/gluster-users >>> > > > > >
Hi, I see.Thank you for your reply. Thanks, Xin ? 2016-03-21 16:34:26?"Atin Mukherjee" <amukherj at redhat.com> ???> > >On 03/21/2016 01:30 PM, songxin wrote: >> Hi? >> Thank you for your reply. >> Could you help me to answer my questions as below. >> >> Now C node has replace the B node in replicate volume. >> And then I want to use B node to replace C node. >> >> 13.shutdown C node >> 14.may continue create files on mount point >> 15.power up B node (the old files in /var/lib/gluster/ are still there) >> 16.B disk is mount on B node(the disk are non-volatile) >> >> at this point my question: >> 1.Must I remove all files in /var/lib/gluster/ and create a glusterd.info in dirctory /var/lib/glusterd/ based on peers of A node on B node? >> 2.how to sync the two bricks?Can heal full work? >> 3."heal full" must be run on the node whose uuid is biggest in volume? >The answer is same, if you want to now replace C with B, you'd need to >cleanup /var/lib/glusterd on B, and follow the same steps of procedure >of peer replacement. > >I was just wondering what use case does have this requirement of >frequent peer replacement procedures. Can you elaborate? >> >> Thanks, >> Xin >> >> >> >> >> At 2016-03-21 14:04:25, "Atin Mukherjee" <amukherj at redhat.com> wrote: >>> >>> >>>On 03/19/2016 06:50 AM, songxin wrote: >>>> Hi Gaurav Garg, >>>> >>>> Thank you for you reply.It is very helpful for me. >>>> And I have a little more questions as below. >>>> >>>> precondition: >>>> 1.I have three node, A node, B node and C node. >>>> 2.A node is always runing. >>>> 3.B node and C node has same ip,so just only one can be running. >>>> 4.there are two external hard disks, A disk and B disk >>>> 5.At first the rootfs are emtpy on all nodes >>>> >>>> >>>> reprodue steps: >>>> 1.A disk is mounted on A node as A brick >>>> 2.B disk is mounted on B node as B brick >>>> 3.start glusterd on A node >>>> 4.start glusterd on B node >>>> 5.create a replicate volume use A brich and B brick >>>> 6.start volume >>>> 7.a directory is mounted on volume >>>> 8.create some files on mount point >>>> 9.shutdown B node >>>> 10.may continue create files on mount point >>>> 11.power up C node (rootfs is empty on C node) >>>> 12.Bdisk is mount on C node (the disk are non-volatile ) >>>> >>>> at this point my question: >>>> 1.Must I create a glusterd.info in dirctory /var/lib/glusterd/ based on >>>> peers of A node on C node, if I want to use the C node replace the B node? >>>> I found that the glusterd will create a new glusterd.info after I start >>>> glusterd on C node if I can't create a glusterd.info.Is that ok? >>>This is where all your problem starts as per the multiple mails I have >>>seen from you and now it makes sense why you/Abhishek see multiple UUIDs >>>for the same host. >>> >>>You'd need to copy glusterd.info file from Node B and place it on Node C >>>to indicate Node C is just a replacement server and for that You'd need >>>to follow the steps which Gaurav had mentioned in the mail thread. >>> >>>Post that if gluster cluster is operational (peers are in sync) I don't >>>see any reason why heal full won't work. >>>> 2.how to sync the two bricks?Can heal full work? >>>> >>>> Now C node has replace the B node in replicate volume. >>>> And then I want to use B node to replace C node. >>>> >>>> 13.shutdown C node >>>> 14.may continue create files on mount point >>>> 15.power up B node (the old files in /var/lib/gluster/ are still there) >>>> 12.Bdisk is mount on B node(the disk are non-volatile) >>>> >>>> at this point my question: >>>> 1.Must I remove all files in /var/lib/gluster/ and create a >>>> glusterd.info in dirctory /var/lib/glusterd/ based on peers of A node on >>>> B node? >>>> 2.how to sync the two bricks?Can heal full work? >>>> 3."heal full" must be run on the node whose uuid is biggest in volume? >>>> >>>> >>>> >>>> Thanks, >>>> Xin >>>> >>>> >>>> >>>> >>>> At 2016-03-19 02:25:58, "Gaurav Garg" <ggarg at redhat.com> wrote: >>>>>Hi songxin, >>>>> >>>>>both method are almost same for recovering the replicated volume. i forgot to mentioned one steps: >>>>> >>>>> #gluster volume heal $vol full >>>>> >>>>>IMO this solution should also apply for gluster 3.4 >>>>> >>>>>Thanks, >>>>> >>>>>Regards, >>>>>Gaurav >>>>> >>>>>----- Original Message ----- >>>>>From: "songxin" <songxin_1980 at 126.com> >>>>>To: "Gaurav Garg" <ggarg at redhat.com> >>>>>Cc: "gluster-user" <gluster-users at gluster.org>, "Alastair Neil" <ajneil.tech at gmail.com> >>>>>Sent: Friday, March 18, 2016 7:26:50 AM >>>>>Subject: Re:Re: [Gluster-users] How to recover after one node breakdown >>>>> >>>>>Hi, >>>>> >>>>>Thank you for your reply. >>>>>And I found a link that is about how to use a server to replace a old one as below. >>>>> >>>>> >>>>>http://www.gluster.org/community/documentation/index.php/Gluster_3.4:_Brick_Restoration_-_Replace_Crashed_Server >>>>> >>>>> >>>>>My question is following. >>>>>1.I found that your recovery step is a little different with the link about sync the volume.I want to know if both methods are right for recovering the replicate volume? >>>>>2.Is this link is just fit for glusterfs 3.4?My glusterfs version is 3.7.6.Is the the link fit the glusterfs 3.7.6? >>>>> >>>>> >>>>>Thanks, >>>>>Xin >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>At 2016-03-17 14:51:34, "Gaurav Garg" <ggarg at redhat.com> wrote: >>>>>>>> Could I run some glusterfs command on good node to recover the replicate volume? if I don't copy the files ,including glusterd.info and other files,from good node to new node. >>>>>> >>>>>> >>>>>>running glusterfs command is not enough to recover the replicate volume. for recovery you need to follow following steps. >>>>>> >>>>>>1) remove /var/lib/glusterd/* data from new node (if in any case it present) then start glusterd on new node >>>>>>1) kill glusterd on new node >>>>>>2) from 1st node (which is in good condition) execute #gluster peer status command and copy the uuid from the peer status (you will see one failed node entry with hostname and uuid) and replace this UUID in node file /var/lib/glusterd/glusterd.info >>>>>>from the 1st node you can also get the uuid of failed node by doing #cat /var/lib/glusterd/peer/* it will show uuid of failed node along with hostname/ip-address of failed node. >>>>>>3) copy /var/lib/glusterd/peers/* to new node. >>>>>>4) you need to rename one of the /var/lib/glusterd/peers/* file (you can find that file in new node by just matching the uuid of new node (/var/lib/glusterd/glusterd.info) with /var/lib/glusterd/peers/* file name) with the uuid of 1st node (/var/lib/glusterd/glusterd.info) and modify the content of the same file with having uuid of 1st node and hostname of 1st node. >>>>>>5) now start glusterd on new node. >>>>>>6) your volume will recover. >>>>>>> >>>>>>above steps are mandatory steps to recover failed node. >>>>>> >>>>>>Thanks, >>>>>> >>>>>>Regards, >>>>>>Gaurav >>>>>> >>>>>>----- Original Message ----- >>>>>>From: "songxin" <songxin_1980 at 126.com> >>>>>>To: "Alastair Neil" <ajneil.tech at gmail.com> >>>>>>Cc: gluster-users at gluster.org >>>>>>Sent: Thursday, March 17, 2016 8:56:58 AM >>>>>>Subject: Re: [Gluster-users] How to recover after one node breakdown >>>>>> >>>>>>Thank you very much for your reply. >>>>>> >>>>>>In fact it is that I use a new node ,of which rootfs is new , to replace the failed node. >>>>>>And the new node has same IP address with the failed one. >>>>>> >>>>>>The brick is on a external hard disk.Because the hard disk is mounted on the node ?so the data on the brick of failed node will not be loss but may be async with the brick of good node.And the brick of failed node will be mounted on the new node. >>>>>> >>>>>>Now my recovery steps is run some glusterfs command on good node as below, after starting the glusterd on new node. >>>>>>1.remove brick of new node from volume(the volume type is changed from replicate to distribute) >>>>>>2.peer detach the new node ip(the new node ip is same as failed node) >>>>>>3.peer probe the new node ip >>>>>>3.add brick of new node to volume(the volume type is change to replicate) >>>>>> >>>>>>But many problem,like data async or peer state is error etc, will happen. >>>>>> >>>>>>My question is below. >>>>>> >>>>>>Could I run some glusterfs command on good node to recover the replicate volume? if I don't copy the files ,including glusterd.info and other files,from good node to new node. >>>>>> >>>>>>Thanks >>>>>>Xin >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>???? iPhone >>>>>> >>>>>>? 2016?3?17??04:54?Alastair Neil < ajneil.tech at gmail.com > ??? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>hopefully you have a back up of /var/lib/glusterd/ glusterd.info and /var/lib/glusterd/peers, if so I think you can copy them back to and restart glusterd and the volume info should get populated from the other node. If not you can probably reconstruct these from these files on the other node. >>>>>> >>>>>>i.e: >>>>>>On the unaffected node the peers directory should have an entry for the failed node containing the uuid of the failed node. The glusterd.info file should enable you to recreate the peer file on the failed node. >>>>>> >>>>>> >>>>>>On 16 March 2016 at 09:25, songxin < songxin_1980 at 126.com > wrote: >>>>>> >>>>>> >>>>>> >>>>>>Hi, >>>>>>Now I face a problem. >>>>>>Reproduc step is as below. >>>>>>1.I create a replicate volume using two brick on two board >>>>>>2.start the volume >>>>>>3.one board is breakdown and all >>>>>>files in the rootfs ,including /var/lib/glusterd/*,are lost. >>>>>>4.reboot the board and ip is not change. >>>>>> >>>>>>My question: >>>>>>How to recovery the replicate volume? >>>>>> >>>>>>Thanks, >>>>>>Xin >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>_______________________________________________ >>>>>>Gluster-users mailing list >>>>>>Gluster-users at gluster.org >>>>>>http://www.gluster.org/mailman/listinfo/gluster-users >>>>>> >>>>>> >>>>>>_______________________________________________ >>>>>>Gluster-users mailing list >>>>>>Gluster-users at gluster.org >>>>>>http://www.gluster.org/mailman/listinfo/gluster-users >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Gluster-users mailing list >>>> Gluster-users at gluster.org >>>> http://www.gluster.org/mailman/listinfo/gluster-users >>>> >> >> >> >> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160321/e58d5509/attachment.html>
songxin
2016-Mar-22 09:37 UTC
[Gluster-users] question about gluster volume heal info split-brain
Hi? I have a quesition about heal info split-brain. I know that the gfid mismatch is a kind of split-brain and the parent directory should be show split-brain. In my case the "gluster volume heal info split-brain" show that no file is split-brain, though same filename has diffetent gfid on two bricks of a replicate volume. And access the file it will who Input/output error. precondition: 2.A node ip:10.32.0.48; 3.B node ip:10.32.1.144 4.A brick: /opt/lvmdir/c2/brick on A node 5.B brick: /opt/lvmdir/c2/brick on B node reproduce: 1.create a replicate volume use two brick, A brick and B brick (on A node) 2.start volume (on A node) 2.mount mount point (/mnt/c) on the volume (on A node) 3.mount mount point (/mnt/c) on the volume (on B node) 4.access the mount point (A node and B node) 5.reboot B node 6.start glusterd (B node) 7.remove B brick from replicate volume (A node) 8.peer detach 10.32.1.144 (A node) 9.peer probe 10.32.1.144 (A node) 10.add B brick to volume (A node) 11.after some time? go to step 5 logs on A node: stat: cannot stat '/mnt/c/public_html/cello/ior_files/nameroot.ior': Input/output error getfattr -d -m . -e hex opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior # file: opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior trusted.afr.dirty=0x000000000000000000000000 trusted.bit-rot.version=0x000000000000000256e812da0007bf13 trusted.gfid=0xc18f775d94de42879235d1331d85c860 getfattr -d -m . -e hex opt/lvmdir/c2/brick/public_html/cello/ior_files # file: opt/lvmdir/c2/brick/public_html/cello/ior_files trusted.afr.c_glusterfs-client-1=0x000000000000000000000000 trusted.afr.c_glusterfs-client-207=0x000000000000000000000002 trusted.afr.c_glusterfs-client-209=0x000000000000000000000000 trusted.afr.c_glusterfs-client-215=0x000000000000000000000000 trusted.afr.c_glusterfs-client-39=0x000000000000000000000000 trusted.afr.c_glusterfs-client-47=0x000000000000000000000000 trusted.afr.c_glusterfs-client-49=0x000000000000000000000002 trusted.afr.c_glusterfs-client-51=0x000000000000000000000000 trusted.afr.dirty=0x000000000000000000000000 trusted.gfid=0xd9cd3be03fa44d1e8a8da8523535ef0a trusted.glusterfs.dht=0x000000010000000000000000ffffffff logs on B node: stat: cannot stat '/mnt/c/public_html/cello/ior_files/nameroot.ior': Input/output error getfattr -d -m . -e hex opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior # file: opt/lvmdir/c2/brick/public_html/cello/ior_files/nameroot.ior trusted.bit-rot.version=0x000000000000000256e813c50008b4e2 trusted.gfid=0x32145e0378864767989335f37c108409 getfattr -d -m . -e hex opt/lvmdir/c2/brick/public_html/cello/ior_files # file: opt/lvmdir/c2/brick/public_html/cello/ior_files trusted.afr.c_glusterfs-client-112=0x000000000000000000000000 trusted.afr.c_glusterfs-client-116=0x000000000000000000000000 trusted.afr.c_glusterfs-client-128=0x000000000000000000000000 trusted.afr.c_glusterfs-client-130=0x000000000000000000000000 trusted.afr.c_glusterfs-client-150=0x000000000000000000000000 trusted.afr.c_glusterfs-client-164=0x000000000000000000000000 trusted.afr.c_glusterfs-client-166=0x000000000000000000000000 trusted.afr.c_glusterfs-client-194=0x000000000000000000000000 trusted.afr.c_glusterfs-client-196=0x000000000000000000000000 trusted.afr.c_glusterfs-client-200=0x000000000000000000000000 trusted.afr.c_glusterfs-client-224=0x000000000000000000000000 trusted.afr.c_glusterfs-client-26=0x000000000000000000000000 trusted.afr.c_glusterfs-client-36=0x000000000000000000000000 trusted.afr.c_glusterfs-client-38=0x000000000000000000000000 trusted.afr.c_glusterfs-client-40=0x000000000000000000000000 trusted.afr.c_glusterfs-client-50=0x000000000000000000000000 trusted.afr.c_glusterfs-client-54=0x000000000000000000000000 trusted.afr.c_glusterfs-client-58=0x000000000000000000000002 trusted.afr.c_glusterfs-client-64=0x000000000000000000000000 trusted.afr.c_glusterfs-client-66=0x000000000000000000000000 trusted.afr.c_glusterfs-client-70=0x000000000000000000000000 trusted.afr.c_glusterfs-client-76=0x000000000000000000000000 trusted.afr.c_glusterfs-client-84=0x000000000000000000000000 trusted.afr.c_glusterfs-client-90=0x000000000000000000000000 trusted.afr.c_glusterfs-client-98=0x000000000000000000000000 trusted.afr.dirty=0x000000000000000000000000 trusted.gfid=0xd9cd3be03fa44d1e8a8da8523535ef0a trusted.glusterfs.dht=0x000000010000000000000000ffffffff Thanks, Xin -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160322/54fc13f5/attachment.html>
songxin
2016-Mar-24 10:53 UTC
[Gluster-users] about directory .glusterfs/indices/xattrop/ after adding brick
Hi, I have a question about files in .glusterfs/indices/xattrop/. reproduce: 1.create a replicate volume using two brick 2.kill A brick process 3.create some files in mount point 4.run "gluster volume heal gv0 info" show some files need heal 5.ls B_brick/.glusterfs/indices/xattrop/ show gfid of files that need heal 6.remove A brick from the volume 7.Add A brick into the volume 8.run "gluster volume heal gv0 info" show no files need heal 9.ls B_brick_dir/.glusterfs/indices/xattrop/ show no gfid 10.running "ls A_brick_dir" show no files that create in mount point 11.getfattr -m. -d -e hex B_brick_dir/files show that files in A brick need heal my question: 1.if adding brick to volume will clear the directory .glusterfs/indices/xattrop/* Thanks, Xin -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160324/8c8da7bb/attachment.html>
Hi, When I start the glusterd some error happened. And the log is following. [2016-11-08 07:58:34.989365] I [MSGID: 100030] [glusterfsd.c:2318:main] 0-/usr/sbin/glusterd: Started running /usr/sbin/glusterd version 3.7.6 (args: /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO) [2016-11-08 07:58:34.998356] I [MSGID: 106478] [glusterd.c:1350:init] 0-management: Maximum allowed open file descriptors set to 65536 [2016-11-08 07:58:35.000667] I [MSGID: 106479] [glusterd.c:1399:init] 0-management: Using /system/glusterd as working directory [2016-11-08 07:58:35.024508] I [MSGID: 106514] [glusterd-store.c:2075:glusterd_restore_op_version] 0-management: Upgrade detected. Setting op-version to minimum : 1 [2016-11-08 07:58:35.025356] E [MSGID: 106206] [glusterd-store.c:2562:glusterd_store_update_volinfo] 0-management: Failed to get next store iter [2016-11-08 07:58:35.025401] E [MSGID: 106207] [glusterd-store.c:2844:glusterd_store_retrieve_volume] 0-management: Failed to update volinfo for c_glusterfs volume [2016-11-08 07:58:35.025463] E [MSGID: 106201] [glusterd-store.c:3042:glusterd_store_retrieve_volumes] 0-management: Unable to restore volume: c_glusterfs [2016-11-08 07:58:35.025544] E [MSGID: 101019] [xlator.c:428:xlator_init] 0-management: Initialization of volume 'management' failed, review your volfile again [2016-11-08 07:58:35.025582] E [graph.c:322:glusterfs_graph_init] 0-management: initializing translator failed [2016-11-08 07:58:35.025629] E [graph.c:661:glusterfs_graph_activate] 0-graph: init failed [2016-11-08 07:58:35.026109] W [glusterfsd.c:1236:cleanup_and_exit] (-->/usr/sbin/glusterd(glusterfs_volumes_init-0x1b260) [0x1000a718] -->/usr/sbin/glusterd(glusterfs_process_volfp-0x1b3b8) [0x1000a5a8] -->/usr/sbin/glusterd(cleanup_and_exit-0x1c02c) [0x100098bc] ) 0-: received signum (0), shutting down And then I found that the size of vols/volume_name/info is 0.It cause glusterd shutdown. But I found that vols/volume_name_info.tmp is not 0. And I found that there is a brick file vols/volume_name/bricks/xxxx.brick is 0, but vols/volume_name/bricks/xxxx.brick.tmp is not 0. I read the function code glusterd_store_volinfo () in glusterd-store.c . I know that the info.tmp will be rename to info in function glusterd_store_volume_atomic_update(). But my question is that why the info file is 0 but info.tmp is not 0. Thanks, Xin -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161110/056f53b1/attachment.html>