Hi, We have in production 3 nodes for a volume storing VM files on 3.7.6, suddenly the third node isn't seen by the others. For exemple in the heal infos : Status: Brick is not connected The network seems to be working fine, the 3 nodes are in cluster and can ping each other. gluster volume status lists the 3 bricks and I restarted the daemon on the third node to be sure, still the same result. The brick is stored on xfs on /mnt/vg1-storage and all the files seems to be there, it's not read only or anything. Where can I check ? Thanks -- Kevin Lemonnier PGP Fingerprint : 89A5 2283 04A0 E6E9 0111 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160523/88b5ff20/attachment.sig>
Looks like it crashed, if I do a /etc/init.d/gluster-server stop on the third node and ps aux | grep gluster after that there is still a lot of processes listed. Should I kill everything ? On Mon, May 23, 2016 at 04:31:28PM +0200, Kevin Lemonnier wrote:> Hi, > > We have in production 3 nodes for a volume storing VM files on 3.7.6, > suddenly the third node isn't seen by the others. For exemple in the heal > infos : > > Status: Brick is not connected > > The network seems to be working fine, the 3 nodes are in cluster and can ping > each other. gluster volume status lists the 3 bricks and I restarted the daemon > on the third node to be sure, still the same result. > The brick is stored on xfs on /mnt/vg1-storage and all the files seems to be there, > it's not read only or anything. > > Where can I check ? > Thanks > > > -- > Kevin Lemonnier > PGP Fingerprint : 89A5 2283 04A0 E6E9 0111> _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users-- Kevin Lemonnier PGP Fingerprint : 89A5 2283 04A0 E6E9 0111 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160523/510e7828/attachment.sig>
On 24 May 2016 at 00:31, Kevin Lemonnier <lemonnierk at ulrar.net> wrote:> Status: Brick is not connected > > The network seems to be working fine, the 3 nodes are in cluster and can ping > each other. gluster volume status lists the 3 bricks and I restarted the daemon > on the third node to be sure, still the same result.I had that exact same problem recently when running tests with 3.7.11, the only thing that resolved was to kill *all* gluster processes (glusterd, glusterfs, glusterfsd) on the node and restart glusterd -- Lindsay