Hi, I'm using GlusterFS since many years and never see this problem, but this is the second time in one week ... I have 3 volumes with 2 bricks and 1 volume crash with no reason, I just have to stop/start the volume to make it up again. The only logs I can find are in syslog :> Jun 26 11:27:44 nfs05 srv-cdn[7879]: pending frames: > Jun 26 11:27:44 nfs05 srv-cdn[7879]: frame : type(0) op(10) > Jun 26 11:27:44 nfs05 srv-cdn[7879]: patchset: > git://git.gluster.com/glusterfs.git > Jun 26 11:27:44 nfs05 srv-cdn[7879]: signal received: 11 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: time of crash: > Jun 26 11:27:44 nfs05 srv-cdn[7879]: 2016-06-26 09:27:44 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: configuration details: > Jun 26 11:27:44 nfs05 srv-cdn[7879]: argp 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: backtrace 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: dlfcn 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: libpthread 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: llistxattr 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: setfsid 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: spinlock 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: epoll.h 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: xattr.h 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: st_atim.tv_nsec 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: package-string: glusterfs 3.7.11 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: --------- > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: pending frames: > Jun 26 11:27:44 nfs06 srv-cdn[1787]: frame : type(0) op(10) > Jun 26 11:27:44 nfs06 srv-cdn[1787]: patchset: > git://git.gluster.com/glusterfs.git > Jun 26 11:27:44 nfs06 srv-cdn[1787]: signal received: 11 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: time of crash: > Jun 26 11:27:44 nfs06 srv-cdn[1787]: 2016-06-26 09:27:44 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: configuration details: > Jun 26 11:27:44 nfs06 srv-cdn[1787]: argp 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: backtrace 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: dlfcn 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: libpthread 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: llistxattr 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: setfsid 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: spinlock 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: epoll.h 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: xattr.h 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: st_atim.tv_nsec 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: package-string: glusterfs 3.7.11 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: --------- >Thanks for your help Regards -- Yann Lemari? iRaiser - Support Technique iRaiser Logotype <http://www.iraiser.eu> ylemarie at iraiser.eu <mailto:ylemarie at iraiser.eu> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160627/9ee178c0/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: achfiaca.png Type: image/png Size: 1842 bytes Desc: not available URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160627/9ee178c0/attachment.png>
Grep for "backtrace" in /var/log/glusterfs, that should give you a particular log file name by which you can determine which process crashed. You should also be able to see a coredump. Why don't you attach the same for further analysis? ~Atin On Mon, Jun 27, 2016 at 1:17 PM, Yann LEMARIE <ylemarie at iraiser.eu> wrote:> Hi, > > I'm using GlusterFS since many years and never see this problem, but this > is the second time in one week ... > > I have 3 volumes with 2 bricks and 1 volume crash with no reason, I just > have to stop/start the volume to make it up again. > The only logs I can find are in syslog : > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: pending frames: > Jun 26 11:27:44 nfs05 srv-cdn[7879]: frame : type(0) op(10) > Jun 26 11:27:44 nfs05 srv-cdn[7879]: patchset: git:// > git.gluster.com/glusterfs.git > Jun 26 11:27:44 nfs05 srv-cdn[7879]: signal received: 11 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: time of crash: > Jun 26 11:27:44 nfs05 srv-cdn[7879]: 2016-06-26 09:27:44 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: configuration details: > Jun 26 11:27:44 nfs05 srv-cdn[7879]: argp 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: backtrace 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: dlfcn 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: libpthread 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: llistxattr 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: setfsid 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: spinlock 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: epoll.h 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: xattr.h 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: st_atim.tv_nsec 1 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: package-string: glusterfs 3.7.11 > Jun 26 11:27:44 nfs05 srv-cdn[7879]: --------- > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: pending frames: > Jun 26 11:27:44 nfs06 srv-cdn[1787]: frame : type(0) op(10) > Jun 26 11:27:44 nfs06 srv-cdn[1787]: patchset: git:// > git.gluster.com/glusterfs.git > Jun 26 11:27:44 nfs06 srv-cdn[1787]: signal received: 11 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: time of crash: > Jun 26 11:27:44 nfs06 srv-cdn[1787]: 2016-06-26 09:27:44 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: configuration details: > Jun 26 11:27:44 nfs06 srv-cdn[1787]: argp 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: backtrace 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: dlfcn 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: libpthread 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: llistxattr 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: setfsid 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: spinlock 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: epoll.h 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: xattr.h 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: st_atim.tv_nsec 1 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: package-string: glusterfs 3.7.11 > Jun 26 11:27:44 nfs06 srv-cdn[1787]: --------- > > > Thanks for your help > > > Regards > -- > Yann Lemari? > iRaiser - Support Technique > [image: iRaiser Logotype] <http://www.iraiser.eu> > <ylemarie at iraiser.eu>ylemarie at iraiser.eu > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160627/08af0452/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: achfiaca.png Type: image/png Size: 1842 bytes Desc: not available URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160627/08af0452/attachment.png>
On Mon, 2016-06-27 at 09:47 +0200, Yann LEMARIE wrote:> Hi, > > I'm using GlusterFS since many years and never see this problem, but > this is the second time in one week ... > > I have 3 volumes with 2 bricks and 1 volume crash with no reason,Did you observe the crash while mounting the volume? Or can you be more specific on what were you doing just before you saw the crash? Can you please share the output of `gluster volume info <VOLNAME>`?> I just have to stop/start the volume to make it up again. > The only logs I can find are in syslog : > > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: pending frames: > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: frame : type(0) op(10) > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: patchset: > > git://git.gluster.com/glusterfs.git > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: signal received: 11 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: time of crash: > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: 2016-06-26 09:27:44 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: configuration details: > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: argp 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: backtrace 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: dlfcn 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: libpthread 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: llistxattr 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: setfsid 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: spinlock 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: epoll.h 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: xattr.h 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: st_atim.tv_nsec 1 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: package-string: glusterfs > > 3.7.11 > > Jun 26 11:27:44 nfs05 srv-cdn[7879]: --------- > > > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: pending frames: > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: frame : type(0) op(10) > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: patchset: > > git://git.gluster.com/glusterfs.git > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: signal received: 11 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: time of crash: > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: 2016-06-26 09:27:44 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: configuration details: > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: argp 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: backtrace 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: dlfcn 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: libpthread 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: llistxattr 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: setfsid 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: spinlock 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: epoll.h 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: xattr.h 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: st_atim.tv_nsec 1 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: package-string: glusterfs > > 3.7.11 > > Jun 26 11:27:44 nfs06 srv-cdn[1787]: --------- > > > ? > Thanks for your help > > > Regards > --? > Yann Lemari? > iRaiser - Support Technique > ? > ylemarie at iraiser.eu > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users