Joseph Lorenzini
2017-Jan-25 13:27 UTC
[Gluster-users] possible kernel panic with glusterd
Hi Atin, I assume you are referring to the /var/log/glustershd.log. If so, that file never gets created. Joe On Wed, Jan 25, 2017 at 6:14 AM Atin Mukherjee <amukherj at redhat.com> wrote:> On Wed, Jan 25, 2017 at 5:15 PM, Joseph Lorenzini <jaloren at gmail.com> > wrote: > > Hi all, > > I have recently started exploring the DFS solution space and was doing > some basic setup and testing with gluster. I set up a pool of three nodes > following the quick start guide. That seemed to work fine. > > However, shortly after that, I noticed that one of the servers in the pool > was becoming non-responsive -- as in the entire VM was completely hung and > i had to use the hypervisor to force a reboot. I sshed into the server and > startd poking around. glusterd was shut off. I started it up ....and the > following happened: > > Message from syslogd at Jan 25 05:20:47 ... > kernel:[ 288.145027] NMI watchdog: BUG: soft lockup - CPU#1 stuck for > 22s! [glusterd:2374] > > > Could you attach the glusterd log file to enable us to look at why > glusterd got shutdown? > > > > At which point, the VM became completely unresponsive again. > > All servers are the same. They are running centos 7.3, linux kernel > 3.10.0-514.2.2.el7.x86_64. The glusterfs-server is 3.8. > > Since I just started investigating gluster, it is certainly possible that > I misconfigured something on that one node. However, a kernel hang/panic > seems like an excessive response :). If anyone would have any ideas or > suggestions about what may be happening here or additional places I should > look into to find out what is going on, I am all ears. > > Thanks, > Joe > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users > > > > > -- > > ~ Atin (atinm) >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170125/f24c1ade/attachment.html>
Samikshan Bairagya
2017-Jan-25 13:44 UTC
[Gluster-users] possible kernel panic with glusterd
On 01/25/2017 06:57 PM, Joseph Lorenzini wrote:> Hi Atin, > > I assume you are referring to the /var/log/glustershd.log. If so, that file > never gets created.You'd find the glusterd log file as /var/log/glusterfs/etc-glusterfs-glusterd.vol.log AFAR for glusterfs-3.8. ~ Samikshan> > Joe > > On Wed, Jan 25, 2017 at 6:14 AM Atin Mukherjee <amukherj at redhat.com> wrote: > >> On Wed, Jan 25, 2017 at 5:15 PM, Joseph Lorenzini <jaloren at gmail.com> >> wrote: >> >> Hi all, >> >> I have recently started exploring the DFS solution space and was doing >> some basic setup and testing with gluster. I set up a pool of three nodes >> following the quick start guide. That seemed to work fine. >> >> However, shortly after that, I noticed that one of the servers in the pool >> was becoming non-responsive -- as in the entire VM was completely hung and >> i had to use the hypervisor to force a reboot. I sshed into the server and >> startd poking around. glusterd was shut off. I started it up ....and the >> following happened: >> >> Message from syslogd at Jan 25 05:20:47 ... >> kernel:[ 288.145027] NMI watchdog: BUG: soft lockup - CPU#1 stuck for >> 22s! [glusterd:2374] >> >> >> Could you attach the glusterd log file to enable us to look at why >> glusterd got shutdown? >> >> >> >> At which point, the VM became completely unresponsive again. >> >> All servers are the same. They are running centos 7.3, linux kernel >> 3.10.0-514.2.2.el7.x86_64. The glusterfs-server is 3.8. >> >> Since I just started investigating gluster, it is certainly possible that >> I misconfigured something on that one node. However, a kernel hang/panic >> seems like an excessive response :). If anyone would have any ideas or >> suggestions about what may be happening here or additional places I should >> look into to find out what is going on, I am all ears. >> >> Thanks, >> Joe >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-users >> >> >> >> >> -- >> >> ~ Atin (atinm) >> > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >