search for: socket_event_poll_in

.../usr/sbin/glusterfs(mgmt_getspec_cbk+0x398)[0x40c888] /usr/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa5)[0x38d1a0f4d5] /usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0x120)[0x38d1a0fcd0] /usr/lib64/libgfrpc.so.0(rpc_transport_notify+0x28)[0x38d1a0aeb8] /usr/lib64/glusterfs/3.3.0/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7f1d47b8f784] /usr/lib64/glusterfs/3.3.0/rpc-transport/socket.so(socket_event_handler+0xc7)[0x7f1d47b8f867] /usr/lib64/libglusterfs.so.0[0x38d1e3e4a4] /usr/sbin/glusterfs(main+0x58a)[0x40731a] /lib64/libc.so.6(__libc_start_main+0xfd)[0x38d0a1ecdd] /usr/sbin/glusterfs[0x404289] --------- An...

...a=0x3b5c630, event=<value optimized out>, data=<value optimized out>) at rpc-clnt.c:905 #12 0x00007fc4f25faeb8 in rpc_transport_notify (this=<value optimized out>, event=<value optimized out>, data=<value optimized out>) at rpc-transport.c:489 #13 0x00007fc4eeeb0764 in socket_event_poll_in (this=0x3b6c060) at socket.c:1677 #14 0x00007fc4eeeb0847 in socket_event_handler (fd=<value optimized out>, idx=265, data=0x3b6c060, poll_in=1, poll_out=0, poll_err=<value optimized out>) at socket.c:1792 #15 0x00007fc4f2846464 in event_dispatch_epoll_handler (event_pool=0x177cdf0) at e...

...tor/protocol/client.so(protocol_client_interpret+0x1ef)[0xb750c91f] /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(protocol_client_pollin+0xcd)[0xb750cb1d] /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(notify+0x23c)[0xb751359c] /usr/local/lib/glusterfs/2.0.7/transport/socket.so(socket_event_poll_in+0x3c)[0xb74abb2c] /usr/local/lib/glusterfs/2.0.7/transport/socket.so(socket_event_handler+0xa3)[0xb74abf73] /usr/local/lib/libglusterfs.so.0[0xb7ed7409] /usr/local/lib/libglusterfs.so.0(event_dispatch+0x21)[0xb7ed6251] glusterfs(main+0xc4f)[0x804bb3f] /lib/tls/libc.so.6(__libc_start_main+0xc8)[0xb7...

...fff74002210, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:799 #12 0x00003fff885514fc in rpc_transport_notify (this=<optimized out>, event=<optimized out>, data=<optimized out>) at rpc-transport.c:546 #13 0x00003fff847fcd44 in socket_event_poll_in (this=this at entry=0x3fff74002210) at socket.c:2236 #14 0x00003fff847ff89c in socket_event_handler (fd=<optimized out>, idx=<optimized out>, data=0x3fff74002210, poll_in=<optimized out>, poll_out=<optimized out>, poll_err=<optimized out>) at socket.c:2349 #15 0x0000...

...sterfs/3.10.5/xlator/mgmt/glusterd.so #11 0x00007f41460cfad5 in rpc_clnt_handle_reply () from /usr/lib64/libgfrpc.so.0 #12 0x00007f41460d0c85 in rpc_clnt_notify () from /usr/lib64/libgfrpc.so.0 #13 0x00007f41460cbd68 in rpc_transport_notify () from /usr/lib64/libgfrpc.so.0 #14 0x00007f413ae8dccd in socket_event_poll_in () from /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so #15 0x00007f413ae8effe in socket_event_handler () from /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so #16 0x00007f4146362806 in event_dispatch_epoll_worker () from /usr/lib64/libglusterfs.so.0 #17 0x0000003d99c07aa1 in start_thread ()...

...terd.so > #11 0x00007f41460cfad5 in rpc_clnt_handle_reply () from > /usr/lib64/libgfrpc.so.0 > #12 0x00007f41460d0c85 in rpc_clnt_notify () from /usr/lib64/libgfrpc.so.0 > #13 0x00007f41460cbd68 in rpc_transport_notify () from > /usr/lib64/libgfrpc.so.0 > #14 0x00007f413ae8dccd in socket_event_poll_in () from > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > #15 0x00007f413ae8effe in socket_event_handler () from > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > #16 0x00007f4146362806 in event_dispatch_epoll_worker () from > /usr/lib64/libglusterfs.so.0 > #17 0x00000...

...mized out>, event=<optimized out>, data=<optimized out>) at > rpcsvc.c:799 > > #12 0x00003fff885514fc in rpc_transport_notify (this=<optimized out>, > event=<optimized out>, data=<optimized out>) at rpc-transport.c:546 > > #13 0x00003fff847fcd44 in socket_event_poll_in (this=this at entry=0x3fff74002210) > at socket.c:2236 > > #14 0x00003fff847ff89c in socket_event_handler (fd=<optimized out>, > idx=<optimized out>, data=0x3fff74002210, poll_in=<optimized out>, > poll_out=<optimized out>, poll_err=<optimized out>) at s...

Without the glusterd log file and the core file or the backtrace I can't comment anything. On Wed, Dec 6, 2017 at 3:09 PM, ABHISHEK PALIWAL <abhishpaliwal at gmail.com> wrote: > Any suggestion.... > > On Dec 6, 2017 11:51, "ABHISHEK PALIWAL" <abhishpaliwal at gmail.com> wrote: > >> Hi Team, >> >> We are getting the crash in glusterd after

On Mon, 4 Sep 2017 at 20:04, Serkan ?oban <cobanserkan at gmail.com> wrote: > I have been using a 60 server 1560 brick 3.7.11 cluster without > problems for 1 years. I did not see this problem with it. > Note that this problem does not happen when I install packages & start > glusterd & peer probe and create the volumes. But after glusterd > restart. > > Also

...xbe)[0xf61f1c1e] /usr/local/akamai/lib/libgfrpc.so.0(rpc_clnt_handle_reply+0xc2)[0xf7736c42] /usr/local/akamai/lib/libgfrpc.so.0(rpc_clnt_notify+0xa2)[0xf7736e62] /usr/local/akamai/lib/libgfrpc.so.0(rpc_transport_notify+0x35)[0xf77314c5] /usr/local/akamai/lib/glusterfs/3.1.0/rpc-transport/socket.so(socket_event_poll_in+0x50)[0xf5f08500] /usr/local/akamai/lib/glusterfs/3.1.0/rpc-transport/socket.so(socket_event_handler+0x15b)[0xf5f0867b] /usr/local/akamai/lib/libglusterfs.so.0[0xf7771cff] /usr/local/akamai/lib/libglusterfs.so.0(event_dispatch+0x21)[0xf7770a21] glusterfsc(main+0x48c)[0x804c45c] /lib/tls/i686/cmov/l...

...sterfs/3.10.5/xlator/mgmt/glusterd.so #13 0x000000311020fad5 in rpc_clnt_handle_reply () from /usr/lib64/libgfrpc.so.0 #14 0x0000003110210c85 in rpc_clnt_notify () from /usr/lib64/libgfrpc.so.0 #15 0x000000311020bd68 in rpc_transport_notify () from /usr/lib64/libgfrpc.so.0 #16 0x00007f9283492ccd in socket_event_poll_in () from /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so #17 0x00007f9283493ffe in socket_event_handler () from /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so #18 0x000000310fe87806 in event_dispatch_epoll_worker () from /usr/lib64/libglusterfs.so.0 #19 0x0000003d99c07aa1 in start_thread ()...

Dear all, I'm running GlusterFS 3.4 alpha2 together with oVirt 3.2. This is solely a test system and it doesn't have much data or anything important in it. Currently it has only 2 VM's running and disk usage is around 15 GB. I have been trying to set up a geo-replication for disaster recovery testing. For geo-replication I did following: All machines are running CentOS 6.4 and using

.../mgmt/glusterd.so > #13 0x000000311020fad5 in rpc_clnt_handle_reply () from /usr/lib64/libgfrpc.so.0 > #14 0x0000003110210c85 in rpc_clnt_notify () from /usr/lib64/libgfrpc.so.0 > #15 0x000000311020bd68 in rpc_transport_notify () from /usr/lib64/libgfrpc.so.0 > #16 0x00007f9283492ccd in socket_event_poll_in () from > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > #17 0x00007f9283493ffe in socket_event_handler () from > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > #18 0x000000310fe87806 in event_dispatch_epoll_worker () from > /usr/lib64/libglusterfs.so.0 > #19 0x00000...

...00000311020fad5 in rpc_clnt_handle_reply () from > /usr/lib64/libgfrpc.so.0 > > #14 0x0000003110210c85 in rpc_clnt_notify () from > /usr/lib64/libgfrpc.so.0 > > #15 0x000000311020bd68 in rpc_transport_notify () from > /usr/lib64/libgfrpc.so.0 > > #16 0x00007f9283492ccd in socket_event_poll_in () from > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > > #17 0x00007f9283493ffe in socket_event_handler () from > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > > #18 0x000000310fe87806 in event_dispatch_epoll_worker () from > > /usr/lib64/libgluste...

...rom > > > /usr/lib64/libgfrpc.so.0 > > > #14 0x0000003110210c85 in rpc_clnt_notify () from > > > /usr/lib64/libgfrpc.so.0 > > > #15 0x000000311020bd68 in rpc_transport_notify () from > > > /usr/lib64/libgfrpc.so.0 > > > #16 0x00007f9283492ccd in socket_event_poll_in () from > > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > > > #17 0x00007f9283493ffe in socket_event_handler () from > > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > > > #18 0x000000310fe87806 in event_dispatch_epoll_worker () from > >...

...311020fad5 in rpc_clnt_handle_reply () from > > /usr/lib64/libgfrpc.so.0 > > #14 0x0000003110210c85 in rpc_clnt_notify () from /usr/lib64/libgfrpc.so.0 > > #15 0x000000311020bd68 in rpc_transport_notify () from > > /usr/lib64/libgfrpc.so.0 > > #16 0x00007f9283492ccd in socket_event_poll_in () from > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > > #17 0x00007f9283493ffe in socket_event_handler () from > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so > > #18 0x000000310fe87806 in event_dispatch_epoll_worker () from > > /usr/lib64/libgluste...

...usr/lib64/libgfrpc.so.0 >> > > #14 0x0000003110210c85 in rpc_clnt_notify () from >> > > /usr/lib64/libgfrpc.so.0 >> > > #15 0x000000311020bd68 in rpc_transport_notify () from >> > > /usr/lib64/libgfrpc.so.0 >> > > #16 0x00007f9283492ccd in socket_event_poll_in () from >> > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so >> > > #17 0x00007f9283493ffe in socket_event_handler () from >> > > /usr/lib64/glusterfs/3.10.5/rpc-transport/socket.so >> > > #18 0x000000310fe87806 in event_dispatch_epoll_worker ()...

Serkan, I have gone through other mails in the mail thread as well but responding to this one specifically. Is this a source install or an RPM install ? If this is an RPM install, could you please install the glusterfs-debuginfo RPM and retry to capture the gdb backtrace. If this is a source install, then you'll need to configure the build with --enable-debug and reinstall and retry

Here you can find 10 stack trace samples from glusterd. I wait 10 seconds between each trace. https://www.dropbox.com/s/9f36goq5xn3p1yt/glusterd_pstack.zip?dl=0 Content of the first stack trace is here: Thread 8 (Thread 0x7f7a8cd4e700 (LWP 43069)): #0 0x0000003aa5c0f00d in nanosleep () from /lib64/libpthread.so.0 #1 0x000000303f837d57 in ?? () from /usr/lib64/libglusterfs.so.0 #2