Raghavendra G
2009-Nov-19 17:09 UTC
[Gluster-users] Strange server locks isuess with 2.0.7 - updating
Hi Marek, Is it possible for you to provide a backtrace of the crash? And can you file a bug report at bugs.gluster.com? Also if possible can you upgrade to 2.8 release and confirm whether you face the same problem? regards, ----- Original Message ----- From: "Marek" <mb at kis.p.lodz.pl> To: gluster-users at gluster.org Sent: Thursday, November 19, 2009 7:20:44 PM GMT +04:00 Abu Dhabi / Muscat Subject: Re: [Gluster-users] Strange server locks isuess with 2.0.7 - updating Andre Felipe Machado wrote:> Hello, > Try to disable flush behind option. > It behaves *badly* with slow speed net, drives and machines... > It can cause the cited lock up behaviour. > Your server got caught in I/O dead lock. > Regards. > Andre Felipe Machado > http://www.techforce.com.brThanks, we tried disable flush behind option (we switch to a default writebehind options) but after a couple of hours glusterfs (on client) died with signall 11: pending frames: frame : type(1) op(LOOKUP) patchset: v2.0.7 signal received: 11 time of crash: 2009-11-14 03:37:28 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 2.0.7 [0xffffe400] /lib/tls/libc.so.6(calloc+0x8e)[0xb7dcfffe] /usr/local/lib/glusterfs/2.0.7/xlator/cluster/replicate.so(afr_lookup+0x32)[0xb74dfd12] /usr/local/lib/glusterfs/2.0.7/xlator/cluster/distribute.so(dht_lookup_directory+0x10a)[0xb74ce21a] /usr/local/lib/glusterfs/2.0.7/xlator/cluster/distribute.so(dht_lookup_cbk+0x10e)[0xb74d3d9e] /usr/local/lib/glusterfs/2.0.7/xlator/cluster/replicate.so(afr_lookup_cbk+0x47b)[0xb74e0a7b] /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(client_lookup_cbk+0x334)[0xb751ca84] /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(protocol_client_interpret+0x1ef)[0xb750c91f] /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(protocol_client_pollin+0xcd)[0xb750cb1d] /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(notify+0x23c)[0xb751359c] /usr/local/lib/glusterfs/2.0.7/transport/socket.so(socket_event_poll_in+0x3c)[0xb74abb2c] /usr/local/lib/glusterfs/2.0.7/transport/socket.so(socket_event_handler+0xa3)[0xb74abf73] /usr/local/lib/libglusterfs.so.0[0xb7ed7409] /usr/local/lib/libglusterfs.so.0(event_dispatch+0x21)[0xb7ed6251] glusterfs(main+0xc4f)[0x804bb3f] /lib/tls/libc.so.6(__libc_start_main+0xc8)[0xb7d7cea8] glusterfs[0x8049d71] --------- ===============================================================================Version : glusterfs 2.0.7 built on Nov 5 2009 12:49:05 git: v2.0.7 Starting Time: 2009-11-16 12:41:08 Command line : glusterfs --log-file=/var/log/gluster-client -f /usr/local/etc/glusterfs/glusterfs-client.vol /var/glustertest/ PID : 28270 System name : Linux Nodename : mb Kernel Release : 2.6.26.2 Hardware Identifier: i686 _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Marek
2009-Nov-25 13:27 UTC
[Gluster-users] Strange server locks isuess with 2.0.7 - updating
OK, we switched to 2.0.8. Till now we haven't any crash (yet), but some bricks still caught I/O dead lock (even with disabled writebehind translator). Some ppl on this group say that they have glusterfs running for a long period of time (months?) without aby problems, maybe they should share with us how they did this :-)? regards, Raghavendra G napisa?(a):> Hi Marek, > > Is it possible for you to provide a backtrace of the crash? And can you file a bug report at bugs.gluster.com? Also if possible can you upgrade to 2.8 release and confirm whether you face the same problem? > > regards, > ----- Original Message ----- > From: "Marek" <mb at kis.p.lodz.pl> > To: gluster-users at gluster.org > Sent: Thursday, November 19, 2009 7:20:44 PM GMT +04:00 Abu Dhabi / Muscat > Subject: Re: [Gluster-users] Strange server locks isuess with 2.0.7 - updating > > Andre Felipe Machado wrote: >> Hello, >> Try to disable flush behind option. >> It behaves *badly* with slow speed net, drives and machines... >> It can cause the cited lock up behaviour. >> Your server got caught in I/O dead lock. >> Regards. >> Andre Felipe Machado >> http://www.techforce.com.br > > Thanks, we tried disable flush behind option (we switch to a default writebehind options) but after > a couple of hours glusterfs (on client) died with signall 11: > > pending frames: > frame : type(1) op(LOOKUP) > > patchset: v2.0.7 > signal received: 11 > time of crash: 2009-11-14 03:37:28 > configuration details: > argp 1 > backtrace 1 > dlfcn 1 > fdatasync 1 > libpthread 1 > llistxattr 1 > setfsid 1 > spinlock 1 > epoll.h 1 > xattr.h 1 > st_atim.tv_nsec 1 > package-string: glusterfs 2.0.7 > [0xffffe400] > /lib/tls/libc.so.6(calloc+0x8e)[0xb7dcfffe] > /usr/local/lib/glusterfs/2.0.7/xlator/cluster/replicate.so(afr_lookup+0x32)[0xb74dfd12] > /usr/local/lib/glusterfs/2.0.7/xlator/cluster/distribute.so(dht_lookup_directory+0x10a)[0xb74ce21a] > /usr/local/lib/glusterfs/2.0.7/xlator/cluster/distribute.so(dht_lookup_cbk+0x10e)[0xb74d3d9e] > /usr/local/lib/glusterfs/2.0.7/xlator/cluster/replicate.so(afr_lookup_cbk+0x47b)[0xb74e0a7b] > /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(client_lookup_cbk+0x334)[0xb751ca84] > /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(protocol_client_interpret+0x1ef)[0xb750c91f] > /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(protocol_client_pollin+0xcd)[0xb750cb1d] > /usr/local/lib/glusterfs/2.0.7/xlator/protocol/client.so(notify+0x23c)[0xb751359c] > /usr/local/lib/glusterfs/2.0.7/transport/socket.so(socket_event_poll_in+0x3c)[0xb74abb2c] > /usr/local/lib/glusterfs/2.0.7/transport/socket.so(socket_event_handler+0xa3)[0xb74abf73] > /usr/local/lib/libglusterfs.so.0[0xb7ed7409] > /usr/local/lib/libglusterfs.so.0(event_dispatch+0x21)[0xb7ed6251] > glusterfs(main+0xc4f)[0x804bb3f] > /lib/tls/libc.so.6(__libc_start_main+0xc8)[0xb7d7cea8] > glusterfs[0x8049d71] > --------- > ===============================================================================> Version : glusterfs 2.0.7 built on Nov 5 2009 12:49:05 > git: v2.0.7 > Starting Time: 2009-11-16 12:41:08 > Command line : glusterfs --log-file=/var/log/gluster-client -f /usr/local/etc/glusterfs/glusterfs-client.vol /var/glustertest/ > PID : 28270 > System name : Linux > Nodename : mb > Kernel Release : 2.6.26.2 > Hardware Identifier: i686 > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >