Hello, Gluster 3.3.0 distributed replicated, ext4 bricks. Since this morning I am unable to check the status of the filesystem: ----- cut here ----- # gluster volume status all operation failed Failed to get names of volumes ----- cut here ----- Extract from cli.log: ----- cut here ----- [2012-08-27 11:10:00.341089] W [rpc-transport.c:174:rpc_transport_load] 0-rpc-transport: missing 'option transport-type'. defaulting to "socket" [2012-08-27 11:10:00.439743] E [cli-rpc-ops.c:5657:gf_cli_status_volume_all] 0-cli: status all failed [2012-08-27 11:10:00.439791] I [input.c:46:cli_batch] 0-: Exiting with: -22 ----- cut here ----- "gluster peer" reports all nodes connected but it seems to be known as not really reliable. Any suggestion will be really appreciated. Best regards
* Tao Lin <linbaiye at gmail.com> [2012 08 27, 17:54]:> There are issues with gluster on ext4,you have to use other file > systems(eg. xfs, ext3) instead of ext4.If you are referring to http://joejulian.name/blog/glusterfs-bit-by-ext4-structure-change/ I don't think I was experienceing that problem, since I shouldn't have an affected kernel version. Update: now I am able to issue the command, and indeed noticed that one of the nodes was offline. In the brick log I found: ----- cut here ----- patchset: git://git.gluster.com/glusterfs.git signal received: 7 time of crash: 2012-08-24 23:22:30 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.3.0 /lib/libc.so.6(+0x33af0)[0x7f024da35af0] /usr/lib/libglusterfs.so.0(__dentry_grep+0x8e)[0x7f024e7879de] /usr/lib/libglusterfs.so.0(inode_grep+0x66)[0x7f024e787c56] /usr/lib/glusterfs/3.3.0/xlator/protocol/server.so(resolve_entry_simple+0x91)[0x7f02491eb641] /usr/lib/glusterfs/3.3.0/xlator/protocol/server.so(server_resolve_entry+0x24)[0x7f02491ebd14] /usr/lib/glusterfs/3.3.0/xlator/protocol/server.so(server_resolve+0x98)[0x7f02491ebb88] /usr/lib/glusterfs/3.3.0/xlator/protocol/server.so(server_resolve_all+0x9e)[0x7f02491ebcbe] /usr/lib/glusterfs/3.3.0/xlator/protocol/server.so(resolve_and_resume+0x14)[0x7f02491ebd84] /usr/lib/glusterfs/3.3.0/xlator/protocol/server.so(server_lookup+0x18f)[0x7f024920525f] /usr/lib/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x293)[0x7f024e550ce3] /usr/lib/libgfrpc.so.0(rpcsvc_notify+0x93)[0x7f024e550e53] /usr/lib/libgfrpc.so.0(rpc_transport_notify+0x28)[0x7f024e5518b8] /usr/lib/glusterfs/3.3.0/rpc-transport/socket.so(socket_event_poll_in+0x34)[0x7f024acd0734] /usr/lib/glusterfs/3.3.0/rpc-transport/socket.so(socket_event_handler+0xc7)[0x7f024acd0817] /usr/lib/libglusterfs.so.0(+0x3e394)[0x7f024e79b394] /usr/sbin/glusterfsd(main+0x58a)[0x407aaa] /lib/libc.so.6(__libc_start_main+0xfd)[0x7f024da20c4d] /usr/sbin/glusterfsd[0x404a59] ----- cut here ----- Regards
* s19n <mailbox at s19n.net> [2012 08 27, 11:25]:> ----- cut here ----- > # gluster volume status all > operation failed > > Failed to get names of volumes > ----- cut here -----Additional update: seems that the command (and many others) is not working because of a 'local lock' being held. I have tried running the command on the host holding the lock as well as on other hosts, with no success. etc-glusterfs-glusterd.vol.log extract follows: ----- cut here ----- [2012-08-27 17:41:17.842441] I [glusterd-volume-ops.c:583:glusterd_handle_cli_statedump_volume] 0-glusterd: Received statedump request for volume storage with options [2012-08-27 17:41:17.842510] E [glusterd-utils.c:277:glusterd_lock] 0-glusterd: Unable to get lock for uuid: 34b665ea-d315-489b-bd0f-172bb6b85ee1, lock held by: 34b665ea-d315-489b-bd0f-172bb6b85ee1 [2012-08-27 17:41:17.842527] E [glusterd-handler.c:453:glusterd_op_txn_begin] 0-management: Unable to acquire local lock, ret: -1 ----- cut here ----- Could you please describe why these locks are being held and, if possible, how to clear them? Thank you very much for your kind attention, Best regards