Stanislav Alekseev
2011-May-02 14:23 UTC
[Gluster-users] Glusterfs distributed volume stops responding with bailing out frame type(GlusterFS 3.1) op(LOOKUP(27))
Glusterfs 3.1.1 volume stops responding after some time until I perform "volume stop; volume start". The following messages are displayed in the client's log: 04:29:41.286651. timeout = 1800 [2011-05-02 05:29:44.952153] E [rpc-clnt.c:199:call_bail] cccbstore1-client-0: bailing out frame type(GlusterFS 3.1) op(LOOKUP(27)) xid = 0x311c sent = 2011-05-02 04:59:43.110723. timeout = 1800 [2011-05-02 05:59:46.773153] E [rpc-clnt.c:199:call_bail] cccbstore1-client-0: bailing out frame type(GlusterFS 3.1) op(LOOKUP(27)) xid = 0x311d sent = 2011-05-02 05:29:44.952650. timeout = 1800 [2011-05-02 06:29:48.597153] E [rpc-clnt.c:199:call_bail] cccbstore1-client-0: bailing out frame type(GlusterFS 3.1) op(LOOKUP(27)) xid = 0x311e sent = 2011-05-02 05:59:46.773627. timeout = 1800 [2011-05-02 06:59:50.435152] E [rpc-clnt.c:199:call_bail] cccbstore1-client-0: bailing out frame type(GlusterFS 3.1) op(LOOKUP(27)) xid = 0x311f sent = 2011-05-02 06:29:48.597779. timeout = 1800 [2011-05-02 07:19:46.460771] C [rpc-clnt.c:430:rpc_clnt_fill_request_info] rpc-clnt: cannot lookup the saved frame corresponding to xid (12556) for msg arrived on transport cccbstore1-client-0 [2011-05-02 07:19:46.461094] E [rpc-clnt.c:338:saved_frames_unwind] (-->/usr/lib64/libgfrpc.so.0(rpc_clnt_notify+0xb9) [0x2b512972f769] (-->/usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x7e) [0x2b512972ef1e] (-->/usr/lib64/libgfrpc.so.0(saved_frames_destroy+0xe) [0x2b512972ee8e]))) rpc-clnt: forced unwinding frame type(GlusterFS 3.1) op(LOOKUP(27)) called at 2011-05-02 06:59:50.435645 [2011-05-02 07:19:46.658640] I [client-handshake.c:993:select_server_supported_programs] cccbstore1-client-0: Using Program GlusterFS-3.1.0, Num (1298437), Version (310) [2011-05-02 07:19:46.659111] I [client-handshake.c:829:client_setvolume_cbk] cccbstore1-client-0: Connected to 155.52.49.32:24010, attached to remote volume '/u01/cccbstore1'. [2011-05-02 07:20:41.726069] I [client-handshake.c:993:select_server_supported_programs] cccbstore1-client-0: Using Program GlusterFS-3.1.0, Num (1298437), Version (310) [2011-05-02 07:20:41.726583] I [client-handshake.c:829:client_setvolume_cbk] cccbstore1-client-0: Connected to 155.52.49.32:24010, attached to remote volume '/u01/cccbstore1'. [2011-05-02 07:20:42.737992] I [client-handshake.c:993:select_server_supported_programs] cccbstore1-client-3: Using Program GlusterFS-3.1.0, Num (1298437), Version (310) [2011-05-02 07:20:42.738430] I [client-handshake.c:829:client_setvolume_cbk] cccbstore1-client-3: Connected to 155.52.49.35:24010, attached to remote volume '/u01/cccbstore1'. [2011-05-02 07:20:42.748585] I [client-handshake.c:993:select_server_supported_programs] cccbstore1-client-2: Using Program GlusterFS-3.1.0, Num (1298437), Version (310) [2011-05-02 07:20:42.749036] I [client-handshake.c:829:client_setvolume_cbk] cccbstore1-client-2: Connected to 155.52.49.34:24010, attached to remote volume '/u01/cccbstore1'. [2011-05-02 07:20:42.762387] I [client-handshake.c:993:select_server_supported_programs] cccbstore1-client-1: Using Program GlusterFS-3.1.0, Num (1298437), Version (310) [2011-05-02 07:20:42.763134] I [client-handshake.c:829:client_setvolume_cbk] cccbstore1-client-1: Connected to 155.52.49.33:24010, attached to remote volume '/u01/cccbstore1'. Please advice. Thanks.