search for: rpc_clnt_submit

...c:2843:gf_rdma_writev] 2-scratch-client-0: rdma is not connected to peer (172.17.2.254:49154) [2018-02-22 18:07:45.221309] W [MSGID: 103037] [rdma.c:3016:gf_rdma_submit_request] 2-rpc-transport/rdma: sending request to peer (172.17.2.254:49154) failed [2018-02-22 18:07:45.221320] W [rpc-clnt.c:1694:rpc_clnt_submit] 2-scratch-client-0: failed to submit rpc-request (XID: 0x54987 Program: GlusterFS 3.3, ProgVers: 330, Proc: 13) to rpc-transport (scratch-client-0) [2018-02-22 18:07:45.221334] W [MSGID: 114031] [client-rpc-fops.c:855:client3_3_writev_cbk] 2-scratch-client-0: remote operation failed [Transport end...

...client-2: forced unwinding frame type(GlusterFS 3.1) op(READ(12)) called at 2011-07-11 10:21:58.737035 [2011-07-11 10:21:58.739262] I [client3_1-fops.c:2302:client3_1_readv_cbk] 0-glusterfs: remote operation failed: Transport endpoint is not connected [2011-07-11 10:21:58.742823] W [rpc-clnt.c:1411:rpc_clnt_submit] 0-gluster-vol01-client-2: failed to submit rpc-request (XID: 0x57276x Program: GlusterFS 3.1, ProgVers: 310, Proc: 12) to rpc-transport (gluster-vol01-client-2) [2011-07-11 10:21:58.742850] I [client3_1-fops.c:2302:client3_1_readv_cbk] 0-gluster-vol01-client-2: remote operation failed: Transport e...

...9:50.397107] E [socket.c:3203:socket_connect] 0-glusterfs: connection attempt on 10.2.44.23:24007 failed, (Network is unreachable) [2017-06-29 17:59:50.397138] I [socket.c:3507:socket_submit_request] 0-glusterfs: not connected (priv->connected = 0) [2017-06-29 17:59:50.397162] W [rpc-clnt.c:1693:rpc_clnt_submit] 0-glusterfs: failed to submit rpc-request (XID: 0x3 Program: Gluster Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs) I think that important message is ?Network is unreachable?. Question 1. Could you please tell me, is that normal when you have many bricks? Networks is definitely stab...

.../win-share shadow:sort = desc shadow:snapprefix = ^S[A-Za-z0-9]*p$ shadow:format = _GMT-%Y.%m.%d-%H.%M.%S ------------- when we try to excess the share we get the following errormessage insite the logfile: -------------------- [2017-01-03 13:55:14.757049] T [rpc-clnt.c:1659:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x4a Program: GlusterFS 3.3, ProgVers: 330, Proc: 18) to rpc-transport (gv1-client-1 ) [2017-01-03 13:55:14.757291] T [rpc-clnt.c:687:rpc_clnt_reply_init] 0-gv1-client-2: received rpc message (RPC XID: 0x4b Program: GlusterFS 3.3, ProgVers: 330, Proc: 18) from r...

...clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xfdf60)[0x3fff9ec56fe0] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked-0x26060)[0x3fff9ebd9e20] (--> /usr/lib64/libgfrpc.so.0(+0x1a614)[0x3fff9ebda614] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_submit-0x29300)[0x3fff9ebd69b0] (--> /usr/lib64/glusterfs/3.10.5/xlator/protocol/client.so(+0x182e0)[0x3fff939182e0] ))))) 0-: 10.50.80.102:49152: ping timer event already removed [2017-09-20 13:34:23.345149] D [rpc-clnt-ping.c:93:rpc_clnt_remove_ping_timer_locked] (--> /usr/lib64/libglusterfs.s...

...[rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f36b3260192] (--> /lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked+0x8b)[0x7f36b302f9db] (--> /lib 64/libgfrpc.so.0(+0x13fd4)[0x7f36b302ffd4] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x451)[0x7f36b302cf01] (--> /usr/lib64/glusterfs/3.7.20/xlator/protocol/client.so(client_submit_request+0x1fc)[0x7f36a599c33c] ))))) 0-: 10.3.48.179:49155: ping timer event already remove d /DAEMON/DEBUG [2017-05-31T15:51:26.325230+02:00] [] [] [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_lock...

...[rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f36b3260192] (--> /lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked+0x8b)[0x7f36b302f9db] (--> /lib 64/libgfrpc.so.0(+0x13fd4)[0x7f36b302ffd4] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x451)[0x7f36b302cf01] (--> /usr/lib64/glusterfs/3.7.20/xlator/protocol/client.so(client_submit_request+0x1fc)[0x7f36a599c33c] ))))) 0-: 10.3.48.179:49155: ping timer event already remove d /DAEMON/DEBUG [2017-05-31T15:51:26.325230+02:00] [] [] [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_timer_lock...

...et.c:3203:socket_connect] 0-glusterfs: > connection attempt on 10.2.44.23:24007 failed, (Network is unreachable) > [2017-06-29 17:59:50.397138] I [socket.c:3507:socket_submit_request] > 0-glusterfs: not connected (priv->connected = 0) > [2017-06-29 17:59:50.397162] W [rpc-clnt.c:1693:rpc_clnt_submit] > 0-glusterfs: failed to submit rpc-request (XID: 0x3 Program: Gluster > Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs) > > I think that important message is ?Network is unreachable?. > > Question > 1. Could you please tell me, is that normal when you have many br...

...-clnt-ping.c:98:rpc_clnt_remove_ping_timer_locked] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x192)[0x7f36b3260192] (--> /lib64/libgfrpc.so.0(rpc_clnt_remove_ping_timer_locked+0x8b)[0x7f36b302f9db] (--> /lib > 64/libgfrpc.so.0(+0x13fd4)[0x7f36b302ffd4] (--> /lib64/libgfrpc.so.0(rpc_clnt_submit+0x451)[0x7f36b302cf01] (--> /usr/lib64/glusterfs/3.7.20/xlator/protocol/client.so(client_submit_request+0x1fc)[0x7f36a599c33c] ))))) 0-: 10.3.48.179:49155: ping timer event already remove > d > /DAEMON/DEBUG [2017-05-31T15:51:26.325230+02:00] [] [] [rpc-clnt-ping.c:98:rpc_clnt_remove_ping_...

...> 0-glusterfs: > > connection attempt on 10.2.44.23:24007 failed, (Network is unreachable) > > [2017-06-29 17:59:50.397138] I [socket.c:3507:socket_submit_request] > > 0-glusterfs: not connected (priv->connected = 0) > > [2017-06-29 17:59:50.397162] W [rpc-clnt.c:1693:rpc_clnt_submit] > > 0-glusterfs: failed to submit rpc-request (XID: 0x3 Program: Gluster > > Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs) > > > > I think that important message is ?Network is unreachable?. > > > > Question > > 1. Could you please tell me, i...

Looks like it is an issue with architecture compatibility in RPC layer (ie, with XDRs and how it is used). Just glance the logs of the client process where you saw the errors, which could give some hints. If you don't understand the logs, share them, so we will try to look into it. -Amar On Wed, Sep 20, 2017 at 2:40 AM, Walter Deignan <WDeignan at uline.com> wrote: > I recently

...[client3_1-fops.c:4379:client3_1_xattrop] 0-poolsave-client-0: failed to send the fop: Transport endpoint is not connected [2011-06-09 17:01:35.817662] I [socket.c:2272:socket_submit_request] 0-poolsave-client-1: not connected (priv->connected = 0) [2011-06-09 17:01:35.817698] W [rpc-clnt.c:1411:rpc_clnt_submit] 0-poolsave-client-1: failed to submit rpc-request (XID: 0x576x Program: GlusterFS 3.1, ProgVers: 310, Proc: 33) to rpc-transport (poolsave-client-1) [2011-06-09 17:01:35.817721] W [client3_1-fops.c:4735:client3_1_inodelk] 0-poolsave-client-0: failed to send the fop: Transport endpoint is not conne...

...s: >> > connection attempt on 10.2.44.23:24007 failed, (Network is unreachable) >> > [2017-06-29 17:59:50.397138] I [socket.c:3507:socket_submit_request] >> > 0-glusterfs: not connected (priv->connected = 0) >> > [2017-06-29 17:59:50.397162] W [rpc-clnt.c:1693:rpc_clnt_submit] >> > 0-glusterfs: failed to submit rpc-request (XID: 0x3 Program: Gluster >> > Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs) >> > >> > I think that important message is ?Network is unreachable?. >> > >> > Question >> > 1....

...t;         shadow:snapprefix = ^S[A-Za-z0-9]*p$ >         shadow:format = _GMT-%Y.%m.%d-%H.%M.%S > ------------- > > when we try to excess the share we get the following errormessage insite > the logfile: > -------------------- > [2017-01-03 13:55:14.757049] T [rpc-clnt.c:1659:rpc_clnt_submit] > 0-rpc-clnt: submitted request (XID: 0x4a Program: GlusterFS 3.3, > ProgVers: 330, Proc: 18) to rpc-transport (gv1-client-1 > ) > [2017-01-03 13:55:14.757291] T [rpc-clnt.c:687:rpc_clnt_reply_init] > 0-gv1-client-2: received rpc message (RPC XID: 0x4b Program: GlusterFS > 3.3, P...

...> connection attempt on 10.2.44.23:24007 failed, (Network is unreachable) >>> > [2017-06-29 17:59:50.397138] I [socket.c:3507:socket_submit_request] >>> > 0-glusterfs: not connected (priv->connected = 0) >>> > [2017-06-29 17:59:50.397162] W [rpc-clnt.c:1693:rpc_clnt_submit] >>> > 0-glusterfs: failed to submit rpc-request (XID: 0x3 Program: Gluster >>> > Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs) >>> > >>> > I think that important message is ?Network is unreachable?. >>> > >>> > Q...

I recently compiled the 3.10-5 client from source on a few PPC64 systems running RHEL 7.3. They are mounting a Gluster volume which is hosted on more traditional x86 servers. Everything seems to be working properly except for creating new directories from the PPC64 clients. The mkdir command gives a "Input/output error" and for the first few minutes the new directory is

...ux-gnu/libgfrpc.so.0(rpc_transport_notify+0x23)[0x7f012f6945b3] ))) )) 0-: /var/run/gluster/quotad.socket: ping timer event already removed [2017-04-03 22:20:24.705710] T [cli-quotad-client.c:100:cli_quotad_notify] 0-glusterfs: got RPC_CLNT_DISCONNECT [2017-04-03 22:20:24.705718] T [rpc-clnt.c:1598:rpc_clnt_submit] 0-rpc-clnt: submitted request (XID: 0x1 Program: Gluster CLI, ProgVers: 2, Proc: 1) to rpc-transport (glusterfs) [2017-04-03 22:20:24.705739] D [rpc-clnt-ping.c:281:rpc_clnt_start_ping] 0-glusterfs: ping timeout is 0, returning [2017-04-03 22:20:24.705723] D [MSGID: 0] [event-epoll.c:591:event_dis...

...mpt on 10.2.44.23:24007 failed, (Network is > unreachable) > >>> > [2017-06-29 17:59:50.397138] I [socket.c:3507:socket_submit_request] > >>> > 0-glusterfs: not connected (priv->connected = 0) > >>> > [2017-06-29 17:59:50.397162] W [rpc-clnt.c:1693:rpc_clnt_submit] > >>> > 0-glusterfs: failed to submit rpc-request (XID: 0x3 Program: Gluster > >>> > Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs) > >>> > > >>> > I think that important message is ?Network is unreachable?. > >>>...

...nnected from gluster1-client-2. Client process will keep trying to connect to glusterd until brick's port is available [2017-06-17 09:08:26.037127] I [socket.c:3401:socket_submit_request] 0-gluster1-client-1: not connected (priv->connected = 0) [2017-06-17 09:08:26.037145] W [rpc-clnt.c:1657:rpc_clnt_submit] 0-gluster1-client-1: failed to submit rpc-request (XID: 0x10e237 Program: GlusterFS 3.3, ProgVers: 330, Proc: 27) to rpc-transport (gluster1-client-1) [2017-06-17 09:08:26.037172] W [MSGID: 114031] [client-rpc-fops.c:2933:client3_3_lookup_cbk] 0-gluster1-client-1: remote operation failed. Path: /b...