ABHISHEK PALIWAL
2016-Nov-11 04:50 UTC
[Gluster-users] getting "Transport endpoint is not connected" in glusterfs mount log file.
Hi, Its an urgent case. Atleast provide your views on this On Wed, Nov 9, 2016 at 11:08 AM, ABHISHEK PALIWAL <abhishpaliwal at gmail.com> wrote:> Hi, > > We could see that sync is getting failed to sync the GlusterFS bricks due > to error trace "Transport endpoint is not connected " > > [2016-10-31 04:06:03.627395] E [MSGID: 114031] [client-rpc-fops.c:1673:client3_3_finodelk_cbk] > 0-c_glusterfs-client-9: remote operation failed [Transport endpoint is not > connected] > [2016-10-31 04:06:03.628381] I [socket.c:3308:socket_submit_request] > 0-c_glusterfs-client-9: not connected (priv->connected = 0) > [2016-10-31 04:06:03.628432] W [rpc-clnt.c:1586:rpc_clnt_submit] > 0-c_glusterfs-client-9: failed to submit rpc-request (XID: 0x7f5f Program: > GlusterFS 3.3, ProgVers: 330, Proc: 30) to rpc-transport > (c_glusterfs-client-9) > [2016-10-31 04:06:03.628466] E [MSGID: 114031] [client-rpc-fops.c:1673:client3_3_finodelk_cbk] > 0-c_glusterfs-client-9: remote operation failed [Transport endpoint is not > connected] > [2016-10-31 04:06:03.628475] I [MSGID: 108019] [afr-lk-common.c:1086:afr_lock_blocking] > 0-c_glusterfs-replicate-0: unable to lock on even one child > [2016-10-31 04:06:03.628539] I [MSGID: 108019] [afr-transaction.c:1224:afr_post_blocking_inodelk_cbk] > 0-c_glusterfs-replicate-0: Blocking inodelks failed. > [2016-10-31 04:06:03.628630] W [fuse-bridge.c:1282:fuse_err_cbk] > 0-glusterfs-fuse: 20790: FLUSH() ERR => -1 (Transport endpoint is not > connected) > [2016-10-31 04:06:03.629149] E [rpc-clnt.c:362:saved_frames_unwind] (--> > /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8ab79f58] > (--> /usr/lib64/libgfrpc.so.0(saved_frames_unwind-0x1b7a0)[0x3fff8ab1dc90] > (--> /usr/lib64/libgfrpc.so.0(saved_frames_destroy-0x1b638)[0x3fff8ab1de10] > (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19af8)[0x3fff8ab1fb18] > (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8ab20808] > ))))) 0-c_glusterfs-client-9: forced unwinding frame type(GlusterFS 3.3) > op(LOOKUP(27)) called at 2016-10-31 04:06:03.624346 (xid=0x7f5a) > [2016-10-31 04:06:03.629183] I [rpc-clnt.c:1847:rpc_clnt_reconfig] > 0-c_glusterfs-client-9: changing port to 49391 (from 0) > [2016-10-31 04:06:03.629210] W [MSGID: 114031] [client-rpc-fops.c:2971:client3_3_lookup_cbk] > 0-c_glusterfs-client-9: remote operation failed. Path: /loadmodules_norepl/ > CXC1725605_P93A001/cello/emasviews (b0e5a94e-a432-4dce-b86f-a551555780a2) > [Transport endpoint is not connected] > > > Could you please tell us the reason why we are getting these trace and how > to resolve this. > > Logs are attached here please share your analysis. > > Thanks in advanced > > -- > Regards > Abhishek Paliwal >-- Regards Abhishek Paliwal -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161111/5f3b079f/attachment.html>
Mohammed Rafi K C
2016-Nov-11 06:20 UTC
[Gluster-users] getting "Transport endpoint is not connected" in glusterfs mount log file.
Hi Abhishek, Could you please see if you are bricks are healthy or not, may be you can do a gluster volume status or you can look into the logs. If bricks are not running can you please attach the bricks logs in /var/log/gluster/bricks/ . Rafi KC On 11/11/2016 10:20 AM, ABHISHEK PALIWAL wrote:> Hi, > > Its an urgent case. > > Atleast provide your views on this > > On Wed, Nov 9, 2016 at 11:08 AM, ABHISHEK PALIWAL > <abhishpaliwal at gmail.com <mailto:abhishpaliwal at gmail.com>> wrote: > > Hi, > > We could see that sync is getting failed to sync the GlusterFS > bricks due to error trace "Transport endpoint is not connected " > > [2016-10-31 04:06:03.627395] E [MSGID: 114031] > [client-rpc-fops.c:1673:client3_3_finodelk_cbk] > 0-c_glusterfs-client-9: remote operation failed [Transport > endpoint is not connected] > [2016-10-31 04:06:03.628381] I > [socket.c:3308:socket_submit_request] 0-c_glusterfs-client-9: not > connected (priv->connected = 0) > [2016-10-31 04:06:03.628432] W [rpc-clnt.c:1586:rpc_clnt_submit] > 0-c_glusterfs-client-9: failed to submit rpc-request (XID: 0x7f5f > Program: GlusterFS 3.3, ProgVers: 330, Proc: 30) to rpc-transport > (c_glusterfs-client-9) > [2016-10-31 04:06:03.628466] E [MSGID: 114031] > [client-rpc-fops.c:1673:client3_3_finodelk_cbk] > 0-c_glusterfs-client-9: remote operation failed [Transport > endpoint is not connected] > [2016-10-31 04:06:03.628475] I [MSGID: 108019] > [afr-lk-common.c:1086:afr_lock_blocking] > 0-c_glusterfs-replicate-0: unable to lock on even one child > [2016-10-31 04:06:03.628539] I [MSGID: 108019] > [afr-transaction.c:1224:afr_post_blocking_inodelk_cbk] > 0-c_glusterfs-replicate-0: Blocking inodelks failed. > [2016-10-31 04:06:03.628630] W [fuse-bridge.c:1282:fuse_err_cbk] > 0-glusterfs-fuse: 20790: FLUSH() ERR => -1 (Transport endpoint is > not connected) > [2016-10-31 04:06:03.629149] E > [rpc-clnt.c:362:saved_frames_unwind] (--> > /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8ab79f58] > (--> > /usr/lib64/libgfrpc.so.0(saved_frames_unwind-0x1b7a0)[0x3fff8ab1dc90] > (--> > /usr/lib64/libgfrpc.so.0(saved_frames_destroy-0x1b638)[0x3fff8ab1de10] > (--> > /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19af8)[0x3fff8ab1fb18] > (--> > /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8ab20808] > ))))) 0-c_glusterfs-client-9: forced unwinding frame > type(GlusterFS 3.3) op(LOOKUP(27)) called at 2016-10-31 > 04:06:03.624346 (xid=0x7f5a) > [2016-10-31 04:06:03.629183] I [rpc-clnt.c:1847:rpc_clnt_reconfig] > 0-c_glusterfs-client-9: changing port to 49391 (from 0) > [2016-10-31 04:06:03.629210] W [MSGID: 114031] > [client-rpc-fops.c:2971:client3_3_lookup_cbk] > 0-c_glusterfs-client-9: remote operation failed. Path: > /loadmodules_norepl/CXC1725605_P93A001/cello/emasviews > (b0e5a94e-a432-4dce-b86f-a551555780a2) [Transport endpoint is not > connected] > > > Could you please tell us the reason why we are getting these trace > and how to resolve this. > > Logs are attached here please share your analysis. > > Thanks in advanced > > -- > Regards > Abhishek Paliwal > > > > > -- > > > > > Regards > Abhishek Paliwal > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161111/767c61aa/attachment.html>
Pranith Kumar Karampuri
2016-Nov-11 06:35 UTC
[Gluster-users] getting "Transport endpoint is not connected" in glusterfs mount log file.
As per the logs, the mount is not able to connect to both the bricks. Are the connections fine? On Fri, Nov 11, 2016 at 10:20 AM, ABHISHEK PALIWAL <abhishpaliwal at gmail.com> wrote:> Hi, > > Its an urgent case. > > Atleast provide your views on this > > On Wed, Nov 9, 2016 at 11:08 AM, ABHISHEK PALIWAL <abhishpaliwal at gmail.com > > wrote: > >> Hi, >> >> We could see that sync is getting failed to sync the GlusterFS bricks due >> to error trace "Transport endpoint is not connected " >> >> [2016-10-31 04:06:03.627395] E [MSGID: 114031] >> [client-rpc-fops.c:1673:client3_3_finodelk_cbk] 0-c_glusterfs-client-9: >> remote operation failed [Transport endpoint is not connected] >> [2016-10-31 04:06:03.628381] I [socket.c:3308:socket_submit_request] >> 0-c_glusterfs-client-9: not connected (priv->connected = 0) >> [2016-10-31 04:06:03.628432] W [rpc-clnt.c:1586:rpc_clnt_submit] >> 0-c_glusterfs-client-9: failed to submit rpc-request (XID: 0x7f5f Program: >> GlusterFS 3.3, ProgVers: 330, Proc: 30) to rpc-transport >> (c_glusterfs-client-9) >> [2016-10-31 04:06:03.628466] E [MSGID: 114031] >> [client-rpc-fops.c:1673:client3_3_finodelk_cbk] 0-c_glusterfs-client-9: >> remote operation failed [Transport endpoint is not connected] >> [2016-10-31 04:06:03.628475] I [MSGID: 108019] >> [afr-lk-common.c:1086:afr_lock_blocking] 0-c_glusterfs-replicate-0: >> unable to lock on even one child >> [2016-10-31 04:06:03.628539] I [MSGID: 108019] >> [afr-transaction.c:1224:afr_post_blocking_inodelk_cbk] >> 0-c_glusterfs-replicate-0: Blocking inodelks failed. >> [2016-10-31 04:06:03.628630] W [fuse-bridge.c:1282:fuse_err_cbk] >> 0-glusterfs-fuse: 20790: FLUSH() ERR => -1 (Transport endpoint is not >> connected) >> [2016-10-31 04:06:03.629149] E [rpc-clnt.c:362:saved_frames_unwind] (--> >> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xb5c80)[0x3fff8ab79f58] >> (--> /usr/lib64/libgfrpc.so.0(saved_frames_unwind-0x1b7a0)[0x3fff8ab1dc90] >> (--> /usr/lib64/libgfrpc.so.0(saved_frames_destroy-0x1b638)[0x3fff8ab1de10] >> (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_connection_cleanup-0x19af8)[0x3fff8ab1fb18] >> (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_notify-0x18e68)[0x3fff8ab20808] >> ))))) 0-c_glusterfs-client-9: forced unwinding frame type(GlusterFS 3.3) >> op(LOOKUP(27)) called at 2016-10-31 04:06:03.624346 (xid=0x7f5a) >> [2016-10-31 04:06:03.629183] I [rpc-clnt.c:1847:rpc_clnt_reconfig] >> 0-c_glusterfs-client-9: changing port to 49391 (from 0) >> [2016-10-31 04:06:03.629210] W [MSGID: 114031] >> [client-rpc-fops.c:2971:client3_3_lookup_cbk] 0-c_glusterfs-client-9: >> remote operation failed. Path: /loadmodules_norepl/CXC1725605_P93A001/cello/emasviews >> (b0e5a94e-a432-4dce-b86f-a551555780a2) [Transport endpoint is not >> connected] >> >> >> Could you please tell us the reason why we are getting these trace and >> how to resolve this. >> >> Logs are attached here please share your analysis. >> >> Thanks in advanced >> >> -- >> Regards >> Abhishek Paliwal >> > > > > -- > > > > > Regards > Abhishek Paliwal > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://www.gluster.org/mailman/listinfo/gluster-users >-- Pranith -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20161111/eae5ec81/attachment.html>