thr3ads.net - Gluster users - [Gluster-users] Failed to get fd context for a non-anonymous fd [Dec 2018]

If this information is useful, please help other people find it:
Share via:

max.degraaf at kpn.com

2018-Dec-14 06:06 UTC

[Gluster-users] Failed to get fd context for a non-anonymous fd

Hi,

We updated our glusterfs servers and clients from 3.10 to 4.1.6. Although all
seems to be working just fine we see errors like these on almost all glusterfs
servers:

[2018-12-14 02:35:03.524378] I [addr.c:55:compare_addr_and_update]
0-/apps/glusterfs-gst: allowed = "*", received addr =
"10.159.240.195"
[2018-12-14 02:35:03.524410] I [login.c:111:gf_auth] 0-auth/login: allowed user
names: 9d79471c-ce22-4e48-ba0d-81f297ed5ba1
[2018-12-14 02:35:03.524419] I [MSGID: 115029]
[server-handshake.c:495:server_setvolume] 0-gst-server: accepted client from
CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
(version: 4.1.6)
[2018-12-14 02:35:03.572494] I [MSGID: 115036] [server.c:483:server_rpc_notify]
0-gst-server: disconnecting connection from
CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
[2018-12-14 02:35:03.572549] I [MSGID: 115013]
[server-helpers.c:286:do_fd_cleanup] 0-gst-server: fd cleanup on
<gfid:2aed88da-0664-4070-9202-b856eb1e3c96>
[2018-12-14 02:35:03.572697] I [socket.c:3844:socket_submit_reply]
0-tcp.gst-server: not connected (priv->connected = -1)
[2018-12-14 02:35:03.572718] E [rpcsvc.c:1378:rpcsvc_submit_generic]
0-rpc-service: failed to submit message (XID: 0x18, Program: GlusterFS 3.3,
ProgVers: 330, Proc: 42) to rpc-transport (tcp.gst-server)
[2018-12-14 02:35:03.572807] E [server.c:137:server_submit_reply]
(-->/lib64/libgfrpc.so.0(+0x817e) [0x7f022c85717e]
-->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0x335b9)
[0x7f02179665b9]
-->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0xae2e)
[0x7f021793de2e] ) 0-: Reply submission failed
[2018-12-14 02:35:03.572823] E [server.c:144:server_submit_reply]
(-->/lib64/libgfrpc.so.0(+0x817e) [0x7f022c85717e]
-->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0x335b9)
[0x7f02179665b9]
-->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0xae96)
[0x7f021793de96] ) 0-: Reply submission failed
[2018-12-14 02:35:03.572877] E [MSGID: 113040]
[posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context
for a non-anonymous fd, file: (null), gfid: 2aed88da-0664-4070-9202-b856eb1e3c96
[2018-12-14 02:35:03.572889] W [MSGID: 113006]
[posix-inode-fd-ops.c:2018:posix_flush] 0-gst-posix: pfd is NULL on
fd=0x7f01e80092c8 [Invalid argument]
[2018-12-14 02:35:03.573034] I [MSGID: 101055] [client_t.c:444:gf_client_unref]
0-gst-server: Shutting down connection
CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
[2018-12-14 02:35:33.750415] I [addr.c:55:compare_addr_and_update]
0-/apps/glusterfs-gst: allowed = "*", received addr =
"10.159.240.227"
[2018-12-14 02:35:33.750439] I [login.c:111:gf_auth] 0-auth/login: allowed user
names: 9d79471c-ce22-4e48-ba0d-81f297ed5ba1
[2018-12-14 02:35:33.750447] I [MSGID: 115029]
[server-handshake.c:495:server_setvolume] 0-gst-server: accepted client from
CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
(version: 4.1.6)
[2018-12-14 02:35:33.817381] I [MSGID: 115036] [server.c:483:server_rpc_notify]
0-gst-server: disconnecting connection from
CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
[2018-12-14 02:35:33.818343] I [MSGID: 101055] [client_t.c:444:gf_client_unref]
0-gst-server: Shutting down connection
CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0

We have 3 separate environments and all 3 have a similar setup with 2 glusterfs
servers with 6 to 10 glusterfs clients connected. No heavy load at all, all
running on CentOS 7.1 en 7.3.

Any thoughts?

Thanks,

Max

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20181214/48b0de26/attachment.html>

Raghavendra Gowdappa

2018-Dec-14 06:31 UTC

head link

[Gluster-users] Failed to get fd context for a non-anonymous fd

On Fri, Dec 14, 2018 at 11:49 AM <max.degraaf at kpn.com> wrote:
> Hi,
>
>
>
> We updated our glusterfs servers and clients from 3.10 to 4.1.6. Although
> all seems to be working just fine we see errors like these on almost all
> glusterfs servers:
>
>
>
> [2018-12-14 02:35:03.524378] I [addr.c:55:compare_addr_and_update]
> 0-/apps/glusterfs-gst: allowed = "*", received addr =
"10.159.240.195"
>
> [2018-12-14 02:35:03.524410] I [login.c:111:gf_auth] 0-auth/login: allowed
> user names: 9d79471c-ce22-4e48-ba0d-81f297ed5ba1
>
> [2018-12-14 02:35:03.524419] I [MSGID: 115029]
> [server-handshake.c:495:server_setvolume] 0-gst-server: accepted client
> from
>
CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
> (version: 4.1.6)
>
> [2018-12-14 02:35:03.572494] I [MSGID: 115036]
> [server.c:483:server_rpc_notify] 0-gst-server: disconnecting connection
> from
>
CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
>
> [2018-12-14 02:35:03.572549] I [MSGID: 115013]
> [server-helpers.c:286:do_fd_cleanup] 0-gst-server: fd cleanup on
> <gfid:2aed88da-0664-4070-9202-b856eb1e3c96>
>
A cleanup was initiated for this fd as client got disconnected.

> [2018-12-14 02:35:03.572697] I [socket.c:3844:socket_submit_reply]
> 0-tcp.gst-server: not connected (priv->connected = -1)
>
> [2018-12-14 02:35:03.572718] E [rpcsvc.c:1378:rpcsvc_submit_generic]
> 0-rpc-service: failed to submit message (XID: 0x18, Program: GlusterFS 3.3,
> ProgVers: 330, Proc: 42) to rpc-transport (tcp.gst-server)
>
> [2018-12-14 02:35:03.572807] E [server.c:137:server_submit_reply]
> (-->/lib64/libgfrpc.so.0(+0x817e) [0x7f022c85717e]
> -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0x335b9)
> [0x7f02179665b9]
> -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0xae2e)
> [0x7f021793de2e] ) 0-: Reply submission failed
>
> [2018-12-14 02:35:03.572823] E [server.c:144:server_submit_reply]
> (-->/lib64/libgfrpc.so.0(+0x817e) [0x7f022c85717e]
> -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0x335b9)
> [0x7f02179665b9]
> -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0xae96)
> [0x7f021793de96] ) 0-: Reply submission failed
>
> [2018-12-14 02:35:03.572877] E [MSGID: 113040]
> [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd
> context for a non-anonymous fd, file: (null), gfid:
> 2aed88da-0664-4070-9202-b856eb1e3c96
>
This, most likely is an operation racing with the cleanup initiated. Since
this condition is triggered by a disconnect, this looks to be benign,
unless you've seen any functional issues at application layer.

> [2018-12-14 02:35:03.572889] W [MSGID: 113006]
> [posix-inode-fd-ops.c:2018:posix_flush] 0-gst-posix: pfd is NULL on
> fd=0x7f01e80092c8 [Invalid argument]
>
> [2018-12-14 02:35:03.573034] I [MSGID: 101055]
> [client_t.c:444:gf_client_unref] 0-gst-server: Shutting down connection
>
CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
>
> [2018-12-14 02:35:33.750415] I [addr.c:55:compare_addr_and_update]
> 0-/apps/glusterfs-gst: allowed = "*", received addr =
"10.159.240.227"
>
> [2018-12-14 02:35:33.750439] I [login.c:111:gf_auth] 0-auth/login: allowed
> user names: 9d79471c-ce22-4e48-ba0d-81f297ed5ba1
>
> [2018-12-14 02:35:33.750447] I [MSGID: 115029]
> [server-handshake.c:495:server_setvolume] 0-gst-server: accepted client
> from
>
CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
> (version: 4.1.6)
>
> [2018-12-14 02:35:33.817381] I [MSGID: 115036]
> [server.c:483:server_rpc_notify] 0-gst-server: disconnecting connection
> from
>
CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
>
> [2018-12-14 02:35:33.818343] I [MSGID: 101055]
> [client_t.c:444:gf_client_unref] 0-gst-server: Shutting down connection
>
CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
>
>
>
> We have 3 separate environments and all 3 have a similar setup with 2
> glusterfs servers with 6 to 10 glusterfs clients connected. No heavy load
> at all, all running on CentOS 7.1 en 7.3.
>
>
>
> Any thoughts?
>
>
>
> Thanks,
>
>
>
> Max
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20181214/41cb022f/attachment.html>

Gluster users - Dec 2018 - Failed to get fd context for a non-anonymous fd

[Gluster-users] Failed to get fd context for a non-anonymous fd

[Gluster-users] Failed to get fd context for a non-anonymous fd