thr3ads.net - Gluster users - [Gluster-users] Issues with SSL connections [Jul 2016]

If this information is useful, please help other people find it:
Share via:

tommy.yardley at baesystems.com

2016-Jul-20 10:46 UTC

[Gluster-users] Issues with SSL connections

Hi all,

I posted earlier concerning rolling back on the version of gluster we are using.

After being successful in doing so I've come to realise the issue we were
facing may not have been introduced by the latest versions of gluster but are
instead an issue with configuration.

We haven't changed how we were installing and setting up glusterFS but
recently we are having issues with SSL, NFS and in turn the log file filling up
the root partition.

I've attached some relevant extracts from logs below of the issue we are
facing (some may not be relevant) - I haven't been able to find anything in
the mailing list that solves the issue.

>From glusterd log:
```
[2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init]
0-snapd: setting frame-timeout to 600
[2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL support
for glusterd is ENABLED
[2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed to
open /etc/ssl/dhparam.pem, DH ciphers are disabled
```
```
[2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv] 0-socket.management:
writev on 127.0.0.1:65510 failed (No data available)
[2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller]
0-socket.management: poll error on socket
```
```
[2016-07-20 10:25:31.777638] I [MSGID: 106502]
[glusterd-handler.c:2821:__glusterd_handle_friend_update] 0-management: Received
my uuid as Friend
[2016-07-20 10:25:31.777925] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has
disconnected from glusterd.
[2016-07-20 10:25:31.777974] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management:
glustershd has disconnected from glusterd.
[2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
```
And then the following streams and fills up the log file ~GBs in the matter of
hours
```
[2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack]
0-socket.management:   error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version
number
[2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller]
0-socket.management: server setup failed
[2016-07-20 10:25:34.217120] I [MSGID: 106006]
[glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has
disconnected from glusterd.
[2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-227
[2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-165
[2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-65
[2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection]
0-socket.management: peer CN = ip-172-31-169-59
[2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller]
0-socket.management: error in polling loop
[2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection]
0-socket.management: SSL connect error
[2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack]
0-socket.management:   error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version
number
```

I have also attached nfs.log and glustershd.log

Thanks in advance for all the help!
Tommy
Please consider the environment before printing this email. This message should
be regarded as confidential. If you have received this email in error please
notify the sender and destroy it immediately. Statements of intent shall only
become binding when confirmed in hard copy by an authorised signatory. The
contents of this email may relate to dealings with other companies under the
control of BAE Systems Applied Intelligence Limited, details of which can be
found at http://www.baesystems.com/Businesses/index.htm.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160720/99082959/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nfs.log
Type: application/octet-stream
Size: 11071 bytes
Desc: nfs.log
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160720/99082959/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: glustershd.log
Type: application/octet-stream
Size: 35686 bytes
Desc: glustershd.log
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160720/99082959/attachment-0001.obj>

Atin Mukherjee

2016-Jul-20 11:24 UTC

head link

[Gluster-users] Issues with SSL connections

+ Mohit

We are aware of this issue and there is a RHBZ [1] filed for it.

We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not
3.7.14.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215

Thanks,
Atin

On Wed, Jul 20, 2016 at 4:16 PM, tommy.yardley at baesystems.com <
tommy.yardley at baesystems.com> wrote:
> Hi all,
>
>
>
> I posted earlier concerning rolling back on the version of gluster we are
> using.
>
>
>
> After being successful in doing so I?ve come to realise the issue we were
> facing may not have been introduced by the latest versions of gluster but
> are instead an issue with configuration.
>
>
>
> We haven?t changed how we were installing and setting up glusterFS but
> recently we are having issues with SSL, NFS and in turn the log file
> filling up the root partition.
>
>
>
> I?ve attached some relevant extracts from logs below of the issue we are
> facing (some may not be relevant) ? I haven?t been able to find anything in
> the mailing list that solves the issue.
>
>
>
>
>
> From glusterd log:
>
>
>
> ```
>
> [2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init]
> 0-snapd: setting frame-timeout to 600
>
> [2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL
> support for glusterd is ENABLED
>
> [2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed
> to open /etc/ssl/dhparam.pem, DH ciphers are disabled
>
> ```
>
> ```
>
> [2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv]
> 0-socket.management: writev on 127.0.0.1:65510 failed (No data available)
>
> [2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller]
> 0-socket.management: poll error on socket
>
> ```
>
> ```
>
> [2016-07-20 10:25:31.777638] I [MSGID: 106502]
> [glusterd-handler.c:2821:__glusterd_handle_friend_update] 0-management:
> Received my uuid as Friend
>
> [2016-07-20 10:25:31.777925] I [MSGID: 106006]
> [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
> has disconnected from glusterd.
>
> [2016-07-20 10:25:31.777974] I [MSGID: 106006]
> [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management:
> glustershd has disconnected from glusterd.
>
> [2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller]
> 0-socket.management: error in polling loop
>
> ```
>
> And then the following streams and fills up the log file ~GBs in the
> matter of hours
>
> ```
>
> [2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller]
> 0-socket.management: error in polling loop
>
> [2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection]
> 0-socket.management: SSL connect error
>
> [2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack]
> 0-socket.management:   error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
> version number
>
> [2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller]
> 0-socket.management: server setup failed
>
> [2016-07-20 10:25:34.217120] I [MSGID: 106006]
> [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs
> has disconnected from glusterd.
>
> [2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection]
> 0-socket.management: peer CN = ip-172-31-169-227
>
> [2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller]
> 0-socket.management: error in polling loop
>
> [2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection]
> 0-socket.management: peer CN = ip-172-31-169-165
>
> [2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller]
> 0-socket.management: error in polling loop
>
> [2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection]
> 0-socket.management: peer CN = ip-172-31-169-65
>
> [2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller]
> 0-socket.management: error in polling loop
>
> [2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection]
> 0-socket.management: peer CN = ip-172-31-169-59
>
> [2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller]
> 0-socket.management: error in polling loop
>
> [2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection]
> 0-socket.management: SSL connect error
>
> [2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack]
> 0-socket.management:   error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong
> version number
>
> ```
>
>
>
> I have also attached nfs.log and glustershd.log
>
>
>
> Thanks in advance for all the help!
>
> Tommy
> Please consider the environment before printing this email. This message
> should be regarded as confidential. If you have received this email in
> error please notify the sender and destroy it immediately. Statements of
> intent shall only become binding when confirmed in hard copy by an
> authorised signatory. The contents of this email may relate to dealings
> with other companies under the control of BAE Systems Applied Intelligence
> Limited, details of which can be found at
> http://www.baesystems.com/Businesses/index.htm.
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>


-- 

--Atin
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20160720/02ca4039/attachment.html>

Gluster users - Jul 2016 - Issues with SSL connections

[Gluster-users] Issues with SSL connections

[Gluster-users] Issues with SSL connections