tommy.yardley at baesystems.com
2016-Jul-20 12:00 UTC
[Gluster-users] Issues with SSL connections
Ah great ? good to see that it will be fixed soon. Is this also what is causing the issues with NFS? Thanks, Tommy From: Atin Mukherjee [mailto:amukherj at redhat.com] Sent: 20 July 2016 12:24 To: Yardley, Tommy (UK Guildford); Mohit Agrawal Cc: gluster-users at gluster.org Subject: Re: [Gluster-users] Issues with SSL connections + Mohit We are aware of this issue and there is a RHBZ [1] filed for it. We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not 3.7.14. [1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215 Thanks, Atin On Wed, Jul 20, 2016 at 4:16 PM, tommy.yardley at baesystems.com<mailto:tommy.yardley at baesystems.com> <tommy.yardley at baesystems.com<mailto:tommy.yardley at baesystems.com>> wrote: Hi all, I posted earlier concerning rolling back on the version of gluster we are using. After being successful in doing so I?ve come to realise the issue we were facing may not have been introduced by the latest versions of gluster but are instead an issue with configuration. We haven?t changed how we were installing and setting up glusterFS but recently we are having issues with SSL, NFS and in turn the log file filling up the root partition. I?ve attached some relevant extracts from logs below of the issue we are facing (some may not be relevant) ? I haven?t been able to find anything in the mailing list that solves the issue. From glusterd log: ``` [2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600 [2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL support for glusterd is ENABLED [2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed to open /etc/ssl/dhparam.pem, DH ciphers are disabled ``` ``` [2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv] 0-socket.management: writev on 127.0.0.1:65510<http://127.0.0.1:65510> failed (No data available) [2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller] 0-socket.management: poll error on socket ``` ``` [2016-07-20 10:25:31.777638] I [MSGID: 106502] [glusterd-handler.c:2821:__glusterd_handle_friend_update] 0-management: Received my uuid as Friend [2016-07-20 10:25:31.777925] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd. [2016-07-20 10:25:31.777974] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: glustershd has disconnected from glusterd. [2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop ``` And then the following streams and fills up the log file ~GBs in the matter of hours ``` [2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop [2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error [2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number [2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller] 0-socket.management: server setup failed [2016-07-20 10:25:34.217120] I [MSGID: 106006] [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs has disconnected from glusterd. [2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-227 [2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop [2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-165 [2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop [2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-65 [2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop [2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection] 0-socket.management: peer CN = ip-172-31-169-59 [2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller] 0-socket.management: error in polling loop [2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection] 0-socket.management: SSL connect error [2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack] 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number ``` I have also attached nfs.log and glustershd.log Thanks in advance for all the help! Tommy Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems Applied Intelligence Limited, details of which can be found at http://www.baesystems.com/Businesses/index.htm. _______________________________________________ Gluster-users mailing list Gluster-users at gluster.org<mailto:Gluster-users at gluster.org> http://www.gluster.org/mailman/listinfo/gluster-users -- --Atin Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems Applied Intelligence Limited, details of which can be found at http://www.baesystems.com/Businesses/index.htm. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160720/f394045d/attachment.html>
On Wednesday 20 July 2016, tommy.yardley at baesystems.com < tommy.yardley at baesystems.com> wrote:> Ah great ? good to see that it will be fixed soon. > > > > Is this also what is causing the issues with NFS? > >Could be, Mohit will get back.> > > Thanks, > > Tommy > > > > *From:* Atin Mukherjee [mailto:amukherj at redhat.com > <javascript:_e(%7B%7D,'cvml','amukherj at redhat.com');>] > *Sent:* 20 July 2016 12:24 > *To:* Yardley, Tommy (UK Guildford); Mohit Agrawal > *Cc:* gluster-users at gluster.org > <javascript:_e(%7B%7D,'cvml','gluster-users at gluster.org');> > *Subject:* Re: [Gluster-users] Issues with SSL connections > > > > + Mohit > > We are aware of this issue and there is a RHBZ [1] filed for it. > > We have a plan to fix that in coming 3.7.x release, probably 3.7.15 if not > 3.7.14. > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1309215 > > Thanks, > > Atin > > > > On Wed, Jul 20, 2016 at 4:16 PM, tommy.yardley at baesystems.com > <javascript:_e(%7B%7D,'cvml','tommy.yardley at baesystems.com');> < > tommy.yardley at baesystems.com > <javascript:_e(%7B%7D,'cvml','tommy.yardley at baesystems.com');>> wrote: > > Hi all, > > > > I posted earlier concerning rolling back on the version of gluster we are > using. > > > > After being successful in doing so I?ve come to realise the issue we were > facing may not have been introduced by the latest versions of gluster but > are instead an issue with configuration. > > > > We haven?t changed how we were installing and setting up glusterFS but > recently we are having issues with SSL, NFS and in turn the log file > filling up the root partition. > > > > I?ve attached some relevant extracts from logs below of the issue we are > facing (some may not be relevant) ? I haven?t been able to find anything in > the mailing list that solves the issue. > > > > > > From glusterd log: > > > > ``` > > [2016-07-20 10:25:31.700017] I [rpc-clnt.c:1004:rpc_clnt_connection_init] > 0-snapd: setting frame-timeout to 600 > > [2016-07-20 10:25:31.700076] I [socket.c:3927:socket_init] 0-snapd: SSL > support for glusterd is ENABLED > > [2016-07-20 10:25:31.700191] E [socket.c:4005:socket_init] 0-snapd: failed > to open /etc/ssl/dhparam.pem, DH ciphers are disabled > > ``` > > ``` > > [2016-07-20 10:25:31.742672] W [socket.c:589:__socket_rwv] > 0-socket.management: writev on 127.0.0.1:65510 failed (No data available) > > [2016-07-20 10:25:31.742715] E [socket.c:2497:socket_poller] > 0-socket.management: poll error on socket > > ``` > > ``` > > [2016-07-20 10:25:31.777638] I [MSGID: 106502] > [glusterd-handler.c:2821:__glusterd_handle_friend_update] 0-management: > Received my uuid as Friend > > [2016-07-20 10:25:31.777925] I [MSGID: 106006] > [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs > has disconnected from glusterd. > > [2016-07-20 10:25:31.777974] I [MSGID: 106006] > [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: > glustershd has disconnected from glusterd. > > [2016-07-20 10:25:31.778527] E [socket.c:2502:socket_poller] > 0-socket.management: error in polling loop > > ``` > > And then the following streams and fills up the log file ~GBs in the > matter of hours > > ``` > > [2016-07-20 10:25:31.865991] E [socket.c:2502:socket_poller] > 0-socket.management: error in polling loop > > [2016-07-20 10:25:32.670678] E [socket.c:352:ssl_setup_connection] > 0-socket.management: SSL connect error > > [2016-07-20 10:25:32.670726] E [socket.c:206:ssl_dump_error_stack] > 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong > version number > > [2016-07-20 10:25:32.670744] E [socket.c:2389:socket_poller] > 0-socket.management: server setup failed > > [2016-07-20 10:25:34.217120] I [MSGID: 106006] > [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify] 0-management: nfs > has disconnected from glusterd. > > [2016-07-20 10:25:34.301599] I [socket.c:347:ssl_setup_connection] > 0-socket.management: peer CN = ip-172-31-169-227 > > [2016-07-20 10:25:34.303179] E [socket.c:2502:socket_poller] > 0-socket.management: error in polling loop > > [2016-07-20 10:25:34.378333] I [socket.c:347:ssl_setup_connection] > 0-socket.management: peer CN = ip-172-31-169-165 > > [2016-07-20 10:25:34.379965] E [socket.c:2502:socket_poller] > 0-socket.management: error in polling loop > > [2016-07-20 10:25:35.389754] I [socket.c:347:ssl_setup_connection] > 0-socket.management: peer CN = ip-172-31-169-65 > > [2016-07-20 10:25:35.390996] E [socket.c:2502:socket_poller] > 0-socket.management: error in polling loop > > [2016-07-20 10:25:35.642357] I [socket.c:347:ssl_setup_connection] > 0-socket.management: peer CN = ip-172-31-169-59 > > [2016-07-20 10:25:35.642815] E [socket.c:2502:socket_poller] > 0-socket.management: error in polling loop > > [2016-07-20 10:25:35.671336] E [socket.c:352:ssl_setup_connection] > 0-socket.management: SSL connect error > > [2016-07-20 10:25:35.671376] E [socket.c:206:ssl_dump_error_stack] > 0-socket.management: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong > version number > > ``` > > > > I have also attached nfs.log and glustershd.log > > > > Thanks in advance for all the help! > > Tommy > > Please consider the environment before printing this email. This message > should be regarded as confidential. If you have received this email in > error please notify the sender and destroy it immediately. Statements of > intent shall only become binding when confirmed in hard copy by an > authorised signatory. The contents of this email may relate to dealings > with other companies under the control of BAE Systems Applied Intelligence > Limited, details of which can be found at > http://www.baesystems.com/Businesses/index.htm. > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > <javascript:_e(%7B%7D,'cvml','Gluster-users at gluster.org');> > http://www.gluster.org/mailman/listinfo/gluster-users > > > > > -- > > > > --Atin > Please consider the environment before printing this email. This message > should be regarded as confidential. If you have received this email in > error please notify the sender and destroy it immediately. Statements of > intent shall only become binding when confirmed in hard copy by an > authorised signatory. The contents of this email may relate to dealings > with other companies under the control of BAE Systems Applied Intelligence > Limited, details of which can be found at > http://www.baesystems.com/Businesses/index.htm. >-- Atin Sent from iPhone -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160720/a620c167/attachment.html>