CentOS - May 2015 - Could not complete SSL handshake to Amazon EC2 host

Hello,

 I am trying to monitor a host in the Amazon EC2 cloud.

Yet when I try to check NRPE from the monitoring host I am getting an SSL
handshake error:

[root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H ops.jokefire.com
CHECK_NRPE: Error - Could not complete SSL handshake.

And if I telnet into the host on port 5666 to see if the FW port is open,
the connection closes right away:

[root at monitor1:~] #telnet ops.somewhere.com 5666
Trying 54.225.218.125...
Connected to ops.somewhere.com.
Escape character is '^]'.
Connection closed by foreign host.

You can see there it connects, but then it closes immediately after the
connection.

 I have NRPE running on the host I want to monitor:

[root at ops:~] #lsof -i :5666
COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
xinetd  1434 root    5u  IPv4   4063       TCP *:nrpe (LISTEN)

And I have the IP of my nagios server listed in the xinetd conf file:

[root at ops:~] #cat /etc/xinetd.d/nrpe
# default: on
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
        flags           = REUSE
        socket_type     = stream
        port            = 5666
        wait            = no
        user            = nagios
        group           = nagios
        server          = /usr/local/nagios/bin/nrpe
        server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
        log_on_failure  += USERID
        disable         = no
        only_from       = 127.0.0.1 xx.xx.xx.xx   # <- representing my real
nagios server IP
}



And I have my default security group for that host open on port 5666 to the
world for this experiment.  I plan on locking that down again to the single
IP of my monitoring host once I get this resolved.

Does anyone have any suggestions on how I can get that problem solved?

Thanks,
Tim

-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

Hi
Does the deamon run under xinetd? Then  you have to configure the only_from
in  */etc/**xinetd.d**/**nrpe* to.

Regards
Eric
Am 01.05.2015 06:46 schrieb "Tim Dunphy" <bluethundr at
gmail.com>:
> Hello,
>
>  I am trying to monitor a host in the Amazon EC2 cloud.
>
> Yet when I try to check NRPE from the monitoring host I am getting an SSL
> handshake error:
>
> [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
> ops.jokefire.com
> CHECK_NRPE: Error - Could not complete SSL handshake.
>
> And if I telnet into the host on port 5666 to see if the FW port is open,
> the connection closes right away:
>
> [root at monitor1:~] #telnet ops.somewhere.com 5666
> Trying 54.225.218.125...
> Connected to ops.somewhere.com.
> Escape character is '^]'.
> Connection closed by foreign host.
>
> You can see there it connects, but then it closes immediately after the
> connection.
>
>  I have NRPE running on the host I want to monitor:
>
> [root at ops:~] #lsof -i :5666
> COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
> xinetd  1434 root    5u  IPv4   4063       TCP *:nrpe (LISTEN)
>
> And I have the IP of my nagios server listed in the xinetd conf file:
>
> [root at ops:~] #cat /etc/xinetd.d/nrpe
> # default: on
> # description: NRPE (Nagios Remote Plugin Executor)
> service nrpe
> {
>         flags           = REUSE
>         socket_type     = stream
>         port            = 5666
>         wait            = no
>         user            = nagios
>         group           = nagios
>         server          = /usr/local/nagios/bin/nrpe
>         server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
>         log_on_failure  += USERID
>         disable         = no
>         only_from       = 127.0.0.1 xx.xx.xx.xx   # <- representing my
real
> nagios server IP
> }
>
>
>
> And I have my default security group for that host open on port 5666 to the
> world for this experiment.  I plan on locking that down again to the single
> IP of my monitoring host once I get this resolved.
>
> Does anyone have any suggestions on how I can get that problem solved?
>
> Thanks,
> Tim
>
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>

Hi Eric,

 Thanks for your reply. I do have nrpe running under xinetd on the host I'm
trying to monitor.

 And running the nrpe checl locally:

[root at ops:~] #/usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.15

[root at ops:~] #grep only_from /etc/xinetd.d/nrpe
        only_from       = 127.0.0.1 216.120.248.126

And I do have port 5666 open on the security group for this host.

And I made sure the local firewall was stopped, because I am blocking ports
with the security groups instead.

[root at ops:~] #service iptables status
Firewall is stopped.

It's only when checking from the monitoring host that nrpe fails:

[root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H ops.jokefire.com
CHECK_NRPE: Error - Could not complete SSL handshake.

Really, really puzzling. This is driving me up a wall!! I hopeI can solve
this soon....

Thanks for any and all help with this one!!
Tim

On Fri, May 1, 2015 at 1:02 AM, Eric Lehmann <e.lehmann88 at gmail.com>
wrote:
> Hi
> Does the deamon run under xinetd? Then  you have to configure the only_from
> in  */etc/**xinetd.d**/**nrpe* to.
>
> Regards
> Eric
> Am 01.05.2015 06:46 schrieb "Tim Dunphy" <bluethundr at
gmail.com>:
>
> > Hello,
> >
> >  I am trying to monitor a host in the Amazon EC2 cloud.
> >
> > Yet when I try to check NRPE from the monitoring host I am getting an
SSL
> > handshake error:
> >
> > [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
> > ops.jokefire.com
> > CHECK_NRPE: Error - Could not complete SSL handshake.
> >
> > And if I telnet into the host on port 5666 to see if the FW port is
open,
> > the connection closes right away:
> >
> > [root at monitor1:~] #telnet ops.somewhere.com 5666
> > Trying 54.225.218.125...
> > Connected to ops.somewhere.com.
> > Escape character is '^]'.
> > Connection closed by foreign host.
> >
> > You can see there it connects, but then it closes immediately after
the
> > connection.
> >
> >  I have NRPE running on the host I want to monitor:
> >
> > [root at ops:~] #lsof -i :5666
> > COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
> > xinetd  1434 root    5u  IPv4   4063       TCP *:nrpe (LISTEN)
> >
> > And I have the IP of my nagios server listed in the xinetd conf file:
> >
> > [root at ops:~] #cat /etc/xinetd.d/nrpe
> > # default: on
> > # description: NRPE (Nagios Remote Plugin Executor)
> > service nrpe
> > {
> >         flags           = REUSE
> >         socket_type     = stream
> >         port            = 5666
> >         wait            = no
> >         user            = nagios
> >         group           = nagios
> >         server          = /usr/local/nagios/bin/nrpe
> >         server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
> >         log_on_failure  += USERID
> >         disable         = no
> >         only_from       = 127.0.0.1 xx.xx.xx.xx   # <- representing
my
> real
> > nagios server IP
> > }
> >
> >
> >
> > And I have my default security group for that host open on port 5666
to
> the
> > world for this experiment.  I plan on locking that down again to the
> single
> > IP of my monitoring host once I get this resolved.
> >
> > Does anyone have any suggestions on how I can get that problem solved?
> >
> > Thanks,
> > Tim
> >
> > --
> > GPG me!!
> >
> > gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
> > _______________________________________________
> > CentOS mailing list
> > CentOS at centos.org
> > http://lists.centos.org/mailman/listinfo/centos
> >
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>


-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

On Fri, 2015-05-01 at 00:46 -0400, Tim Dunphy wrote:
> [root at ops:~] #cat /etc/xinetd.d/nrpe
> # default: on
> # description: NRPE (Nagios Remote Plugin Executor)
> service nrpe
> {
>         flags           = REUSE
>         socket_type     = stream
>         port            = 5666
>         wait            = no
>         user            = nagios
>         group           = nagios
>         server          = /usr/local/nagios/bin/nrpe
>         server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
>         log_on_failure  += USERID
>         disable         = no
>         only_from       = 127.0.0.1 xx.xx.xx.xx   # <- representing my
real
> nagios server IP
> }
> 
> Does anyone have any suggestions on how I can get that problem solved?
> 
> Thanks,
> Tim
> 
Does /usr/local/nagios/etc/nrpe.cfg exist and is it readable by user or
group 'nagios'?  Did the user:group 'nagios' get created when
you did
the installation?  Those were my two routine stumbles before I automated
rollouts.

Regards,

Yes, also it could be nagios use another configs location. Check: whereis
nagios.
Am 01.05.2015 13:44 schrieb "Brian Miller" <centos at
fullnote.com>:
> On Fri, 2015-05-01 at 00:46 -0400, Tim Dunphy wrote:
>
> > [root at ops:~] #cat /etc/xinetd.d/nrpe
> > # default: on
> > # description: NRPE (Nagios Remote Plugin Executor)
> > service nrpe
> > {
> >         flags           = REUSE
> >         socket_type     = stream
> >         port            = 5666
> >         wait            = no
> >         user            = nagios
> >         group           = nagios
> >         server          = /usr/local/nagios/bin/nrpe
> >         server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
> >         log_on_failure  += USERID
> >         disable         = no
> >         only_from       = 127.0.0.1 xx.xx.xx.xx   # <- representing
my
> real
> > nagios server IP
> > }
>
> >
> > Does anyone have any suggestions on how I can get that problem solved?
> >
> > Thanks,
> > Tim
> >
>
> Does /usr/local/nagios/etc/nrpe.cfg exist and is it readable by user or
> group 'nagios'?  Did the user:group 'nagios' get created
when you did
> the installation?  Those were my two routine stumbles before I automated
> rollouts.
>
> Regards,
>
>
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>

>
> >And I made sure the local firewall was stopped, because I am blocking
> ports
> >with the security groups instead.
> As an aside, I wouldn't do this unless running in a VPC as there are
> other hosts in the general cloud and many are malicious.

Hmmm... you make an excellent point! I picked up this habit from an AWS
shop I used to work at. But what you just said will make me reconsider!

>
> >It's only when checking from the monitoring host that nrpe fails:
> Check /var/log/messages to see if xinetd says anything.

I tailed /var/log/messages while hitting the client with check_nrpe from
the monitoring host. However, that didn't cause an entry in the messages
log.

> Also nrpe needs
> to be told from where connections are allowed whether running under an
> inetd or self-daemonized.

Yep! I've set the only_from to have only the loopback address and the IP
for the monitoring host in /etc/xinetd.d/npre.

>
> Also check the NRPE reviews on exchange.nagios.org, where the issue is
> discussed.

Cool! Thanks. I'll check it out, and see if I can find anything useful.

I appreciate the input!

Also I really appreciate the ongoing dialog with the community on this
issue. I'm grasping at straws at this point. And all the attempts at help
have been really great! I hope we can still get to the bottom of this!

Tim

On Sat, May 2, 2015 at 11:45 AM, Mark Milhollan <mlm at pixelgate.net>
wrote:
> On Fri, 1 May 2015, Tim Dunphy wrote:
>
> >And I made sure the local firewall was stopped, because I am blocking
> ports
> >with the security groups instead.
>
> As an aside, I wouldn't do this unless running in a VPC as there are
> other hosts in the general cloud and many are malicious.
>
> >It's only when checking from the monitoring host that nrpe fails:
>
> Check /var/log/messages to see if xinetd says anything.  Also nrpe needs
> to be told from where connections are allowed whether running under an
> inetd or self-daemonized.
>
> Also check the NRPE reviews on exchange.nagios.org, where the issue is
> discussed.
>
>
> /mark
>


-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

>
> Not just /var/log/messages.  Doesn't nrpe have a log file?  Maybe even
> secure.

Hmmm I don't find any log specific to nrpe. In other words I don't see
/var/log/nrpe.log or whatever. :)

And when I tail -f /var/log/secure or /var/log/messages I don't see any
entries turning up in them when I hit the client with check_nrpe. I was
checking the logs on the client itself.

>
> >> Also nrpe needs to be told from where connections are allowed
whether
> >> running under an inetd or self-daemonized.
> >
> >Yep! I've set the only_from to have only the loopback address and
the IP
> >for the monitoring host in /etc/xinetd.d/npre.

>
> Not the xinetd config, the nrpe config (too).

Hmmmm. but the nrpe.confg file is ignored in the case of allowed
hosts.
>From the nrpe config:
# NOTE: This option is ignored if NRPE is running under either inetd or
xinetd

allowed_hosts=127.0.0.1

Thanks for the input tho, I genuinely appreciate it!

On Sat, May 2, 2015 at 4:05 PM, Mark Milhollan <mlm at pixelgate.net>
wrote:
> On Sat, 2 May 2015, Tim Dunphy wrote:
>
> >>>It's only when checking from the monitoring host that nrpe
fails:
> >>Check /var/log/messages to see if xinetd says anything.
> >
> >I tailed /var/log/messages while hitting the client with check_nrpe
from
> >the monitoring host. However, that didn't cause an entry in the
messages
> >log.
>
> Not just /var/log/messages.  Doesn't nrpe have a log file?  Maybe even
> secure.
>
> >> Also nrpe needs to be told from where connections are allowed
whether
> >> running under an inetd or self-daemonized.
> >
> >Yep! I've set the only_from to have only the loopback address and
the IP
> >for the monitoring host in /etc/xinetd.d/npre.
>
> Not the xinetd config, the nrpe config (too).
>
>
> /mark
>


-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

On Sat, May 02, 2015 at 06:26:47PM -0400, Tim Dunphy
wrote:
> >
> > Not just /var/log/messages.  Doesn't nrpe have a log file?  Maybe
even
> > secure.
> 
> 
> Hmmm I don't find any log specific to nrpe. In other words I don't
see
> /var/log/nrpe.log or whatever. :)
> 
> And when I tail -f /var/log/secure or /var/log/messages I don't see any
> entries turning up in them when I hit the client with check_nrpe. I was
> checking the logs on the client itself.
Are xinetd log entries written when you connect from localhost?
-- 
Jonathan Billings <billings at negate.org>

On Fri, May 1, 2015 at 10:16 AM, Tim Dunphy <bluethundr at gmail.com>
wrote:
>  I am trying to monitor a host in the Amazon EC2 cloud.
>
> Yet when I try to check NRPE from the monitoring host I am getting an SSL
> handshake error:
>
> [root at monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H
ops.jokefire.com
> CHECK_NRPE: Error - Could not complete SSL handshake.
>
Don't if these links are of any help but worth checking (if you have
not done so)
<http://assets.nagios.com/downloads/nagiosxi/docs/NRPE-Troubleshooting-and-Common-Solutions.pdf>
and
<http://geekpeek.net/could-not-complete-ssl-handshake/>

-- Arun Khan