Puppet users - Nov 2007 - Suddenly can''t access the puppetmaster anymore

Hi,

I have a number of puppets talking to 1 puppetmaster. Everything was
working fine until suddenly this week the puppets are revolting.
Whenever I try to run ''puppetd -v'' I see a lot of messages
like:

  Certificates were not trusted: hostname was not match

I''m sure I did not make any changes to DNS lately and I didn''t
upgrade
puppet on any of the machines (actually I did that now on one of the
clients and the server just to check but that doesn''t fix it). 
I''ve
found some information on this error message pointing to CNAME problems.
It is true that ''puppet'' on my network is a CNAME for the
actual server
name which is called ''bink'', however this hass always been the
case and
until last week everything ran fine.

Is there a way to get some more details on what''s wrong exactly (what
hostname ''was not match'')?  When I use openssl to connect to
the server
I see that the certificate was indeed created for ''bink'' in
stead of
''puppet''. However this worked fine before.  I even tried
making one of
the puppets go to ''bink'' in stead of
''puppet'' but that doesn''t help. I
also tried installing a new client but it doesn''t want to talk to the
server either.

Any ideas what might be wrong?

I''m using the latest Lutterkort rpm''s on CentOS 4.5 and 5.

Thanks in advance,

Nico

Perhaps Ruby has been updated recently.  Newer Rubies check the cert
name strictly.

http://reductivelabs.com/trac/puppet/ticket/896

Certainly with RHEL5.1 this now applies - which came out about a week
ago.  I assume CentOS is mostly in sync with RHEL.

Derek 

-----Original Message-----
From: puppet-users-bounces@madstop.com
[mailto:puppet-users-bounces@madstop.com] On Behalf Of Nico De Ranter
Sent: 20 November 2007 14:59
To: Puppet User Discussion
Subject: [Puppet-users] Suddenly can''t access the puppetmaster anymore


Hi,

I have a number of puppets talking to 1 puppetmaster. Everything was
working fine until suddenly this week the puppets are revolting.
Whenever I try to run ''puppetd -v'' I see a lot of messages
like:

  Certificates were not trusted: hostname was not match

I''m sure I did not make any changes to DNS lately and I didn''t
upgrade
puppet on any of the machines (actually I did that now on one of the
clients and the server just to check but that doesn''t fix it). 
I''ve
found some information on this error message pointing to CNAME problems.
It is true that ''puppet'' on my network is a CNAME for the
actual server
name which is called ''bink'', however this hass always been the
case and
until last week everything ran fine.

Is there a way to get some more details on what''s wrong exactly (what
hostname ''was not match'')?  When I use openssl to connect to
the server
I see that the certificate was indeed created for ''bink'' in
stead of
''puppet''. However this worked fine before.  I even tried
making one of
the puppets go to ''bink'' in stead of
''puppet'' but that doesn''t help. I
also tried installing a new client but it doesn''t want to talk to the
server either.

Any ideas what might be wrong?

I''m using the latest Lutterkort rpm''s on CentOS 4.5 and 5.

Thanks in advance,

Nico

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users
------------------------------------------------------------------------
For important statutory and regulatory disclosures and more information about
Barclays Capital, please visit our web site at http://www.barcap.com.

Internet communications are not secure and therefore the Barclays Group does not
accept legal responsibility for the contents of this message.  Although the
Barclays Group operates anti-virus programmes, it does not accept responsibility
for any damage whatsoever that is caused by viruses being passed.  Any views or
opinions presented are solely those of the author and do not necessarily
represent those of the Barclays Group.  Replies to this email may be monitored
by the Barclays Group for operational or business reasons.

Barclays Capital is the investment banking division of Barclays Bank PLC, a
company registered in England (number 1026167) with its registered office at 1
Churchill Place, London, E14 5HP. This email may relate to or be sent from other
members of the Barclays Group.
------------------------------------------------------------------------

Doh, yes, I remember there was an update of Ruby last week. That must be
it. Especially as the puppetd''s that are still running don''t
complain at
all.

Thanks.

Nico





On Tue, 2007-11-20 at 15:01 +0000, Derek.Whayman@barclayscapital.com
wrote:
> Perhaps Ruby has been updated recently.  Newer Rubies check the cert
> name strictly.
> 
> http://reductivelabs.com/trac/puppet/ticket/896
> 
> Certainly with RHEL5.1 this now applies - which came out about a week
> ago.  I assume CentOS is mostly in sync with RHEL.
> 
> Derek 
> 
> -----Original Message-----
> From: puppet-users-bounces@madstop.com
> [mailto:puppet-users-bounces@madstop.com] On Behalf Of Nico De Ranter
> Sent: 20 November 2007 14:59
> To: Puppet User Discussion
> Subject: [Puppet-users] Suddenly can''t access the puppetmaster
anymore
> 
> 
> Hi,
> 
> I have a number of puppets talking to 1 puppetmaster. Everything was
> working fine until suddenly this week the puppets are revolting.
> Whenever I try to run ''puppetd -v'' I see a lot of
messages like:
> 
>   Certificates were not trusted: hostname was not match
> 
> I''m sure I did not make any changes to DNS lately and I
didn''t upgrade
> puppet on any of the machines (actually I did that now on one of the
> clients and the server just to check but that doesn''t fix it). 
I''ve
> found some information on this error message pointing to CNAME problems.
> It is true that ''puppet'' on my network is a CNAME for the
actual server
> name which is called ''bink'', however this hass always
been the case and
> until last week everything ran fine.
> 
> Is there a way to get some more details on what''s wrong exactly
(what
> hostname ''was not match'')?  When I use openssl to connect
to the server
> I see that the certificate was indeed created for ''bink''
in stead of
> ''puppet''. However this worked fine before.  I even tried
making one of
> the puppets go to ''bink'' in stead of
''puppet'' but that doesn''t help. I
> also tried installing a new client but it doesn''t want to talk to
the
> server either.
> 
> Any ideas what might be wrong?
> 
> I''m using the latest Lutterkort rpm''s on CentOS 4.5 and
5.
> 
> Thanks in advance,
> 
> Nico
> 
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
> ------------------------------------------------------------------------
> For important statutory and regulatory disclosures and more information
about Barclays Capital, please visit our web site at http://www.barcap.com.
> 
> Internet communications are not secure and therefore the Barclays Group
does not accept legal responsibility for the contents of this message.  Although
the Barclays Group operates anti-virus programmes, it does not accept
responsibility for any damage whatsoever that is caused by viruses being passed.
Any views or opinions presented are solely those of the author and do not
necessarily represent those of the Barclays Group.  Replies to this email may be
monitored by the Barclays Group for operational or business reasons.
> 
> Barclays Capital is the investment banking division of Barclays Bank PLC, a
company registered in England (number 1026167) with its registered office at 1
Churchill Place, London, E14 5HP. This email may relate to or be sent from other
members of the Barclays Group.
> ------------------------------------------------------------------------
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

Hmm, I don''t get it.  I added ''server = <fqdn of the
server>''
to /etc/puppet/puppetd.conf. I verified with openssl that the server
name in the certificate indeed matches the fqdn of the server but still
I get the same error. I uninstalled puppet on the client removed all
certificates on the client, removed the certificate for that client on
the server and reinstalled puppet (making sure to specify the fqdn of
the server) on the client but I still get the same error. Sigh.

Nico


On Tue, 2007-11-20 at 16:11 +0100, Nico De Ranter wrote:
> Doh, yes, I remember there was an update of Ruby last week. That must be
> it. Especially as the puppetd''s that are still running
don''t complain at
> all.
> 
> Thanks.
> 
> Nico
> 
> 
> 
> 
> 
> On Tue, 2007-11-20 at 15:01 +0000, Derek.Whayman@barclayscapital.com
> wrote:
> > Perhaps Ruby has been updated recently.  Newer Rubies check the cert
> > name strictly.
> > 
> > http://reductivelabs.com/trac/puppet/ticket/896
> > 
> > Certainly with RHEL5.1 this now applies - which came out about a week
> > ago.  I assume CentOS is mostly in sync with RHEL.
> > 
> > Derek 
> > 
> > -----Original Message-----
> > From: puppet-users-bounces@madstop.com
> > [mailto:puppet-users-bounces@madstop.com] On Behalf Of Nico De Ranter
> > Sent: 20 November 2007 14:59
> > To: Puppet User Discussion
> > Subject: [Puppet-users] Suddenly can''t access the
puppetmaster anymore
> > 
> > 
> > Hi,
> > 
> > I have a number of puppets talking to 1 puppetmaster. Everything was
> > working fine until suddenly this week the puppets are revolting.
> > Whenever I try to run ''puppetd -v'' I see a lot of
messages like:
> > 
> >   Certificates were not trusted: hostname was not match
> > 
> > I''m sure I did not make any changes to DNS lately and I
didn''t upgrade
> > puppet on any of the machines (actually I did that now on one of the
> > clients and the server just to check but that doesn''t fix
it).  I''ve
> > found some information on this error message pointing to CNAME
problems.
> > It is true that ''puppet'' on my network is a CNAME
for the actual server
> > name which is called ''bink'', however this hass
always been the case and
> > until last week everything ran fine.
> > 
> > Is there a way to get some more details on what''s wrong
exactly (what
> > hostname ''was not match'')?  When I use openssl to
connect to the server
> > I see that the certificate was indeed created for
''bink'' in stead of
> > ''puppet''. However this worked fine before.  I even
tried making one of
> > the puppets go to ''bink'' in stead of
''puppet'' but that doesn''t help. I
> > also tried installing a new client but it doesn''t want to
talk to the
> > server either.
> > 
> > Any ideas what might be wrong?
> > 
> > I''m using the latest Lutterkort rpm''s on CentOS 4.5
and 5.
> > 
> > Thanks in advance,
> > 
> > Nico
> > 
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
> >
------------------------------------------------------------------------
> > For important statutory and regulatory disclosures and more
information about Barclays Capital, please visit our web site at
http://www.barcap.com.
> > 
> > Internet communications are not secure and therefore the Barclays
Group does not accept legal responsibility for the contents of this message. 
Although the Barclays Group operates anti-virus programmes, it does not accept
responsibility for any damage whatsoever that is caused by viruses being passed.
Any views or opinions presented are solely those of the author and do not
necessarily represent those of the Barclays Group.  Replies to this email may be
monitored by the Barclays Group for operational or business reasons.
> > 
> > Barclays Capital is the investment banking division of Barclays Bank
PLC, a company registered in England (number 1026167) with its registered office
at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from
other members of the Barclays Group.
> >
------------------------------------------------------------------------
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
> 
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users

I didn''t have too much luck with that either. On Centos4.5 I added
PUPPET_SERVER=fqdn to /etc/sysconfig/puppet

Tim 

-----Original Message-----
From: puppet-users-bounces@madstop.com
[mailto:puppet-users-bounces@madstop.com] On Behalf Of Nico De Ranter
Sent: Tuesday, November 20, 2007 12:07 PM
To: Puppet User Discussion
Subject: Re: [Puppet-users] Suddenly can''t access the puppetmaster
anymore


Hmm, I don''t get it.  I added ''server = <fqdn of the
server>''
to /etc/puppet/puppetd.conf. I verified with openssl that the server
name in the certificate indeed matches the fqdn of the server but still
I get the same error. I uninstalled puppet on the client removed all
certificates on the client, removed the certificate for that client on
the server and reinstalled puppet (making sure to specify the fqdn of
the server) on the client but I still get the same error. Sigh.

Nico


On Tue, 2007-11-20 at 16:11 +0100, Nico De Ranter wrote:
> Doh, yes, I remember there was an update of Ruby last week. That must
be
> it. Especially as the puppetd''s that are still running
don''t complain
at
> all.
> 
> Thanks.
> 
> Nico
> 
> 
> 
> 
> 
> On Tue, 2007-11-20 at 15:01 +0000, Derek.Whayman@barclayscapital.com
> wrote:
> > Perhaps Ruby has been updated recently.  Newer Rubies check the cert
> > name strictly.
> > 
> > http://reductivelabs.com/trac/puppet/ticket/896
> > 
> > Certainly with RHEL5.1 this now applies - which came out about a
week
> > ago.  I assume CentOS is mostly in sync with RHEL.
> > 
> > Derek 
> > 
> > -----Original Message-----
> > From: puppet-users-bounces@madstop.com
> > [mailto:puppet-users-bounces@madstop.com] On Behalf Of Nico De
Ranter
> > Sent: 20 November 2007 14:59
> > To: Puppet User Discussion
> > Subject: [Puppet-users] Suddenly can''t access the
puppetmaster
anymore
> > 
> > 
> > Hi,
> > 
> > I have a number of puppets talking to 1 puppetmaster. Everything was
> > working fine until suddenly this week the puppets are revolting.
> > Whenever I try to run ''puppetd -v'' I see a lot of
messages like:
> > 
> >   Certificates were not trusted: hostname was not match
> > 
> > I''m sure I did not make any changes to DNS lately and I
didn''t
upgrade
> > puppet on any of the machines (actually I did that now on one of the
> > clients and the server just to check but that doesn''t fix
it).  I''ve
> > found some information on this error message pointing to CNAME
problems.
> > It is true that ''puppet'' on my network is a CNAME
for the actual
server
> > name which is called ''bink'', however this hass
always been the case
and
> > until last week everything ran fine.
> > 
> > Is there a way to get some more details on what''s wrong
exactly
(what
> > hostname ''was not match'')?  When I use openssl to
connect to the
server
> > I see that the certificate was indeed created for
''bink'' in stead of
> > ''puppet''. However this worked fine before.  I even
tried making one
of
> > the puppets go to ''bink'' in stead of
''puppet'' but that doesn''t help.
I
> > also tried installing a new client but it doesn''t want to
talk to
the
> > server either.
> > 
> > Any ideas what might be wrong?
> > 
> > I''m using the latest Lutterkort rpm''s on CentOS 4.5
and 5.
> > 
> > Thanks in advance,
> > 
> > Nico
> > 
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
> >
------------------------------------------------------------------------
> > For important statutory and regulatory disclosures and more
information about Barclays Capital, please visit our web site at
http://www.barcap.com.
> > 
> > Internet communications are not secure and therefore the Barclays
Group does not accept legal responsibility for the contents of this
message.  Although the Barclays Group operates anti-virus programmes, it
does not accept responsibility for any damage whatsoever that is caused
by viruses being passed.  Any views or opinions presented are solely
those of the author and do not necessarily represent those of the
Barclays Group.  Replies to this email may be monitored by the Barclays
Group for operational or business reasons.
> > 
> > Barclays Capital is the investment banking division of Barclays Bank
PLC, a company registered in England (number 1026167) with its
registered office at 1 Churchill Place, London, E14 5HP. This email may
relate to or be sent from other members of the Barclays
Group.
> >
------------------------------------------------------------------------
> > _______________________________________________
> > Puppet-users mailing list
> > Puppet-users@madstop.com
> > https://mail.madstop.com/mailman/listinfo/puppet-users
> 
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Tue, 2007-11-20 at 15:58 +0100, Nico De Ranter wrote:
> I have a number of puppets talking to 1 puppetmaster. Everything was
> working fine until suddenly this week the puppets are revolting.
> Whenever I try to run ''puppetd -v'' I see a lot of
messages like:
> 
>   Certificates were not trusted: hostname was not match
The issue has been discussed on this thread [1], I posted a workaround
that worked for me [2] - longer term, the fix is what Derek pointed to.

David

[1]
http://mail.madstop.com/pipermail/puppet-users/2007-October/004692.html

[2]
http://mail.madstop.com/pipermail/puppet-users/2007-October/004703.html