All, I''m upgrading puppet clients from 0.25.5 to 2.7.1. I''ve rolled an RPM for the new version, and I''m: 1. Stopping puppet 2. Upgrading RPM 3. Change the puppet master on the client to point to a new puppet master running 2.7.1. 3. Starting puppet I am seeing this in the log files on the client: Could not evaluate: certificate verify failed Could not retrieve file metadata for puppet://hprov01.h.xxx.com/plugins: certificate verify failed Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not retrieve catalog from remote server: certificate verify failed Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Using cached catalog Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not send report: certificate verify failed Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not run Puppet configuration client: interning empty string After stopping puppet again, removing /var/lib/puppet/ssl and restarting puppet, all is ok. Why do I need to blow away the client side certs? I recently upgraded 0.25.5 to 2.6.8, and I don''t believe I had to do this. I have a couple of hundred servers to upgrade, and I don''t want to have to remove all the client side ssl directories as part of the upgrade process. Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wed, 27 Jul 2011 13:58:25 -0700, Douglas Garstang wrote:> > All, > > I''m upgrading puppet clients from 0.25.5 to 2.7.1. I''ve rolled an RPM for > the new version, and I''m: > > 1. Stopping puppet > 2. Upgrading RPM > 3. Change the puppet master on the client to point to a new puppet master > running 2.7.1. > 3. Starting puppet > > I am seeing this in the log files on the client: > > Could not evaluate: certificate verify failed Could not retrieve file > metadata for puppet://hprov01.h.xxx.com/plugins: certificate verify failed > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not retrieve > catalog from remote server: certificate verify failed > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Using cached catalog > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not send report: > certificate verify failed > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not run Puppet > configuration client: interning empty string > > After stopping puppet again, removing /var/lib/puppet/ssl and restarting > puppet, all is ok. Why do I need to blow away the client side certs? I > recently upgraded 0.25.5 to 2.6.8, and I don''t believe I had to do this. I > have a couple of hundred servers to upgrade, and I don''t want to have to > remove all the client side ssl directories as part of the upgrade process. > > Doug. >It sounds like you have a new server with 2.7.1, in addition to your old server. Did you copy over the master certificates to the new 2.7.1 master from the old one? If the new 2.7.1 master had generated a new certificate, I would expect to get the errors you''re seeing. -- Jacob Helwig ,---- | Join us for PuppetConf, September 22nd and 23rd in Portland, OR | http://bit.ly/puppetconfsig `----
On Wed, Jul 27, 2011 at 2:01 PM, Jacob Helwig <jacob@puppetlabs.com> wrote:> On Wed, 27 Jul 2011 13:58:25 -0700, Douglas Garstang wrote: > > > > All, > > > > I''m upgrading puppet clients from 0.25.5 to 2.7.1. I''ve rolled an RPM for > > the new version, and I''m: > > > > 1. Stopping puppet > > 2. Upgrading RPM > > 3. Change the puppet master on the client to point to a new puppet master > > running 2.7.1. > > 3. Starting puppet > > > > I am seeing this in the log files on the client: > > > > Could not evaluate: certificate verify failed Could not retrieve file > > metadata for puppet://hprov01.h.xxx.com/plugins: certificate verify > failed > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not retrieve > > catalog from remote server: certificate verify failed > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Using cached catalog > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not send > report: > > certificate verify failed > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not run Puppet > > configuration client: interning empty string > > > > After stopping puppet again, removing /var/lib/puppet/ssl and restarting > > puppet, all is ok. Why do I need to blow away the client side certs? I > > recently upgraded 0.25.5 to 2.6.8, and I don''t believe I had to do this. > I > > have a couple of hundred servers to upgrade, and I don''t want to have to > > remove all the client side ssl directories as part of the upgrade > process. > > > > Doug. > > > > It sounds like you have a new server with 2.7.1, in addition to your old > server. Did you copy over the master certificates to the new 2.7.1 > master from the old one? > > If the new 2.7.1 master had generated a new certificate, I would expect > to get the errors you''re seeing. > >I do. I guess I had assumed, wrongly apparently, that a client could have knowledge of multiple servers. Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wed, Jul 27, 2011 at 2:01 PM, Jacob Helwig <jacob@puppetlabs.com> wrote:> On Wed, 27 Jul 2011 13:58:25 -0700, Douglas Garstang wrote: > > > > All, > > > > I''m upgrading puppet clients from 0.25.5 to 2.7.1. I''ve rolled an RPM for > > the new version, and I''m: > > > > 1. Stopping puppet > > 2. Upgrading RPM > > 3. Change the puppet master on the client to point to a new puppet master > > running 2.7.1. > > 3. Starting puppet > > > > I am seeing this in the log files on the client: > > > > Could not evaluate: certificate verify failed Could not retrieve file > > metadata for puppet://hprov01.h.xxx.com/plugins: certificate verify > failed > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not retrieve > > catalog from remote server: certificate verify failed > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Using cached catalog > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not send > report: > > certificate verify failed > > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not run Puppet > > configuration client: interning empty string > > > > After stopping puppet again, removing /var/lib/puppet/ssl and restarting > > puppet, all is ok. Why do I need to blow away the client side certs? I > > recently upgraded 0.25.5 to 2.6.8, and I don''t believe I had to do this. > I > > have a couple of hundred servers to upgrade, and I don''t want to have to > > remove all the client side ssl directories as part of the upgrade > process. > > > > Doug. > > > > It sounds like you have a new server with 2.7.1, in addition to your old > server. Did you copy over the master certificates to the new 2.7.1 > master from the old one? > > If the new 2.7.1 master had generated a new certificate, I would expect > to get the errors you''re seeing. > > -- >Oh, and which files under /var/lib/puppet/ssl on the server would be the relevant master certs? Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wed, Jul 27, 2011 at 2:10 PM, Douglas Garstang <doug.garstang@gmail.com>wrote:> On Wed, Jul 27, 2011 at 2:01 PM, Jacob Helwig <jacob@puppetlabs.com>wrote: > >> On Wed, 27 Jul 2011 13:58:25 -0700, Douglas Garstang wrote: >> > >> > All, >> > >> > I''m upgrading puppet clients from 0.25.5 to 2.7.1. I''ve rolled an RPM >> for >> > the new version, and I''m: >> > >> > 1. Stopping puppet >> > 2. Upgrading RPM >> > 3. Change the puppet master on the client to point to a new puppet >> master >> > running 2.7.1. >> > 3. Starting puppet >> > >> > I am seeing this in the log files on the client: >> > >> > Could not evaluate: certificate verify failed Could not retrieve file >> > metadata for puppet://hprov01.h.xxx.com/plugins: certificate verify >> failed >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not retrieve >> > catalog from remote server: certificate verify failed >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Using cached >> catalog >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not send >> report: >> > certificate verify failed >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could not run >> Puppet >> > configuration client: interning empty string >> > >> > After stopping puppet again, removing /var/lib/puppet/ssl and restarting >> > puppet, all is ok. Why do I need to blow away the client side certs? I >> > recently upgraded 0.25.5 to 2.6.8, and I don''t believe I had to do this. >> I >> > have a couple of hundred servers to upgrade, and I don''t want to have to >> > remove all the client side ssl directories as part of the upgrade >> process. >> > >> > Doug. >> > >> >> It sounds like you have a new server with 2.7.1, in addition to your old >> server. Did you copy over the master certificates to the new 2.7.1 >> master from the old one? >> >> If the new 2.7.1 master had generated a new certificate, I would expect >> to get the errors you''re seeing. >> >> -- >> > > Oh, and which files under /var/lib/puppet/ssl on the server would be the > relevant master certs? > > Doug. > >Hmmm..... that''s not going to work, since the host names of the servers are different, and therefore, so are the cert names. Now I''m really confused. Since the client can''t have knowledge of two servers, this means that if things go south, and I have to switch the client back to the original master, that I will have to remove the certs again. There''s got to be an easier way. Doug. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Funny you should mention that. I have been playing around with
separating the
puppetmaster config from the agent config. See here:
http://agawamtech.com/blog/?p=383
You could probably do something similar, and it would allow you to
switch the agent from one master to another.
--
vagn
On 07/27/2011 05:13 PM, Douglas Garstang wrote:> On Wed, Jul 27, 2011 at 2:10 PM, Douglas Garstang
> <doug.garstang@gmail.com <mailto:doug.garstang@gmail.com>>
wrote:
>
> On Wed, Jul 27, 2011 at 2:01 PM, Jacob Helwig
> <jacob@puppetlabs.com <mailto:jacob@puppetlabs.com>> wrote:
>
> On Wed, 27 Jul 2011 13:58:25 -0700, Douglas Garstang wrote:
> >
> > All,
> >
> > I''m upgrading puppet clients from 0.25.5 to 2.7.1.
I''ve
> rolled an RPM for
> > the new version, and I''m:
> >
> > 1. Stopping puppet
> > 2. Upgrading RPM
> > 3. Change the puppet master on the client to point to a new
> puppet master
> > running 2.7.1.
> > 3. Starting puppet
> >
> > I am seeing this in the log files on the client:
> >
> > Could not evaluate: certificate verify failed Could not
> retrieve file
> > metadata for puppet://hprov01.h.xxx.com/plugins
> <http://hprov01.h.xxx.com/plugins>: certificate verify failed
> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could
> not retrieve
> > catalog from remote server: certificate verify failed
> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Using
> cached catalog
> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could
> not send report:
> > certificate verify failed
> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could
> not run Puppet
> > configuration client: interning empty string
> >
> > After stopping puppet again, removing /var/lib/puppet/ssl
> and restarting
> > puppet, all is ok. Why do I need to blow away the client
> side certs? I
> > recently upgraded 0.25.5 to 2.6.8, and I don''t
believe I had
> to do this. I
> > have a couple of hundred servers to upgrade, and I
don''t
> want to have to
> > remove all the client side ssl directories as part of the
> upgrade process.
> >
> > Doug.
> >
>
> It sounds like you have a new server with 2.7.1, in addition
> to your old
> server. Did you copy over the master certificates to the new
> 2.7.1
> master from the old one?
>
> If the new 2.7.1 master had generated a new certificate, I
> would expect
> to get the errors you''re seeing.
>
> --
>
>
> Oh, and which files under /var/lib/puppet/ssl on the server would
> be the relevant master certs?
>
> Doug.
>
>
> Hmmm..... that''s not going to work, since the host names of the
> servers are different, and therefore, so are the cert names. Now
I''m
> really confused. Since the client can''t have knowledge of two
servers,
> this means that if things go south, and I have to switch the client
> back to the original master, that I will have to remove the certs
> again. There''s got to be an easier way.
>
> Doug.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
vagn scott:> Funny you should mention that. I have been playing around with > separating the puppetmaster config from the agent config. See here: > > http://agawamtech.com/blog/?p=383I solved this by having the puppetmasters all proxypass (they''re running in apache+passenger) their CA port (specified in the puppet.conf as a non-standard one) up to a central CA. -- Content-type: lies/all-lies Content-disposition: blatant -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wed, Jul 27, 2011 at 2:09 PM, Douglas Garstang <doug.garstang@gmail.com> wrote:> I do. I guess I had assumed, wrongly apparently, that a client could have > knowledge of multiple servers.Clients can totally work with multiple puppet masters. They can''t however work with multiple CAs. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/28/2011 12:09 AM, Nick Moffitt wrote:> vagn scott: >> Funny you should mention that. I have been playing around with >> separating the puppetmaster config from the agent config. See here: >> >> http://agawamtech.com/blog/?p=383 > > I solved this by having the puppetmasters all proxypass (they''re running > in apache+passenger) their CA port (specified in the puppet.conf as a > non-standard one) up to a central CA.I would recommend the first way, actually the only thing you need to do is to set a seperate ssldir option within the master section (and probably appropriate certname, certdnsnames options to fit your dns alias for the puppetmaster) to seperate the ca from the client config. And why can''t you simply point your client forth and back between multiple masters that do not share a CA? This is how OpenSSL works and how it protects the communication of your puppet nodes. ~pete -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4xBwUACgkQbwltcAfKi39uxwCdGeu+XuZ03lC6fK00SuhvEryY GB8AmgNQ1Gyc+t0K6fA1JwfmLfACMssP =+E2M -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Doug,
In case everyone else''s posts haven''t solved your problem,
there''s a
fiddly but seamless way to migrate between Puppet Masters by keeping
the same CA. It''s easier if you already have a DNS alias for your
master, like "puppet" and that''s also an additional Cert DNS
Name in
the master''s certificate (which is added by default when a master
creates it''s CA cert for the first tile I think). You can check on the
master with this:
openssl x509 -text -in /var/lib/puppet/ssl/certs/$(hostname).pem |
grep -A1 ''Subject Alternative Name''
X509v3 Subject Alternative Name:
DNS:puppet, DNS:oldpuppetmaster.foo.com
Before each of these steps I''d backup /var/lib/puppet/ssl/, just so
you can get back to where you were.
Now on your old Puppet Master, generate a new certificate for your new
Puppet Master, specifying "puppet" as an additional DNS name. You
might even want to add in the DNS name of the old server:
puppetca --generate --certdnsnames oldpuppetmaster.foo.com:puppet
newpuppetmaster.foo.com
You might need to clean up any existing certs for your new server
before you do this, puppetca might complain if it already exists.
Next, copy this new cert and key to the new Puppet Master server, as
well as *all* the Puppet certs your old master has signed as well:
oldpuppetmaster$ cd /var/lib/puppet/ssl
oldpuppetmaster$ scp private_keys/newpuppetmaster.foo.com.pem
root@newpuppetmaster.foo.com:/var/lib/puppet/ssl/private_keys/
newpuppetmaster.foo.com.pem
oldpuppetmaster$ scp ca/signed/newpuppetmaster.foo.com.pem
root@newpuppetmaster.foo.com:/var/lib/puppet/ssl/certs/
newpuppetmaster.foo.com.pem
oldpuppetmaster$ scp ca/ca_crt.pem root@newpuppetmaster.foo.com:/var/
lib/puppet/ssl/certs/ca.pem
oldpuppetmaster$ scp -r ca/ root@newpuppetmaster.foo.com:/var/lib/
puppet/ssl/
Now, take one of your existing clients, poison it''s DNS with /etc/
hosts to point "puppet" to your new server, and see if it can connect.
Theoretically it should be fine... I don''t think I''ve
forgotten
anything ;)
With this method you can seamlessly phase out your old Puppet Master
and all your existing certs are still valid as you haven''t changed CA,
you''ve just moved it.
It''s a probably a bad idea to run both servers simultaneously
considering the SSL serial number - you don''t want to sign different
certs on different servers and reuse the same serial number. I believe
this only affects certificate revocation lists, but it''s best to be
nice. Therefore do your cut over as an "all in one go" operation.
Hope that helps (and please take backups of the ssl dir!),
-Luke
On Jul 27, 10:13 pm, Douglas Garstang <doug.garst...@gmail.com>
wrote:> On Wed, Jul 27, 2011 at 2:10 PM, Douglas Garstang
> <doug.garst...@gmail.com>wrote:
>
>
>
> > On Wed, Jul 27, 2011 at 2:01 PM, Jacob Helwig
<ja...@puppetlabs.com>wrote:
>
> >> On Wed, 27 Jul 2011 13:58:25 -0700, Douglas Garstang wrote:
>
> >> > All,
>
> >> > I''m upgrading puppet clients from 0.25.5 to 2.7.1.
I''ve rolled an RPM
> >> for
> >> > the new version, and I''m:
>
> >> > 1. Stopping puppet
> >> > 2. Upgrading RPM
> >> > 3. Change the puppet master on the client to point to a new
puppet
> >> master
> >> > running 2.7.1.
> >> > 3. Starting puppet
>
> >> > I am seeing this in the log files on the client:
>
> >> > Could not evaluate: certificate verify failed Could not
retrieve file
> >> > metadata for puppet://hprov01.h.xxx.com/plugins: certificate
verify
> >> failed
> >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could
not retrieve
> >> > catalog from remote server: certificate verify failed
> >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Using
cached
> >> catalog
> >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could
not send
> >> report:
> >> > certificate verify failed
> >> > Jul 27 13:53:54 hsqlstor04p1-old puppet-agent[9468]: Could
not run
> >> Puppet
> >> > configuration client: interning empty string
>
> >> > After stopping puppet again, removing /var/lib/puppet/ssl and
restarting
> >> > puppet, all is ok. Why do I need to blow away the client side
certs? I
> >> > recently upgraded 0.25.5 to 2.6.8, and I don''t
believe I had to do this.
> >> I
> >> > have a couple of hundred servers to upgrade, and I
don''t want to have to
> >> > remove all the client side ssl directories as part of the
upgrade
> >> process.
>
> >> > Doug.
>
> >> It sounds like you have a new server with 2.7.1, in addition to
your old
> >> server. Did you copy over the master certificates to the new
2.7.1
> >> master from the old one?
>
> >> If the new 2.7.1 master had generated a new certificate, I would
expect
> >> to get the errors you''re seeing.
>
> >> --
>
> > Oh, and which files under /var/lib/puppet/ssl on the server would be
the
> > relevant master certs?
>
> > Doug.
>
> Hmmm..... that''s not going to work, since the host names of the
servers are
> different, and therefore, so are the cert names. Now I''m really
confused.
> Since the client can''t have knowledge of two servers, this means
that if
> things go south, and I have to switch the client back to the original
> master, that I will have to remove the certs again. There''s got to
be an
> easier way.
>
> Doug.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.