Puppet users - Nov 2010 - Combine multiple CA certificates into a bundle

I am struggling with using multiple puppet CAs.  I think I am missing
information about which files are used for which purposes by different
parts of puppetmasterd, puppetd, puppetca, and the apache/passenger
frontend.

I have an old puppetmaster (and CA), which signed certificates for old
clients.  I want to add a new puppetmaster (and CA) and let it sign
certificates for new clients.  I want any client (old or new) to be able
to work with any master (old or new).  I would prefer not to have to set
up a centralised CA hierarchy; the self-signed CA certificates would be
good enough if I could just figure out how to combine them into usable
bundles.

It seems to me that I should be able to take the $ssldir/ca/ca_crt.pem
files from the two puppetmasters, concatenate them to create a
ca-bundle.pem file, and place the bundle in some suitable place where
any client or server can use the bundle to verify certificates from any
CA.  I haven''t been able to figure out where that suitable place is.
Help?

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, 29 Nov 2010, Alan Barrett wrote:
> I am struggling with using multiple puppet CAs.  I think I am missing
> information about which files are used for which purposes by different
> parts of puppetmasterd, puppetd, puppetca, and the apache/passenger
> frontend.
For example, which file on the puppetmaster ends up being copied
to $localcacert on the client?  Neither of the files referred
to by the puppet.conf variables [puppetmasterd].cacert or
[puppetmasterd].localcacert seems to be copied to the client.

(Running puppet 0.25.5 on all masters and clients.)

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, 29 Nov 2010, Alan Barrett wrote:
> It seems to me that I should be able to take the $ssldir/ca/ca_crt.pem
> files from the two puppetmasters, concatenate them to create a
> ca-bundle.pem file, and place the bundle in some suitable place where
> any client or server can use the bundle to verify certificates from
> any CA.  I haven''t been able to figure out where that suitable
place
> is.  Help?
I have a ca-bundle.pem file which contains the concatenation of the ca.pem
files from the two CAs.

I make both cacert and localcacert on the puppetmaster refer to that file:

   # extract frmo pupept.conf
   [puppetmasterd]
   cacert = $ssldir/certs/ca-bundle.pem
   localcacert = $ssldir/certs/ca-bundle.pem

The first time I run puppetd on the client it creates
/etc/puppet/ssl/certs/ca.cert with the wrong contents (the cert from
the master that it''s talking to atthe moment, not the bundle that I
want).

I manually replace the client''s /etc/puppet/ssl/certs/ca.cert with a
copy of the bundle.

Now the openssl tests described near the end of
<http://projects.puppetlabs.com/projects/1/wiki/Multiple_Certificate_Authorities>
work.  On the master, I run

    cd /etc/puppet/ssl
    openssl s_server -Verify 10 \
	-cert ./certs/${server_fqdn}.pem \
        -key ./private_keys/${server_fqdn}.pem \
	-CAfile ./certs/ca-bundle.pem

On the client, I run:

    cd /etc/puppet/ssl
    openssl s_client -connect ${serevr_fqdn}:4433 -verify 10 \
	-cert ./certs/${client_fqdn}.pem \
        -key ./private_keys/${client_fqdn}.pem \
	-CAfile ./certs/ca.pem # a copy of ca-bundle.pem

and the client successfully connects to the server, and they like
each other''s keys and certificates.

If I keep exactly the same openssl s_server running on the pupeptmaster,
and run puppetd instead of openssl s_client on teh client side, it fails.
I run this command:

    puppetd --onetime --test --debug \
	--server=${server_fqdn} --masterport=4433

and it prints several messages about "debug: /File[foo]: Autorequiring
File[bar]", then this:

    debug: Using cached certificate for ca
    debug: Using cached certificate for ${server_fqdn}
    debug: Using cached certificate_revocation_list for ca
    debug: catalog supports formats [blah blah]; using pson
    err: could not retrieve catalog from remote server: \
	SSL_connect returned=1 errno=0 \
	state=SSLv3 read server certificate B: certificate verify failed

On the server side (the openssl s_server process, not puppet), I see this:

    ACCEPT
    ERROR
    29884:error:14094418:SSL routines:SSL3_READ_BYTES:\
	tlsv1 alert unknown ca:s3_pkt.c:1052:SSL alert number 48
    shutting down SSL
    CONNECTION CLOSED

So, it seems that the puppetd client is doing something different from
the "openssl s_client" command used for testing.  What certificate is
the puppetd client attempting to present, and how can I change that?

There''s no apache+passenger in this test, but in reality I would be
using that on the server.  This is puppet 0.25.5.

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Nov 29, 2010, at 10:13 AM, Alan Barrett wrote:
> On Mon, 29 Nov 2010, Alan Barrett wrote:
>> It seems to me that I should be able to take the $ssldir/ca/ca_crt.pem
>> files from the two puppetmasters, concatenate them to create a
>> ca-bundle.pem file, and place the bundle in some suitable place where
>> any client or server can use the bundle to verify certificates from
>> any CA.  I haven''t been able to figure out where that suitable
place
>> is.  Help?
> 
> So, it seems that the puppetd client is doing something different from
> the "openssl s_client" command used for testing.  What
certificate is
> the puppetd client attempting to present, and how can I change that?
Run this on the client for the config puppet is using:
puppetd --genconfig

Technically this won''t actually use information passed to the puppet
executable using flags what ever starts the service, but this usually
doesn''t matter.

If that is too much information try:
puppetd --genconfig | grep host | grep .pem

You will probably need to run these as root to get the correct config.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, 29 Nov 2010, Patrick wrote:
> > So, it seems that the puppetd client is doing something different from
> > the "openssl s_client" command used for testing.  What
certificate is
> > the puppetd client attempting to present, and how can I change that?
> 
> Run this on the client for the config puppet is using:
> puppetd --genconfig
I use that all the time.  The file names that I passed to "openssl
s_client" are identical to those reported by "puppetd
--genconfig".

Whether or not the clientcrl file (ca_crl.pem) exists seems to have
something to do with the problem but I haven''t figured out the details.
If I delete that file, then the puppetd client can connect, and it
downloads a fresh copy of the CRL, after which it can no longer connect.
I have configured certificate_revocation=false on the server, but it
nevertheless sends the CRL file to the client.

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Have you perhaps revoked a certificate off one CA that matched the serial number
issued by another? And perhaps the second CA issued that particular serial
number to the puppet server itself?

No, I haven''t done this myself, why do you ask?  *whistles tunelessly*

https://projects.puppetlabs.com/issues/4948


On Nov 29, 2010, at 1:44 PM, Alan Barrett wrote:
> On Mon, 29 Nov 2010, Patrick wrote:
>>> So, it seems that the puppetd client is doing something different
from
>>> the "openssl s_client" command used for testing.  What
certificate is
>>> the puppetd client attempting to present, and how can I change
that?
>> 
>> Run this on the client for the config puppet is using:
>> puppetd --genconfig
> 
> I use that all the time.  The file names that I passed to "openssl
> s_client" are identical to those reported by "puppetd
--genconfig".
> 
> Whether or not the clientcrl file (ca_crl.pem) exists seems to have
> something to do with the problem but I haven''t figured out the
details.
> If I delete that file, then the puppetd client can connect, and it
> downloads a fresh copy of the CRL, after which it can no longer connect.
> I have configured certificate_revocation=false on the server, but it
> nevertheless sends the CRL file to the client.
> 
> --apb (Alan Barrett)
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
> 
 - Eric Sorenson - N37 17.255 W121 55.738  - http://twitter.com/ahpook  -

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Mon, 29 Nov 2010, Eric Sorenson wrote:
> Have you perhaps revoked a certificate off one CA that matched the serial
number issued by another? And perhaps the second CA issued that particular
serial number to the puppet server itself?
No, neither CA has ever revoked a certificate.  "openssl crl -text"
reports that the ca_crl.pem file contains no revocations.

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi,

I think you are facing https://projects.puppetlabs.com/issues/3640#note-11

<https://projects.puppetlabs.com/issues/3640#note-11>which would hopefully
be resolved in https://projects.puppetlabs.com/issues/3770.

I must admit that I did not have time to continue troubleshooting this
further...
Ohad

On Mon, Nov 29, 2010 at 11:44 PM, Alan Barrett <apb@cequrux.com> wrote:
> On Mon, 29 Nov 2010, Patrick wrote:
> > > So, it seems that the puppetd client is doing something different
from
> > > the "openssl s_client" command used for testing.  What
certificate is
> > > the puppetd client attempting to present, and how can I change
that?
> >
> > Run this on the client for the config puppet is using:
> > puppetd --genconfig
>
> I use that all the time.  The file names that I passed to "openssl
> s_client" are identical to those reported by "puppetd
--genconfig".
>
> Whether or not the clientcrl file (ca_crl.pem) exists seems to have
> something to do with the problem but I haven''t figured out the
details.
> If I delete that file, then the puppetd client can connect, and it
> downloads a fresh copy of the CRL, after which it can no longer connect.
> I have configured certificate_revocation=false on the server, but it
> nevertheless sends the CRL file to the client.
>
> --apb (Alan Barrett)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Tue, 30 Nov 2010, Ohad Levy wrote:
> I think you are facing https://projects.puppetlabs.com/issues/3640#note-11
> 
> <https://projects.puppetlabs.com/issues/3640#note-11>which would
hopefully
> be resolved in https://projects.puppetlabs.com/issues/3770.
> 
> I must admit that I did not have time to continue troubleshooting this
> further...
Thanks.  I give up now.

--apb (Alan Barrett)

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.