Puppet users - Nov 2010 - Managing private key files; content=>file vs. binary content.

Hi all,

Hi all,

We use puppet for, amongst other things, managing the private-key
files needed for things like SSL certificates for HTTPS web servers.
We have a few constraints on how these are handled, and changes in
recent versions of puppet are making this harder than it perhaps ought
to be to implement, so I''m curious to know how others are handling it.

A site''s private key file should obviously be kept private, and only
those nodes which are running the site should have access to it. This
would seem to rule out using something like

file{"/path/to/foo.key":
source=>"puppet:///keys/foo.key"} , because
any valid puppet client could access foo.key.

It''s possible to secure the file server, but not (as far as I can see)
in a way which is aware of the node''s manifest. So either we''d
have to
keep updating auth.conf with a list of nodes that were allowed to
access each key file (error-prone, we have hundreds of both, and the
node<=>required keys relationship is many-to-many ), or accept that
other nodes could access keys that they shouldn''t be able to.

So, we currently do this:

file{"/path/to/foo.key": content=>file("/keys/foo.key")}

Since (AIUI) nodes can only access the catalog for the FQDN which
matches their certificate, the puppetmaster will ensure that the key
is available only to the hosts that need it.

All good, except that in 0.25 and up (which we''re slowly migrating
to), this often doesn''t work. The rest APIs require UTF-8 content, and
keys are binary, so catalog requests fail if the key happens to
contain bytes which aren''t valid UTF-8. (http://
projects.puppetlabs.com/issues/4832 talks about this a bit, and
includes the observation that "So there’s a design decision after all:
If PSON is to be JSON compatible – no binary data.".

How are other people getting around this? Do you just allow all
clients to access all keys? Is there a native type, or an auth.conf
trick, that I''m missing? Or a more binary-friendly encoding than JSON/
PSON ?

thanks!

Chris

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Chris,

2010/11/10 Chris <chrismay50@gmail.com>
> [...]
> file{"/path/to/foo.key":
source=>"puppet:///keys/foo.key"} , because
> any valid puppet client could access foo.key.
> [...]
>

you are not sticked to the puppet file server, you can also use something
like this:

file {
   "/path/to/file":
     source => "/nfs/$host/file";
}

Make an export for each connected server and restrict access to this one.
Put all private files on the NFS server and you''re done.

Kind regards, Thomas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Thomas

On Nov 10, 12:44 pm, Thomas Bendler <thomas.bend...@cimt.de>
wrote:
> Hi Chris,
>
> 2010/11/10 Chris <chrisma...@gmail.com>
>
> > [...]
> > file{"/path/to/foo.key":
source=>"puppet:///keys/foo.key"} , because
> > any valid puppet client could access foo.key.
> > [...]
>
> you are not sticked to the puppet file server, you can also use something
> like this:
>
> file {
>    "/path/to/file":
>      source => "/nfs/$host/file";
>
> }
>
> Make an export for each connected server and restrict access to this one.
> Put all private files on the NFS server and you''re done.
>
Yes, except that approach suffers from the same administrative
problems as using puppet:/// and auth.conf. HTTPS certs aren''t
specific to hosts. If I have 20 servers all requiring foo.key (because
they all have the foo-application class in their manifest), then
either I have to copy foo.key into 20 different directories, or else
have one export with 20 allowed hosts. And every time I add the foo-
application class to another host, I need to remember to also expose
the key to that host. With large numbers of keys and hosts, and
moderate levels of churn, this becomes difficult to manage and prone
to errors.

The puppetmaster "knows" which hosts are allowed foo.key - i.e. all
the hosts which include the foo-application class. It seems wrong that
I should have to manually duplicate that information somewhere else,
be it in an NFS exports list or an auth.conf file.

I suppose I could do something hacky with storeconfigs to update the
exports on the NFS server when a new host is brought online - but it
doesn''t seem like a very nice solution. It would lead to the first
puppet run failing because the exports weren''t yet updated, for one
thing.

Thanks!

Chris
> Kind regards, Thomas
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> All good, except that in 0.25 and up (which we''re slowly migrating
> to), this often doesn''t work. The rest APIs require UTF-8 content,
and
> keys are binary, so catalog requests fail if the key happens to
> contain bytes which aren''t valid UTF-8. (http://
> projects.puppetlabs.com/issues/4832 talks about this a bit, and
> includes the observation that "So there’s a design decision after all:
> If PSON is to be JSON compatible – no binary data.".
>
> How are other people getting around this? Do you just allow all
> clients to access all keys? Is there a native type, or an auth.conf
> trick, that I''m missing? Or a more binary-friendly encoding than
JSON/
> PSON ?
I also suffer from this problem distributing binary GPG private keys.
I would propose Puppet automatically base-64 encode/decode when a
parameter''s value (in this case a file''s content but it could
be
anything) does not contain valid UTF-8 bytes.  Would that fix the
problem completely?  Would it break anything?

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Nov 10, 2010, at 6:52 AM, Chris wrote:
> Hi Thomas
> 
> On Nov 10, 12:44 pm, Thomas Bendler <thomas.bend...@cimt.de> wrote:
>> Hi Chris,
>> 
>> 2010/11/10 Chris <chrisma...@gmail.com>
>> 
>>> [...]
>>> file{"/path/to/foo.key":
source=>"puppet:///keys/foo.key"} , because
>>> any valid puppet client could access foo.key.
>>> [...]
>> 
>> you are not sticked to the puppet file server, you can also use
something
>> like this:
>> 
>> file {
>>    "/path/to/file":
>>      source => "/nfs/$host/file";
>> 
>> }
>> 
>> Make an export for each connected server and restrict access to this
one.
>> Put all private files on the NFS server and you''re done.
>> 
> 
> Yes, except that approach suffers from the same administrative
> problems as using puppet:/// and auth.conf. HTTPS certs aren''t
> specific to hosts. If I have 20 servers all requiring foo.key (because
> they all have the foo-application class in their manifest), then
> either I have to copy foo.key into 20 different directories, or else
> have one export with 20 allowed hosts. And every time I add the foo-
> application class to another host, I need to remember to also expose
> the key to that host. With large numbers of keys and hosts, and
> moderate levels of churn, this becomes difficult to manage and prone
> to errors.
> 
> The puppetmaster "knows" which hosts are allowed foo.key - i.e.
all
> the hosts which include the foo-application class. It seems wrong that
> I should have to manually duplicate that information somewhere else,
> be it in an NFS exports list or an auth.conf file.
> 
> I suppose I could do something hacky with storeconfigs to update the
> exports on the NFS server when a new host is brought online - but it
> doesn''t seem like a very nice solution. It would lead to the first
> puppet run failing because the exports weren''t yet updated, for
one
> thing.
The best solution I can come up with is creating a hack that uses a define and a
custom ruby function that will Base64Encode and then have the client
Base64Decide (using an exec or custom provider) on the other end.  This comes
from something I''lm building, but probably won''t be done for a
long while.

This is pesudecode which is missing the encode function, decode function, and
some of the glew code.


class binary_embedded_file::setup
{
	$temp = ''/var/lib/puppet/binary_embedded_file''

	file { ''/var/lib/puppet/binary_embedded_file'':
		ensure => directory,
		mode => 750,
		owner => root,
		group => root,
	}
	
	file { ''/usr/local/bin/base64_decode'':
		ensure => present,
		owner => root,
		group => root,
		mode => 755,
	}
}
		


define binary_embedded_file($ensure = present,
	$server_location = nil, $client_location = nil ) {

	include binary_embedded_file::setup

	#Syntax might be wrong
	require( Class[''binary_embedded_file::setup''] )

	#Look at puppet_concat example for how to finish these
	$client_temp_path = 
	$client_temp_path_converted = 


	file { "${name}":
		ensure => $ensure,
		#Add a mode, owner, and group variable
		
		#This syntax might be wrong
		source => $client_temp_path_converted
	}

	file { "${client_temp_path}":
		ensure => $ensure,
		
		content => Base64Encode(file($server_location)),
	}

	#There might be an unintended line wrap here
	exec { ''/usr/local/bin/base64_decode \"$client_temp_path\"
\"$client_temp_path_converted\"'':
		subscribe_only => true,
		subscribe => File["${client_temp_path}"],
		before => File["${name}"]
	}
	
}


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Nov 10, 2010, at 4:29 AM, Chris wrote:
> How are other people getting around this? Do you just allow all
> clients to access all keys? Is there a native type, or an auth.conf
> trick, that I''m missing? Or a more binary-friendly encoding than
JSON/
> PSON ?

I send a different message with a rather long hack as a workaround, but I would
also file a bug for this.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Hi Chris,

2010/11/10 Chris <chrismay50@gmail.com>
> [...]
> Yes, except that approach suffers from the same administrative
> problems as using puppet:/// and auth.conf. HTTPS certs aren''t
> specific to hosts. If I have 20 servers all requiring foo.key (because
> they all have the foo-application class in their manifest), then
> either I have to copy foo.key into 20 different directories, or else
> have one export with 20 allowed hosts. And every time I add the foo-
> application class to another host, I need to remember to also expose
> the key to that host. With large numbers of keys and hosts, and
> moderate levels of churn, this becomes difficult to manage and prone
> to errors.
>
got the point, thought that you need one specific key on each server. So
that should be even simpler, use file with content and put the key in the
content field:

$myKey = "-----BEGIN RSA PRIVATE
KEY-----\nMIICXgIBAAKBgQDTqkVS4/iwKx8LngXQrEShlfSRtcSyOB1IjC5AIGUAJvapq9lz\n..."

file {
  "/path/to/keyFile":
    content => $myKey;
}

Put this into your Webserver class and assign the class only to the
Webservers.

Kind regards, Thomas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> got the point, thought that you need one specific key on each server. So
> that should be even simpler, use file with content and put the key in the
> content field:
>
> $myKey = "-----BEGIN RSA PRIVATE
>
KEY-----\nMIICXgIBAAKBgQDTqkVS4/iwKx8LngXQrEShlfSRtcSyOB1IjC5AIGUAJvapq9lz\n..."
>
> file {
>   "/path/to/keyFile":
>     content => $myKey;
> }
>
> Put this into your Webserver class and assign the class only to the
> Webservers.
This works perfectly for PEM-formatted keys because they''re ASCII,
which is a subset of UTF-8.  Binary keys are not (usually) valid UTF-8
and thus can''t be crammed into a catalog without some encoding.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

2010/11/10 Richard Crowley <r@rcrowley.org>
> [...]
> This works perfectly for PEM-formatted keys because they''re ASCII,
> which is a subset of UTF-8.  Binary keys are not (usually) valid UTF-8
> and thus can''t be crammed into a catalog without some encoding.
>
And why don''t you convert the key to a PEM key before putting it into
puppet? You can use OpenSSL to convert the binary key to a PEM key:

openssl enc -in some-bin.key -out some-pem.key -a

Kind regards, Thomas

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Nov 10, 2010 at 10:39 AM, Thomas Bendler <thomas.bendler@cimt.de>
wrote:
> 2010/11/10 Richard Crowley <r@rcrowley.org>
>>
>> [...]
>> This works perfectly for PEM-formatted keys because they''re
ASCII,
>> which is a subset of UTF-8.  Binary keys are not (usually) valid UTF-8
>> and thus can''t be crammed into a catalog without some
encoding.
>
> And why don''t you convert the key to a PEM key before putting it
into
> puppet? You can use OpenSSL to convert the binary key to a PEM key:
In my particular case because its unclear if ASCII encodings of
trusted.gpg and trustdb.gpg are indeed possible.

In the general case, even completely legitimate (and common) Latin-1
text files can cause Puppet problems because some Latin-1 bytes are
not valid UTF-8.  In my opinion, the content parameter of a file
resource should be able to handle these cases.

Richard

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Nov 10, 2010, at 11:48 AM, Richard Crowley wrote:
> On Wed, Nov 10, 2010 at 10:39 AM, Thomas Bendler
<thomas.bendler@cimt.de> wrote:
>> 2010/11/10 Richard Crowley <r@rcrowley.org>
>>> 
>>> [...]
>>> This works perfectly for PEM-formatted keys because
they''re ASCII,
>>> which is a subset of UTF-8.  Binary keys are not (usually) valid
UTF-8
>>> and thus can''t be crammed into a catalog without some
encoding.
>> 
>> And why don''t you convert the key to a PEM key before putting
it into
>> puppet? You can use OpenSSL to convert the binary key to a PEM key:
> 
> In my particular case because its unclear if ASCII encodings of
> trusted.gpg and trustdb.gpg are indeed possible.
> 
> In the general case, even completely legitimate (and common) Latin-1
> text files can cause Puppet problems because some Latin-1 bytes are
> not valid UTF-8.  In my opinion, the content parameter of a file
> resource should be able to handle these cases.
I think you should file a bug then.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Wed, Nov 10, 2010 at 6:26 PM, Richard Crowley <r@rcrowley.org> wrote:
> > got the point, thought that you need one specific key on each server.
So
> > that should be even simpler, use file with content and put the key in
the
> > content field:
> >
> > $myKey = "-----BEGIN RSA PRIVATE
> >
>
KEY-----\nMIICXgIBAAKBgQDTqkVS4/iwKx8LngXQrEShlfSRtcSyOB1IjC5AIGUAJvapq9lz\n..."
> >
> > file {
> >   "/path/to/keyFile":
> >     content => $myKey;
> > }
> >
> > Put this into your Webserver class and assign the class only to the
> > Webservers.
>
> This works perfectly for PEM-formatted keys because they''re ASCII,
> which is a subset of UTF-8.  Binary keys are not (usually) valid UTF-8
> and thus can''t be crammed into a catalog without some encoding.
>
>
Indeed. I made a mistake in my original post; it''s not the key files
for
apache (which are PEM-formatted ASCII) , but rather those in Java''s JKS
keystore format, that cause problems for me. I could probably create a
workaround by transferring the keys as .PEM format and then converting to
JKS on the client, but it would be a pretty fiddly solution compared to the
option of a binary-safe file() function.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

> > In the general case, even completely legitimate (and common) Latin-1
> > text files can cause Puppet problems because some Latin-1 bytes are
> > not valid UTF-8.  In my opinion, the content parameter of a file
> > resource should be able to handle these cases.
>
> I think you should file a bug then.

I''ve raised http://projects.puppetlabs.com/issues/5261

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 11.11.2010 11:06, Chris wrote:
>
>>> In the general case, even completely legitimate (and common)
Latin-1
>>> text files can cause Puppet problems because some Latin-1 bytes are
>>> not valid UTF-8.  In my opinion, the content parameter of a file
>>> resource should be able to handle these cases.
>>
>> I think you should file a bug then.
>
>
> I''ve raised http://projects.puppetlabs.com/issues/5261
>
Thinking back to the original PSON bug, the workaround back then was to 
use YAML serialization. I did notice that that could make clients crash 
though, I think the 0.25.5 were the afflicted ones.

You may want to give it shot though, anyway. The YAML encoder seems to 
be less picky where encodings are concerned.

Regards,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

>
>
> Thinking back to the original PSON bug, the workaround back then was to use
> YAML serialization. I did notice that that could make clients crash though,
> I think the 0.25.5 were the afflicted ones.
>
> You may want to give it shot though, anyway. The YAML encoder seems to be
> less picky where encodings are concerned.
>
> Alas, that doesn''t work for me either; I get
err: Could not retrieve catalog from remote server: Could not intern from
yaml: can''t convert Array into String

Thanks!

Chris

> Regards,
> Felix
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On 11/19/2010 10:56 AM, Chris May wrote:
> 
>     Thinking back to the original PSON bug, the workaround back then was
>     to use YAML serialization. I did notice that that could make clients
>     crash though, I think the 0.25.5 were the afflicted ones.
> 
>     You may want to give it shot though, anyway. The YAML encoder seems
>     to be less picky where encodings are concerned.
> 
> Alas, that doesn''t work for me either; I get 
> 
> err: Could not retrieve catalog from remote server: Could not intern
> from yaml: can''t convert Array into String
I see, but this is plain stupid. Such things are not supposed to happen.

You may want to find out which part of your catalogue causes this (i.e.,
which subset of your manifest is sufficient to reproduce this behaviour)
and raise a bug.

That is, unless this goes away with newer clients.

Regards,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Fri, Nov 19, 2010 at 11:01 AM, Felix Frank <
felix.frank@alumni.tu-berlin.de> wrote:
> On 11/19/2010 10:56 AM, Chris May wrote:
> >
> >     Thinking back to the original PSON bug, the workaround back then
was
> >     to use YAML serialization. I did notice that that could make
clients
> >     crash though, I think the 0.25.5 were the afflicted ones.
> >
> >     You may want to give it shot though, anyway. The YAML encoder
seems
> >     to be less picky where encodings are concerned.
> >
> > Alas, that doesn''t work for me either; I get
> >
> > err: Could not retrieve catalog from remote server: Could not intern
> > from yaml: can''t convert Array into String
>
> I see, but this is plain stupid. Such things are not supposed to happen.
>
> You may want to find out which part of your catalogue causes this (i.e.,
> which subset of your manifest is sufficient to reproduce this behaviour)
> and raise a bug.
>
This is with 2.6.3 on both client and server; the relevant portion of the
catalog (sufficient to cause the error on its own) is

file{"/tmp/test":
content=>"/var/puppet/private/truststore"}

/var/puppet/private/truststore is a JKS-encoded keystore.

I imagine (though I haven''t actually debugged it) that the error is
because
one element the array being serialized is is the contents of this file,
which includes the non-UTF-8 bytes which cause the serialization error in
PSON.  If that''s true, I would expect that a fix for
http://projects.puppetlabs.com/issues/5261 would probably fix the YAML issue
as well.

thanks

Chris
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.