Puppet users - Apr 2013 - Fileserver on a client rather than a puppetmaster

Background:
The puppetmasters are under the authority and control of a specific group
(not mine ;-) ).  Client sysadmins have control over the node manifests
and, to a certain extent, can modify the puppet modules.

One of the modules (and potentially others) used by my group uses a lot
of files that may get changed frequently.  Rather than embedding them
in the module'' files/ tree, I would like to set up one of our
group''s
clients as a fileserver and have the module reference them from there
rather than from the default location (the puppetmaster).

One major advantage to doing this is that we could update the files
directly, using whatever mechanism suits our purposes best, as opposed
to treating updates as a change to the module and having to update
the module itself.

So far I have found absolutely *no* documentation on how to set up a
client to function as a fileserver.

To experiment, I updated the fileserver.conf and auth.conf on the
fileserver-to-be client, and put some files into it by hand.  I made
sure that the agent was listening for requests.  I then tried playing
with it through the REST API:

http://docs.puppetlabs.com/guides/rest_api.html

Attempts to fetch a file (using http://$CLIENT:8139/$ENV/file_contents/...)
resulted in:

  No specified acceptable formats (*/*) are functional on this machine

Changing the Accept: request header field just resulted in the
''*/*''
in the message changing to match.

Can anyone point me to documentation on how a client can be set up as a 
fileserver,
and what needs to be done to make the files servable?  (Presumably just 
putting
them in place is insufficient; some sort of metadata probably need to 
be[re]built,
is my guess.)

Thanks!

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On 18.04.2013 21:00, Ken Coar wrote:
> Background:
> The puppetmasters are under the authority and control of a specific group
> (not mine ;-) ).  Client sysadmins have control over the node manifests
> and, to a certain extent, can modify the puppet modules.
>
> One of the modules (and potentially others) used by my group uses a lot
> of files that may get changed frequently.  Rather than embedding them
> in the module'' files/ tree, I would like to set up one of our
group''s
> clients as a fileserver and have the module reference them from there
> rather than from the default location (the puppetmaster).
>
> One major advantage to doing this is that we could update the files
> directly, using whatever mechanism suits our purposes best, as opposed
> to treating updates as a change to the module and having to update
> the module itself.
Is there any reason to use a puppet as source? You can put that on an 
NFS share too and reference it by path. Or clone a git repo.


Regards, David


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Friday, April 19, 2013 5:46:37 AM UTC-4, David Schmitt wrote:
>
> Is there any reason to use a puppet as source? You can put that on an 
> NFS share too and reference it by path. Or clone a git repo. 
>
 
Simplicity.  Keeping everything in the puppet milieu, which is already
set up.  Don''t want to introduce new mechanisms and dependencies
rather than making use of what''s already there.

#k

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On 04/19/2013 07:06 PM, Ken Coar wrote:
> Simplicity.  Keeping everything in the puppet milieu, which is already
> set up.  Don''t want to introduce new mechanisms and dependencies
> rather than making use of what''s already there.
I''m not so sure about that.

I do believe that you could do what you have in mind, but you''d need an
additional master, not an agent node "somehow serving files".

Assuming you do setup a master of your own, and setup the manifest so
that the new server will be used like

file { "foo":
  source => "puppet://local.puppet.master/modules/foomod/foo"
}

Even then, your local master would need to share the SSL certificate of
the original master, since you likely cannot make your agent use
different certificates per master.

So in the end, setting up an additional puppet fileserver may imply the
opposite of simplicity. But if you *can* setup an NFS share, rsync
target, svn checkout etc. without much hassle, as David suggests, that
may make things very simple indeed.

Cheers,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Friday, April 19, 2013 12:06:53 PM UTC-5, Ken Coar
wrote:
>
> On Friday, April 19, 2013 5:46:37 AM UTC-4, David Schmitt wrote:
>
>>
>> Is there any reason to use a puppet as source? You can put that on an 
>> NFS share too and reference it by path. Or clone a git repo. 
>>
>  
> Simplicity.  Keeping everything in the puppet milieu, which is already
> set up.  Don''t want to introduce new mechanisms and dependencies
> rather than making use of what''s already there.
>
>
Well, no.  The Puppet agent does not function as a file server, so
you''re
not proposing to make use of anything that''s already there.  And, even
if
the agent could provide file services, you''re still talking about
setting
up an additional server.

More importantly, you are proposing a dramatic complication of the 
management architecture, no matter how you implement it.  You are 
suggesting that the master delegate authority over (some) files and their 
contents to another principal under different administrative control.  The 
specifics of how that other principal fulfills that responsibility pale in 
comparison with the fact that the master is performing such delegation in 
the first place.  Were it me, I would prefer to make the processes as 
distinct as possible so as to minimize confusion of responsibilities.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, April 22, 2013 9:48:21 AM UTC-4, jcbollinger wrote:
>
> Well, no.  The Puppet agent does not function as a file server, so
you''re
> not proposing to make use of anything that''s already there.  And,
even if
> the agent could provide file services, you''re still talking about
setting
> up an additional server.
>
So the fact that the ''file server'' API is listed under
''Shared master &
agent API'' on http://docs.puppetlabs.com/guides/rest_api.html is a
mistake?
 And no, not an additional server -- just adding functionality to one of 
the clients already under the master''s ægis.

More importantly, you are proposing a dramatic complication of the
> management architecture, no matter how you implement it.  You are 
> suggesting that the master delegate authority over (some) files and their 
> contents to another principal under different administrative control.  The 
> specifics of how that other principal fulfills that responsibility pale in 
> comparison with the fact that the master is performing such delegation in 
> the first place.
>
If that''s the rationale, I don''t see that the ability to have
a FQDN in
puppet:// URIs is meaningful at all.  From what you say, the host would 
need to be either absent or the puppetmaster, so what''s the point?

And, actually, I don''t see allowing my group to directly control our
files
to be a complication as much as a feature.  According to the docco, the 
agent supports being a file server, and the syntax supports a file server 
FQDN other than the master''s, and the file server would be a peer of
the
clients within the cert structure.  The alternative (or at least the *main*
alternative)
is to push all these files up to the master, and maintain them there 
through all their vicissitudes of change.  That''s extra work all round,
for
files meaningful to our group and no-one else.

Using an NFS share or the like means setting up a parallel microcosmic 
infrastructure to provide what I read the documentation as saying is 
already available and supported in Puppet 2.7, with all the additional 
development and maintenance that would entail.

Of course, if the API documentation is wrong and the agent *doesn''t*
support
file server operation, I will readily concede that I''m butting at a 
windmill.  But if the docco is *correct*, I think I''m just having
technical
issues -- philosophical questions aside. :-)

Thanks!

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Thursday, May 2, 2013 1:37:53 PM UTC-5, Ken Coar
wrote:
>
> On Monday, April 22, 2013 9:48:21 AM UTC-4, jcbollinger wrote:
>
>>
>> Well, no.  The Puppet agent does not function as a file server, so
you''re
>> not proposing to make use of anything that''s already there. 
And, even if
>> the agent could provide file services, you''re still talking
about setting
>> up an additional server.
>>
>
> So the fact that the ''file server'' API is listed under
''Shared master &
> agent API'' on http://docs.puppetlabs.com/guides/rest_api.html is a
> mistake?  And no, not an additional server -- just adding functionality to 
> one of the clients already under the master''s ægis.
>

Although the ''file server'' item is "under" the
''Shared master & agent API''
section in that it comes later in the document, it is *part of* the next 
section, ''The master HTTP
API<http://docs.puppetlabs.com/guides/rest_api.html#the-master-http-api>
''.  The shared API section lists only the
''resource[s]'' and ''certificate''
verbs.  The docs and I are saying the same thing.

 
>
> More importantly, you are proposing a dramatic complication of the 
>> management architecture, no matter how you implement it.  You are 
>> suggesting that the master delegate authority over (some) files and
their
>> contents to another principal under different administrative control. 
The
>> specifics of how that other principal fulfills that responsibility pale
in
>> comparison with the fact that the master is performing such delegation
in
>> the first place.
>>
>
> If that''s the rationale, I don''t see that the ability to
have a FQDN in
> puppet:// URIs is meaningful at all.  From what you say, the host would 
> need to be either absent or the puppetmaster, so what''s the point?
>
>
Indeed, as far as I am aware, most people do not use an authority component 
in URIs using the puppet: scheme.  I agree that doing so is of limited use, 
but the URI format permits it.  With that said, you could, in principle, 
set up a separate file server, but it would have to be either another 
master or a custom server.  There might be additional issues to sort out 
around SSL, but it should be doable.  Whether it is *advisable* to do so is 
a whole different question, however.

 
> And, actually, I don''t see allowing my group to directly control
our files
> to be a complication as much as a feature.  According to the docco, the 
> agent supports being a file server, and the syntax supports a file server 
> FQDN other than the master''s, and the file server would be a peer
of the
> clients within the cert structure.  The alternative (or at least the *main
> * alternative) is to push all these files up to the master, and maintain 
> them there through all their vicissitudes of change.  That''s extra
work all
> round, for files meaningful to our group and no-one else.
>

The agent does not support the feature, sorry.

In any case, you and I are talking about altogether different aspects of 
the issue.  You are taking a functional perspective (how do I get my files 
distributed) whereas I am taking a site architecture perspective (what is 
the source of truth? who has authority over what?).  Inasmuch as you''re
not
in charge of Puppet, that''s probably not of direct use to you, other
than
being somewhat explanatory, so sorry about that, too.

 
>
> Using an NFS share or the like means setting up a parallel microcosmic 
> infrastructure to provide what I read the documentation as saying is 
> already available and supported in Puppet 2.7, with all the additional 
> development and maintenance that would entail.
>
> Of course, if the API documentation is wrong and the agent
*doesn''t* support
> file server operation, I will readily concede that I''m butting at
a
> windmill.  But if the docco is *correct*, I think I''m just having 
> technical issues -- philosophical questions aside. :-)
>
>
What if the docco is correct and the agent doesn''t support file server 
operation?  :-)

If I were doing it then my first instinct would not be an NFS share but 
rather a version-control repository.  I''d probably choose git, but if 
there''s another you prefer subversion, mercurial, or whatever then that
would be fine, too.  Puppet (or even just a cron job) would periodically 
perform a git pull / svn update / whatever to sync clients with whatever 
state you are currently publishing.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

On Thursday, May 2, 2013 6:37:00 PM UTC-4, jcbollinger wrote:
>
> Although the ''file server'' item is "under" the
''Shared master & agent API''
> section in that it comes later in the document, it is *part of* the next 
> section, ''The master HTTP
API<http://docs.puppetlabs.com/guides/rest_api.html#the-master-http-api>
> ''.  The shared API section lists only the
''resource[s]'' and ''certificate''
> verbs.  The docs and I are saying the same thing.
>
Oh, d''oh.  Doofus alert.. 
> If I were doing it then my first instinct would not be an NFS share but 
> rather a version-control repository.  I''d probably choose git, but
if
> there''s another you prefer subversion, mercurial, or whatever then
that
> would be fine, too.  Puppet (or even just a cron job) would periodically 
> perform a git pull / svn update / whatever to sync clients with whatever 
> state you are currently publishing.
>
 That''s essentially how the embedded-in-the-module method currently in
use
works.

Thanks for your patience with me..

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.