Puppet users - Sep 2013 - Importing declarations into node/class scope

Hello Puppet Users,

I am trying to find a solution for the following use case:

On a web server I want to manage many virtual hosts. Instead of declaring 
all these hosts inside one single class (and therefore file) I would prefer 
to have one vhost declaration per file and then import the declarations 
from all those files into the scope of the node or a class. I know that the 
import statement will not work for me but maybe someone here has a solution 
for  this.

Regards,
Stepan

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Friday, September 6, 2013 1:48:45 AM UTC-5, Stepan Seycek
wrote:
>
> Hello Puppet Users,
>
> I am trying to find a solution for the following use case:
>
> On a web server I want to manage many virtual hosts. Instead of declaring 
> all these hosts inside one single class (and therefore file) I would prefer
> to have one vhost declaration per file and then import the declarations 
> from all those files into the scope of the node or a class. I know that the
> import statement will not work for me but maybe someone here has a solution
> for  this.
>
>
The most direct implementation of what you ask would be to create a 
separate class for each vhost, and then ''include'' them all
into a class or
node.

Alternatively, if the vhost parameters are sufficiently similar, then you 
can use an array resource title, possibly with a wrapper defined type, to 
declare all the vhosts compactly in one place:

site::vhost_wrapper { [
  ''vhost1'',
  ''vhost2'',
  ''vhost3'',
  ...
]:
# parameters as needed
 }


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Hi John,

thanks for your reply. Unfortunately that does not solve my actual problem.
I want to avoid maintaining multiple vhost entries inside one file. Instead
I want to be able to add a vhost by simply adding one file dedicated to
that vhost to a directory. This would allow me to easily integrate puppet
with a tool that rolls out whole application stacks including the vhosts.

Regards,
Stepan





On Fri, Sep 6, 2013 at 3:16 PM, jcbollinger
<John.Bollinger@stjude.org>wrote:
>
>
> On Friday, September 6, 2013 1:48:45 AM UTC-5, Stepan Seycek wrote:
>>
>> Hello Puppet Users,
>>
>> I am trying to find a solution for the following use case:
>>
>> On a web server I want to manage many virtual hosts. Instead of
declaring
>> all these hosts inside one single class (and therefore file) I would
prefer
>> to have one vhost declaration per file and then import the declarations
>> from all those files into the scope of the node or a class. I know that
the
>> import statement will not work for me but maybe someone here has a
solution
>> for  this.
>>
>>
>
> The most direct implementation of what you ask would be to create a
> separate class for each vhost, and then ''include'' them
all into a class or
> node.
>
> Alternatively, if the vhost parameters are sufficiently similar, then you
> can use an array resource title, possibly with a wrapper defined type, to
> declare all the vhosts compactly in one place:
>
> site::vhost_wrapper { [
>   ''vhost1'',
>   ''vhost2'',
>   ''vhost3'',
>   ...
> ]:
> # parameters as needed
>  }
>
>
> John
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, September 9, 2013 1:52:43 AM UTC-5, Stepan Seycek
wrote:
>
> Hi John,
>
> thanks for your reply. Unfortunately that does not solve my actual 
> problem. I want to avoid maintaining multiple vhost entries inside one
file.
>

My suggestion accomplishes that.  Following Puppet best practices and 
autoloader requirements, each vhost class would go into a separate file.

 
> Instead I want to be able to add a vhost by simply adding one file 
> dedicated to that vhost to a directory.
>

You didn''t say the part before.

 
> This would allow me to easily integrate puppet with a tool that rolls out 
> whole application stacks including the vhosts.
>
>
It can be done -- with ''import'', even -- but most of the
options for such a
setup are poor practice.  To suggest a good alternative, I would like to 
better understand your planned usage.  Will you be using Puppet in agent / 
master mode, or standalone mode?  Are you looking for one-time deployment, 
or continuing vhost management?  Will you ever have to support multiple 
servers with different vhosts?  With overlapping vhosts?  Do you need to 
provide for removing unwanted vhosts?  Would it be an unacceptable burden 
to run a second tool after the one that produces the vhost information, 
before it is ready for Puppet?


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Hi John,

Am 09.09.2013 15:39, schrieb jcbollinger:
>
>
> On Monday, September 9, 2013 1:52:43 AM UTC-5, Stepan Seycek wrote:
>
>     Hi John,
>
>     thanks for your reply. Unfortunately that does not solve my actual
>     problem. I want to avoid maintaining multiple vhost entries inside
>     one file.
>
>
>
> My suggestion accomplishes that.  Following Puppet best practices and 
> autoloader requirements, each vhost class would go into a separate file.
>
>     Instead I want to be able to add a vhost by simply adding one file
>     dedicated to that vhost to a directory.
>
>
>
> You didn''t say the part before.
>
>     This would allow me to easily integrate puppet with a tool that
>     rolls out whole application stacks including the vhosts.
>
>
> It can be done -- with ''import'', even -- but most of the
options for
> such a setup are poor practice.  To suggest a good alternative, I 
> would like to better understand your planned usage.  Will you be using 
> Puppet in agent / master mode, or standalone mode?  Are you looking 
> for one-time deployment, or continuing vhost management?  Will you 
> ever have to support multiple servers with different vhosts?  With 
> overlapping vhosts?  Do you need to provide for removing unwanted 
> vhosts? Would it be an unacceptable burden to run a second tool after 
> the one that produces the vhost information, before it is ready for 
> Puppet?
>
Puppet runs in master/agent mode.
Puppet shall be used for continuous vhost management.
There will be multiple servers providing reverse proxy services with 
apache httpd vhosts (different vhosts on different servers).
Removal of vhosts is definitely a requirement for the undeployment of an 
application stack.
An additional tool processing the configuration for puppet is no problem.

I know I could maintain the information about active vhosts in my 
roll-out tool and when submitting to puppet generate a class file that 
would contain all active vhost declarations and would declare removals 
for those that have been active before and are not among the active ones 
any more. However, as mentioned before, I would prefer to generate a 
file for every to-be-added vhost and every to-be-removed vhost.

Regards,
Stepan
>
> John
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to puppet-users+unsubscribe@googlegroups.com.
> To post to this group, send email to puppet-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

On Monday, September 9, 2013 2:17:48 PM UTC-5, Stepan Seycek
wrote:
>
>  Hi John,
>
> Am 09.09.2013 15:39, schrieb jcbollinger:
>  
>
>
> On Monday, September 9, 2013 1:52:43 AM UTC-5, Stepan Seycek wrote: 
>>
>>  Hi John,
>>
>>  thanks for your reply. Unfortunately that does not solve my actual 
>> problem. I want to avoid maintaining multiple vhost entries inside one
file.
>>  
>
>
> My suggestion accomplishes that.  Following Puppet best practices and 
> autoloader requirements, each vhost class would go into a separate file.
>
>  
>
>>  Instead I want to be able to add a vhost by simply adding one file 
>> dedicated to that vhost to a directory.
>>  
>
>
> You didn''t say the part before.
>
>  
>
>>  This would allow me to easily integrate puppet with a tool that rolls 
>> out whole application stacks including the vhosts.
>>
>>   
> It can be done -- with ''import'', even -- but most of the
options for such
> a setup are poor practice.  To suggest a good alternative, I would like to 
> better understand your planned usage.  Will you be using Puppet in agent / 
> master mode, or standalone mode?  Are you looking for one-time deployment, 
> or continuing vhost management?  Will you ever have to support multiple 
> servers with different vhosts?  With overlapping vhosts?  Do you need to 
> provide for removing unwanted vhosts?  Would it be an unacceptable burden 
> to run a second tool after the one that produces the vhost information, 
> before it is ready for Puppet?
>
>   
> Puppet runs in master/agent mode.
>

Ok.

 
> Puppet shall be used for continuous vhost management.
>

Ok.

 
> There will be multiple servers providing reverse proxy services with 
> apache httpd vhosts (different vhosts on different servers).
>

Ok.

So the master will support multiple distinct clients hosting httpd servers, 
each with its own set of vhosts.

 
> Removal of vhosts is definitely a requirement for the undeployment of an 
> application stack.
>

I think you will find that the requirement for removing vhosts torpedoes 
your plan for controlling which vhosts are declared by only adding or 
removing definition files from some directory.  Although Puppet does have a 
mechanism for purging unmanaged resources of some types from the target 
node, it depends on being able to enumerate all the instances present 
there, and I would be surprised if your vhost type had that ability.  If 
it''s a defined type (as opposed to a custom native type) then it surely
doesn''t.

 
> An additional tool processing the configuration for puppet is no problem.
>
> I know I could maintain the information about active vhosts in my roll-out 
> tool and when submitting to puppet generate a class file that would contain
> all active vhost declarations and would declare removals for those that 
> have been active before and are not among the active ones any more. 
> However, as mentioned before, I would prefer to generate a file for every 
> to-be-added vhost and every to-be-removed vhost.
>
>
What about managing the vhosts that are already present and intended to 
stay?  Your desired approach is not very consistent with Puppet''s
general
architecture.  Puppet is a state management tool, so its native usage 
paradigm involves you modeling the target *state* you want, and letting 
Puppet figure out what, if anything it needs to do to put the target 
machine into that state.  With to-be-added and to-be-removed lists, you 
focus on the exact dual of Puppet''s native paradigm: you emphasize the 
transitions between states and leave implicit the details of the desired 
state.  Especially, you leave unmanaged those vhosts that already exist and 
don''t need to be removed.  You can use Puppet that way, but it will be 
harder.

Perhaps a good compromise would be to let your roll-out tool perform 
transition-based changes on your manifest set (so it doesn''t need to 
maintain overall state), and use a post-processor to prepare Puppet 
declarations from the result that capture the overall target state.  For 
example, when you add a vhost to a given server, your roll-out tool drops a 
vhost-specific class definition file in an appropriate directory, and when 
you remove one it does the same, overwriting any previous file for that 
vhost.  Importantly, the tool otherwise avoids removing any of the files it 
previously dropped -- those are your persistent record of the target 
state.  Thus, you might have

modules/proxies/manifests/server1/vhost1.pp
modules/proxies/manifests/server1/vhost2.pp
...

where modules/proxies/manifests/server1/vhost1.pp contains a class 
declaring vhost1 (possibly ensuring it absent):

class proxies::server1::vhost1 {
  include ''apache''
  apache::vhost { ''s1_vhost1'':
    ensure => ''present'', # or ''absent''
    # ...
  }
}

After you run the roll-out tool or otherwise modify the vhosts, you run a 
second tool that scans the contents of the per-server directories and 
creates a class declaring all the vhost classes.  For example, it might 
create

modules/proxies/manifests/server1/all_vhosts.pp:

class proxies::server1::all_vhosts {
  include ''proxies::server1::vhost1''
  include ''proxies::server1::vhost2''
  # ...
}

THAT class is expected to always be defined, and you arrange for it to be 
included in the catalog for server1.


John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.