Puppet users - Nov 2010 - what are the constraints on the contents of 'imported ' files

I am trying to import a hash definition from a separate file but this
fails whereas including the text verbatim in the original manifest
works fine:

class monitor ($master, $interface)  {

#  import "masters.pp"


   $sensor_rule_categories = {
        ''dmzo'' =>
[scan,finger,ftp,telnet,rpc,rservices,ddos,dns,tftp,web-
coldfusion,misc,web-php,x11,attack-responses,
                   oracle,mysql,pop2,pop3,bad-
traffic,dos,exploit,misc,smtp,web-misc,emerging-exploit,emerging-scan,
                   emerging-web,emerging-web_sql_injection,emerging-
web_server,emerging-web_specific_apps,
                   emerging-attack_response,emerging-
virus,emerging,emerging-trojan]
    }


   $rule_files = $sensor_rule_categories[$master]

if I have the definition $sensor_rule_categories in the file ./
masters.pp then I get the error:

err: Could not retrieve catalog from remote server: Error 400 on
SERVER: undefined local variable or method `accesskey'' for
#<Puppet::Parser::AST::HashOrArrayAccess:0x2afda2544b40> at /etc/
puppet/modules/monitor/manifests/init.pp:19 on node
mon263550.insec.auckland.ac.nz

I really want to have this stuff in a separate file as it is generated
elsewhere I want to have a symlink to the file with the defs rather
than have to hack at init.pp itself.

I guess my real question is what are the constraints on the contents
of ''imported '' files?

Also, should I report (where ?) things that make puppet return ruby
errors so they can be ''rescued'' and have a more meaningful
error
message given?  I''m happy to file bug report if there is an intention
to fix things like this.

Russell

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Nov 18, 7:09 pm, "russell.fulton" <russell.ful...@gmail.com>
wrote:
> I am trying to import a hash definition from a separate file but this
> fails whereas including the text verbatim in the original manifest
> works fine:
>
> class monitor ($master, $interface)  {
>
> #  import "masters.pp"
>
>    $sensor_rule_categories = {
[...]
>     }
>
>    $rule_files = $sensor_rule_categories[$master]
[}]
> if I have the definition $sensor_rule_categories in the file ./
> masters.pp then I get the error:
[...]
> I really want to have this stuff in a separate file as it is generated
> elsewhere I want to have a symlink to the file with the defs rather
> than have to hack at init.pp itself.
>
> I guess my real question is what are the constraints on the contents
> of ''imported '' files?
Do not think of the Puppet "import" statement like the C
preprocessor''s #include directive.  They are not analogous. 
Puppet''s
"import" is more like Java''s "import": it makes
complete manifest
subunits (classes and definitions) available within the importing
manifest.

See http://docs.puppetlabs.com/guides/language_tutorial.html#importing-manifests

(though that doesn''t say much more than I already have done).

In addition:

1) You can still put your hash in a separate file by wrapping it in
its own class.  To use it, include the class (after importing its
manifest, if necessary), and reference the variable by its fully-
qualified name.

2) At first I read your comments about init.pp as indicative that
you''re putting everything in the master manifest.  Now I''m not
so
sure, but if that''s indeed what you are doing then I strongly advise
you to structure your manifests into modules instead.

3) If you *are* using modules, and you are following the naming and
structure guidelines, then you should never need to explicitly import
anything.

4) I''m curious why you need to generate your
"$sensor_rule_categories"
hash externally.  There''s nothing particularly wrong with doing that,
but it seems like it would limit your ability to expand your use of
Puppet.
> Also, should I report (where ?) things that make puppet return ruby
> errors so they can be ''rescued'' and have a more
meaningful error
> message given?  I''m happy to file bug report if there is an
intention
> to fix things like this.
The top page of the Puppetlabs bug tracker is

http://projects.puppetlabs.com/

Puppetlabs is very active fixing bugs and developing new features for
Puppet.  They crave issue reports.  (And also bugfixes!  Puppet is
open source, you know.)  However, I cannot speak to how any particular
report will be prioritized or what kind of resolution it might
receive.


Best,

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Thanks very much John.  I had indeed been thinking of import as being
like include.

As it happens I have completely reworked this stuff and now the
monitor module just deals with specifying what packages and users are
pushed out.  It also now imports a two defines which deal with all the
config stuff.   What I have is a monitor machine which has one or more
sensors which intern has one or more instances.  I was trying to do
the everything in the parameterised module -- not a good idea :)

one question can you have more than one instance of a parameterised
module?  I am guessing the answer is no.  I could not find an explicit
answer in the docs.

I''ll trim most of the rest and answer a few other points
inline:
>
> 2) At first I read your comments about init.pp as indicative that
> you''re putting everything in the master manifest.  Now
I''m not so
> sure, but if that''s indeed what you are doing then I strongly
advise
> you to structure your manifests into modules instead.
Agreed :)  and no, the ''master'' manifest just has config data
about
the ''master'' (or template) sensor -- each master may have
several
clones.
and everything is set up using modules.

>
> 3) If you *are* using modules, and you are following the naming and
> structure guidelines, then you should never need to explicitly import
> anything.
Sorry, not sure what you mean by "explicitly import".  You mean
explicitly pulling in bits like that variable definition? then yes I
agree.
>
> 4) I''m curious why you need to generate your
"$sensor_rule_categories"
> hash externally.  There''s nothing particularly wrong with doing
that,
> but it seems like it would limit your ability to expand your use of
> Puppet.
I have a management system that provisions the monitoring systems
previously it did everything and I am now moving to get puppet to do
the heavy lifting.  The idea is that I have all the meta data in
puppet and my management system generates the files for puppet.  I
want the files that the management system produces to be sand alone.
i.e. I don''t want to have the management system to have to insert
stuff in the middle of init.pp.

I now have this refined down to single file and I am rather pleased
with it:

monitor/.../init.pp

# The various mon* nodes (a non-regexp definition will take
precedence)

node ''monitor'' inherits basenode {
   include monitor
}

# the generic node -- installs software only

node /^mon\d+\.insec\.auckland\.ac\.nz$/ inherits monitor {}

# active nodes

import "monitors/*.pp"

monitors/mon263550.pp:

node ''mon263550.insec.auckland.ac.nz'' inherits monitor {
    monitor::sensor { monitor_dmzo_58:
			name => ''dmzo-58'', master =>
''dmzo'', interfaces =>  ''eth1'',
                        rule_categories => [
 
scan,finger,ftp,telnet,rpc,rservices,ddos,dns,tftp,web-
coldfusion,misc,web-php,
                                x11,attack-
responses,oracle,mysql,pop2,pop3,bad-traffic,dos,exploit,misc,smtp,
				web-misc,emerging-exploit,emerging-scan,emerging-web,emerging-
web_sql_injection,
				emerging-web_server,emerging-web_specific_apps,emerging-
attack_response,emerging-virus,
                                emerging,emerging-trojan
			]

     }
}


Sensor is a define and there may be more than one, and the interfaces
can be an array in which case multiple configs are generated.
This file will be generated automatically by my management script for
each monitor machine.

It has taken me about a month to get to this point but I am satisfied
with what I have now (starting completely from scratch with puppet).
There have been two complete reworks in that time.

I''d like to take this opportunity to thank the folk in this group for
their help and the folk at puppetlabs who answered questions
directly.
>
> Puppetlabs is very active fixing bugs and developing new features for
> Puppet.  They crave issue reports.  (And also bugfixes!  Puppet is
> open source, you know.)  However, I cannot speak to how any particular
> report will be prioritized or what kind of resolution it might
> receive.
>
I start using it :)  and if I get time I''ll investigate some of these
things myself -- yes I do Ruby ;-)

Russell

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

On Nov 20, 1:53 pm, "russell.fulton" <russell.ful...@gmail.com>
wrote:
> one question can you have more than one instance of a parameterised
> module?  I am guessing the answer is no.  I could not find an explicit
> answer in the docs.
I think you''re asking about parameterized *classes*, as the Puppet
terminology goes.  The word "module" has a domain-specific meaning in
Puppet (http://docs.puppetlabs.com/guides/modules.html) that doesn''t
involve parameterization as far as I am aware.

I don''t recall off-hand whether a parameterized class can be included
multiple times with different parameters.  That''s pretty much what
defines provide, however, and you already know about them.  Is there
something you''d like to do along those lines for which a define is
insufficient?

[...]
> Sorry, not sure what you mean by "explicitly import".
I mean using "import" statements.  At all.  If you organize your
manifests into Puppet modules and follow the conventions of (a) using
module names as namespaces for their classes and defines, and (b)
giving each class and define a like-named manifest file in its module,
then Puppet will find and load those manifests as needed,
automatically, without any import statements.  (See
http://docs.puppetlabs.com/guides/modules.html#module-lookup for
details).  In contrast to importing manifests explicitly, you could
consider that to be importing the relevant manifests implicitly.
> > 4) I''m curious why you need to generate your
"$sensor_rule_categories"
> > hash externally.  There''s nothing particularly wrong with
doing that,
> > but it seems like it would limit your ability to expand your use of
> > Puppet.
>
> I have a management system that provisions the monitoring systems
> previously it did everything and I am now moving to get puppet to do
> the heavy lifting.  The idea is that I have all the meta data in
> puppet and my management system generates the files for puppet.  I
> want the files that the management system produces to be sand alone.
> i.e. I don''t want to have the management system to have to insert
> stuff in the middle of init.pp.
[...]

Thanks for the explanation.  It sounds like an interesting setup.
> It has taken me about a month to get to this point but I am satisfied
> with what I have now (starting completely from scratch with puppet).
> There have been two complete reworks in that time.
If you''re satisfied with what you''ve got then great. 
I''m glad Puppet
is helping you out.


Cheers,

John

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.