thr3ads.net - Puppet users - [Puppet Users] A standard way to manage monitoring/firewalling in modules [Oct 2013]

If this information is useful, please help other people find it:
Share via:

Alessandro Franceschi

2013-Oct-14 09:56 UTC

[Puppet Users] A standard way to manage monitoring/firewalling in modules

During the works on stdmod naming conventions I wondered if we could find a 
non intrusive and standard way to allow management of monitoring and 
firewalling for modules.
Since there''s not a single solution to that and it would be quite hard
to
find a shared approach in a reasonable time I thought that it could/should 
be enough to define few basic parameters, to be included in a stdmod 
compliant module, leaving the actual implementation outside the module.

I thought about having parameters like:

  $monitor_class         = undef,
  $monitor_options_hash  = { } ,

  $firewall_class        = undef,
  $firewall_options_hash = { } ,


and inside the module just this ( <%= metadata.name %> is the module
name):


  if $<%= metadata.name %>::monitor_class {
    class { $<%= metadata.name %>::monitor_class:
      options_hash => $<%= metadata.name %>::monitor_options_hash,
      scope_hash   => {}, # TODO: Find a good way to inject class''
scope
    }
  }

  if $<%= metadata.name %>::firewall_class {
    class { $<%= metadata.name %>::firewall_class:
      options_hash => $<%= metadata.name %>::firewall_options_hash,
      scope_hash   => {},
    }
  }

$monitor_class is therefore the name of a custom class, external to the module,
where all the monitoring stuff is managed in whatever way.
$monitor_options_hash is an hash of custom configurations that the
$monitor_class knows how to manage .
$scope_hash should be automatically populated by the class''s scope
variables so that informations like name of services, port numbers, names of
processes (which should have a standard naming either if expressed as class
parameters or if defined internally in the class) which are generally used for
monitoring/firewalling are automatically provided to the $monitor_class

What do you think of a similar approach?
Does it make sense?
Would you use it in your modules? (even if you don''t plan to have
monitoring classes? They could be provided by third party modules)
Would you make it in a different way or with different naming? How?

To populate the $scope_hash it could be used something like
https://github.com/example42/puppi/blob/master/lib/puppet/parser/functions/get_class_args.rb
(is there something similar in stdlib? Would it make sense to add it, also
considering  that the original code is by Ken Barber?)


Al

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Dolf Schimmel - Freeaqingme

2013-Oct-16 15:12 UTC

head link

[Puppet Users] Re: A standard way to manage monitoring/firewalling in modules

Hi Al,

I fully concur that we need some sort of standardized approach of 
configuring firewall and monitoring rules. I''m not sure however as to
the
approach you''re proprosing. Two reasons:
A. The interface of all firewall and monitoring modules out there would 
need to have a fugly interface (having two params: options_hash and 
scope_hash). Parameters should be as explicit and descriptive as possible, 
merely defining *_hash is asking for bad designs I think.
B. With the architecture you propose, I would invoke a module as follows:
    class { ntp:
       port => 123,
       protocol => tcp,
       servers => [''10.1.2.3'',
''10.2.3.4'']
       firewall_options_hash => { ''port'' => 123,
''protocol'' => ''tcp'',
destination => [''10.1.2.3'', ''10.2.3.4''],
direction => ''out'', action =>
''allow'' }
   }

  The problem I have with this is that I need to define all options twice. 
I can''t leave this up to the module (ntp in this case) itself, as it 
doesn''t know what options to feed 

For A there''s a simple fix, namely using something similar to the 
create_resources function we could expand a hash to params (although this 
function has failed on me recently with defined types, rather than classes).

For B, it''s a little bit more difficult. Rather than agreeing on using
a
firewall/monitoring-module-agnostic setup, I''d rather propose to define
a
common interface that all firewall and monitoring modules could provide 
(albeit as a wrapper). The PHP community has done a similar thing for 
logging and for them it has worked fairly well ( 
http://www.php-fig.org/psr/3/ ). Their situation is very similar; you have 
multiple modules from different vendors that all need to output their log 
lines to somewhere, and you don''t want to have to configure a dozen log
components. If we could define a common set of parameters for both 
firewalling and monitoring, everybody could implement those for themselves. 
Of course someone could still add additional params, as long as defaults 
are provided. In order not to break Backwards Compatibility, existing 
modules could use some soft of wrapper defined type - if you already have a 
Firewall type, you''d simply add a Firewall::Wrapper type that uses the 
Firewall type. 

My 0.02 BTC

Dolf
-- Freeaqingme
       

On Monday, October 14, 2013 11:56:49 AM UTC+2, Alessandro Franceschi
wrote:>
> During the works on stdmod naming conventions I wondered if we could find 
> a non intrusive and standard way to allow management of monitoring and 
> firewalling for modules.
> Since there''s not a single solution to that and it would be quite
hard to
> find a shared approach in a reasonable time I thought that it could/should 
> be enough to define few basic parameters, to be included in a stdmod 
> compliant module, leaving the actual implementation outside the module.
>
> I thought about having parameters like:
>
>   $monitor_class         = undef,
>   $monitor_options_hash  = { } ,
>
>   $firewall_class        = undef,
>   $firewall_options_hash = { } ,
>
>
> and inside the module just this ( <%= metadata.name %> is the module
name)
> :
>
>
>   if $<%= metadata.name %>::monitor_class {
>     class { $<%= metadata.name %>::monitor_class:
>       options_hash => $<%= metadata.name %>::monitor_options_hash,
>       scope_hash   => {}, # TODO: Find a good way to inject
class'' scope
>     }
>   }
>
>   if $<%= metadata.name %>::firewall_class {
>     class { $<%= metadata.name %>::firewall_class:
>       options_hash => $<%= metadata.name
%>::firewall_options_hash,
>       scope_hash   => {},
>     }
>   }
>
> $monitor_class is therefore the name of a custom class, external to the
module, where all the monitoring stuff is managed in whatever way.
> $monitor_options_hash is an hash of custom configurations that the
$monitor_class knows how to manage .
> $scope_hash should be automatically populated by the class''s scope
variables so that informations like name of services, port numbers, names of
processes (which should have a standard naming either if expressed as class
parameters or if defined internally in the class) which are generally used for
monitoring/firewalling are automatically provided to the $monitor_class
>
> What do you think of a similar approach?
> Does it make sense?
> Would you use it in your modules? (even if you don''t plan to have
monitoring classes? They could be provided by third party modules)
> Would you make it in a different way or with different naming? How?
>
> To populate the $scope_hash it could be used something like
https://github.com/example42/puppi/blob/master/lib/puppet/parser/functions/get_class_args.rb
(is there something similar in stdlib? Would it make sense to add it, also
considering  that the original code is by Ken Barber?)
>
>
> Al
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Alessandro Franceschi

2013-Oct-16 21:52 UTC

head link

[Puppet Users] Re: A standard way to manage monitoring/firewalling in modules

Hi,

Il giorno mercoledì 16 ottobre 2013 17:12:52 UTC+2, Dolf Schimmel - 
Freeaqingme ha scritto:>
> Hi Al,
>
> I fully concur that we need some sort of standardized approach of 
> configuring firewall and monitoring rules. I''m not sure however as
to the
> approach you''re proprosing. Two reasons:
> A. The interface of all firewall and monitoring modules out there would 
> need to have a fugly interface (having two params: options_hash and 
> scope_hash). Parameters should be as explicit and descriptive as possible, 
> merely defining *_hash is asking for bad designs I think.
> B. With the architecture you propose, I would invoke a module as follows:
>     class { ntp:
>        port => 123,
>        protocol => tcp,
>        servers => [''10.1.2.3'',
''10.2.3.4'']
>        firewall_options_hash => { ''port'' => 123,
''protocol'' => ''tcp'',
> destination => [''10.1.2.3'',
''10.2.3.4''], direction => ''out'', action
=>
> ''allow'' }
>    }
>
>  The problem I have with this is that I need to define all options twice.
> I can''t leave this up to the module (ntp in this case) itself, as
it
> doesn''t know what options to feed 
>
> For A there''s a simple fix, namely using something similar to the 
> create_resources function we could expand a hash to params (although this 
> function has failed on me recently with defined types, rather than
classes).
>
Actually in my intention firewall_options_hash should contain only 
configurations related to HOW to firewall/monitor the resources (tools to 
use and module specific settings for the tool).
WHAT to monitor, all the module''s inherent data (ports, service and
process
names...) should be available directly in the module''s class scope and
can
either be passed to the defined monitor_class ( I used the scope_hash 
parameter) or retrieved directly from PuppetDB (as Nigel suggested me in a 
tweet) by the chosen monitor class.

  if $<%= metadata.name %>::monitor_class {
    class { $<%= metadata.name %>::monitor_class:
      options_hash => $<%= metadata.name %>::monitor_options_hash,
      scope_hash   => {}, # TODO: Find a good way to inject class''
scope
    }
  }

 
>  
>
> For B, it''s a little bit more difficult. Rather than agreeing on
using a
> firewall/monitoring-module-agnostic setup, I''d rather propose to
define a
> common interface that all firewall and monitoring modules could provide 
> (albeit as a wrapper). The PHP community has done a similar thing for 
> logging and for them it has worked fairly well ( 
> http://www.php-fig.org/psr/3/ ). Their situation is very similar; you 
> have multiple modules from different vendors that all need to output their 
> log lines to somewhere, and you don''t want to have to configure a
dozen log
> components. If we could define a common set of parameters for both 
> firewalling and monitoring, everybody could implement those for themselves.
> Of course someone could still add additional params, as long as defaults 
> are provided. In order not to break Backwards Compatibility, existing 
> modules could use some soft of wrapper defined type - if you already have a
> Firewall type, you''d simply add a Firewall::Wrapper type that uses
the
> Firewall type. 
>
This involves a common and shared single interface for monitoring and 
firewalling, and by experience I fear it would take quite some time to 
achieve a standard one in Puppet world, if ever.

My proposal involves the possibility of leaving to anyone the choice on how 
they want to firewall/monitor things, in their own class, leaving in the 
generic hash the possibility to set specific settings for the chosen class.
The benefit is that any module that supports those parameters can be 
monitored or firewalled by existing or future classes that to the job.

In both cases we have to add some code to a module, in a proposal like mine 
I simple provide 2 parameters, empty by default, and few optional code to 
declare any existing or future class.
Could look bad, but should work now and in any future (also one where you 
don''t need them and use other ways to retrieve the module''s
data di
firewall/monitor)

What you propose, even if probably saner and more logical, involves placing 
in modules declarations to "common" defines, for which there might be 
different implementations but for sure there must be a naming convention 
on: the name of the define (and if it''s firewall there are around many 
different modules with totally different parameters, occupying the 
namespace), the name of the parameters.
It''s basically what is currently done in ex42 modules, where
there''s a
standard, but it''s a example42-only standard and does not work with
outer
modules.

I''d pay for having  shared and common standard interfaces for
monitoring
and firewalling, but I don''t think is going to be an easy or quick
task.
 
al
>
> My 0.02 BTC
>
> Dolf
> -- Freeaqingme
>        
>
> On Monday, October 14, 2013 11:56:49 AM UTC+2, Alessandro Franceschi wrote:
>>
>> During the works on stdmod naming conventions I wondered if we could
find
>> a non intrusive and standard way to allow management of monitoring and 
>> firewalling for modules.
>> Since there''s not a single solution to that and it would be
quite hard to
>> find a shared approach in a reasonable time I thought that it
could/should
>> be enough to define few basic parameters, to be included in a stdmod 
>> compliant module, leaving the actual implementation outside the module.
>>
>> I thought about having parameters like:
>>
>>   $monitor_class         = undef,
>>   $monitor_options_hash  = { } ,
>>
>>   $firewall_class        = undef,
>>   $firewall_options_hash = { } ,
>>
>>
>> and inside the module just this ( <%= metadata.name %> is the
module
>> name):
>>
>>
>>   if $<%= metadata.name %>::monitor_class {
>>     class { $<%= metadata.name %>::monitor_class:
>>       options_hash => $<%= metadata.name
%>::monitor_options_hash,
>>       scope_hash   => {}, # TODO: Find a good way to inject
class'' scope
>>     }
>>   }
>>
>>   if $<%= metadata.name %>::firewall_class {
>>     class { $<%= metadata.name %>::firewall_class:
>>       options_hash => $<%= metadata.name
%>::firewall_options_hash,
>>       scope_hash   => {},
>>     }
>>   }
>>
>> $monitor_class is therefore the name of a custom class, external to the
module, where all the monitoring stuff is managed in whatever way.
>> $monitor_options_hash is an hash of custom configurations that the
$monitor_class knows how to manage .
>> $scope_hash should be automatically populated by the class''s
scope variables so that informations like name of services, port numbers, names
of processes (which should have a standard naming either if expressed as class
parameters or if defined internally in the class) which are generally used for
monitoring/firewalling are automatically provided to the $monitor_class
>>
>> What do you think of a similar approach?
>> Does it make sense?
>> Would you use it in your modules? (even if you don''t plan to
have monitoring classes? They could be provided by third party modules)
>> Would you make it in a different way or with different naming? How?
>>
>> To populate the $scope_hash it could be used something like
https://github.com/example42/puppi/blob/master/lib/puppet/parser/functions/get_class_args.rb
(is there something similar in stdlib? Would it make sense to add it, also
considering  that the original code is by Ken Barber?)
>>
>>
>> Al
>>
>>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

Puppet users - Oct 2013 - A standard way to manage monitoring/firewalling in modules

[Puppet Users] A standard way to manage monitoring/firewalling in modules

[Puppet Users] Re: A standard way to manage monitoring/firewalling in modules

[Puppet Users] Re: A standard way to manage monitoring/firewalling in modules