thr3ads.net - Puppet users - [Puppet Users] sysctl default values overridden by custom app values [Jan 2011]

If this information is useful, please help other people find it:
Share via:

Jake - USPS

2011-Jan-06 14:23 UTC

[Puppet Users] sysctl default values overridden by custom app values

I am pretty new to puppet and am trying to POC different scenarios I
would envision we would use this product for.  One scenario I ran into
is setting a default kernel.sem value into /etc/sysctl.conf on linux
systems that can be overridden by a custom kernel.sem value for
systems that have specific applications installed that require a
modified value for application tuning.

So, as an example, I am setting the following as the default:

kernel.sem = 16384 262144 128 1023

So all systems should have the above value if they have no
applications installed that would modify that.  On a system that would
have Oracle DB installed, the value needs to be updated to:

kernel.sem = 16384 262144 128 1024

So the last value 1023 becomes 1024.

So to define a system as being a DB system I created a fact that lists
the applications a system will be used for:

service => OracleDB, SCSP

I then have in my site.pp:

include system_defaults
if $service =~ /OracleDB/ {
  include app_oracle
}

system_defaults has:

aug_conf::etc_sysctl_conf {
      "kernel.sem": attr => "kernel.sem", value =>
"16384 262144 128
1023";
}

app_oracle has:

aug_conf::etc_sysctl_conf {
      "kernel.sem": attr => "kernel.sem", value =>
"16384 262144 128
1024";
}

The define code is:

  define etc_sysctl_conf ( $attr = "", $value = "" ) {
    # guid of this entry
    $key = "$attr"

    $context = "/files/etc/sysctl.conf"

    $path_list  = "$attr"
    $path_exact = "*[self::$attr=\"$value\"]"

    augeas { "etc_sysctl_conf/$key":
      context => "$context",
      onlyif  => "match $path_exact size==0",
      changes => [
	# insert/modify new node at the end of tree
	"set $path_list \"$value\"",
      ],
    }
  }

So naturally this conflicts.  What I am doing as a workaround is I
modified system_defaults to instead look like:

# If not OracleDB
if $service =~ /^((?!OracleDB).)*$/ {
  aug_conf::etc_sysctl_conf {
    "kernel.sem": attr => "kernel.sem", value =>
"16384 262144 128
1023";
  }
}

This then makes things not conflict.  There are some issues with the
solution I came up with though:

1) Seems awkward
2) I do not want to have to keep updating the list of applications
that will also modify kernel.sem in my system_defaults recipe as they
come about

I feel like I am not understanding something completely on how I
should be designing this for puppet.  If anyone has any suggestions
please let me know.  Also, if you require more information let me
know.

Thanks,
Jake

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Dan Bode

2011-Jan-06 20:59 UTC

head link

Re: [Puppet Users] sysctl default values overridden by custom app values

On Thu, Jan 6, 2011 at 6:23 AM, Jake - USPS <jacob.m.mccann@usps.gov>
wrote:
> I am pretty new to puppet and am trying to POC different scenarios I
> would envision we would use this product for.  One scenario I ran into
> is setting a default kernel.sem value into /etc/sysctl.conf on linux
> systems that can be overridden by a custom kernel.sem value for
> systems that have specific applications installed that require a
> modified value for application tuning.
>
> So, as an example, I am setting the following as the default:
>
> kernel.sem = 16384 262144 128 1023
>
> So all systems should have the above value if they have no
> applications installed that would modify that.  On a system that would
> have Oracle DB installed, the value needs to be updated to:
>
> kernel.sem = 16384 262144 128 1024
>
> So the last value 1023 becomes 1024.
>
> So to define a system as being a DB system I created a fact that lists
> the applications a system will be used for:
>
> service => OracleDB, SCSP
>
> I then have in my site.pp:
>
> include system_defaults
> if $service =~ /OracleDB/ {
>  include app_oracle
> }
>
> system_defaults has:
>
> aug_conf::etc_sysctl_conf {
>      "kernel.sem": attr => "kernel.sem", value =>
"16384 262144 128
> 1023";
> }
>
> app_oracle has:
>
> aug_conf::etc_sysctl_conf {
>      "kernel.sem": attr => "kernel.sem", value =>
"16384 262144 128
> 1024";
> }
>
> The define code is:
>
>  define etc_sysctl_conf ( $attr = "", $value = "" ) {
>    # guid of this entry
>    $key = "$attr"
>
>    $context = "/files/etc/sysctl.conf"
>
>    $path_list  = "$attr"
>    $path_exact = "*[self::$attr=\"$value\"]"
>
>    augeas { "etc_sysctl_conf/$key":
>      context => "$context",
>      onlyif  => "match $path_exact size==0",
>      changes => [
>        # insert/modify new node at the end of tree
>        "set $path_list \"$value\"",
>      ],
>    }
>  }
>
>Someone has developed a native sysctl type/provider, I would recommend
trying that out.

http://github.com/duritong/puppet-sysctl

> So naturally this conflicts.  What I am doing as a workaround is I
> modified system_defaults to instead look like:
>
> # If not OracleDB
> if $service =~ /^((?!OracleDB).)*$/ {
>  aug_conf::etc_sysctl_conf {
>    "kernel.sem": attr => "kernel.sem", value =>
"16384 262144 128
> 1023";
>  }
> }
>
> This then makes things not conflict.  There are some issues with the
> solution I came up with though:
>
> 1) Seems awkward
> 2) I do not want to have to keep updating the list of applications
> that will also modify kernel.sem in my system_defaults recipe as they
> come about
>
I agree that this approach is less than idea.

Typically, if something needs to be overridable in Puppet, the best way is
to either use class inheritance, or expose a class parameter.

1. How to model with inheritance:

I have a base platform, which specifies my default settings for all
machines:

class role::base(
...
) {
  sysctl{sem:
    value => ''16384 262144 128 1023''
  }
  class {[''ntp'', ''mail'',
''monitoring'']:}
}

I have an oracle role that can overwrite some of the resources from its
parent class

class role::oracle(
...
) inherits role::base {
  Sysctl[''sem''] {
    value => ''16384 262144 128 1024''
  }
  class { ''oracle'':}
}

This approach does have some limitations (namely that a node may represent
more than one role)

2. Managing overridable configuration settings as parameters:

In my base role, I expose the ability to set sem in sysctl as a part of the
API

class role::base(
  $sysctl_sem = ''16384 262144 128 1023''
) {
  sysctl{sem:
    value => $sysctl_sem
  }
}

now, my oracle role should also declare its base configuration, and specify
a value for sysctl_sem:

class role::oracle(
...
) {
  class {''platform'':
    sysctl_sem => ''16384 262144 128 1024''
  }
}

I would tend to use something like #2, except declare both role::oracle and
role::base from an external node classifier (although they do not currently
support param classes, patch pending ;) ) like the Dashboard so that I can
specify the data overrides externally to Puppet.


> I feel like I am not understanding something completely on how I
> should be designing this for puppet.  If anyone has any suggestions
> please let me know.  Also, if you require more information let me
> know.
>
> Thanks,
> Jake
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
>
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Jake - USPS

2011-Jan-07 16:03 UTC

head link

[Puppet Users] Re: sysctl default values overridden by custom app values

Dan,

First, thanks for the sysctl type/provider.  I haven''t tried it yet
but definitely plan to do so after I get this working.  Augeas can be
a bit slow checking/editing files (about 1 sec per item I set) so this
could definitely not only make things easier but also faster.

Also, thanks for explaining a couple of solutions for me to try out.
I sort of understand of what you are doing, but am missing something
cause I am not able to get it to work.  Is there some additional
documentation on doing this?  I am unfamiliar with classes that you
parameterize and can''t find any reference to it in "Pulling
Strings
with Puppet".  I''d like to read up on this and try to learn more
from
it.

I tried to implement sort of what you are showing me with solution #2,
but I am doing something wrong because it won''t work.  So I think
I''m
still missing some understanding about this.

manifests/site.pp:
node default {
  include system_defaults

  if $service =~ /OracleDB/ {
    include app_oracle
    notice ("I am an Oracle Server")
  }
}

modules/system_defaults/manifests/init.pp
include aug_conf
class system_defaults ( $sysctl_sem = "16384 262144 128 1023" ) {
  aug_conf::etc_sysctl_conf {
    "kernel.sem": attr => "kernel.sem", value =>
$sysctl_sem;
  }
}

modules/app_oracle/manifests/init.pp
class app_oracle {
	  class { system_defaults:
	    sysctl_sem => "16384 262144 128 1024";
	  }
}

err: Could not find class app_oracle at /etc/puppet/manifests/site.pp:
5 on node test1.usps.gov

If I take out

	  class { system_defaults:
	    sysctl_sem => "16384 262144 128 1024";
	  }

from app_oracle it is able to "find" app_oracle again but
doesn''t have
what I am trying to do (since I took it out), but it does set it to
the value I specify as default from system_defaults.

Thanks,
Jake

On Jan 6, 2:59 pm, Dan Bode <d...@puppetlabs.com>
wrote:> On Thu, Jan 6, 2011 at 6:23 AM, Jake - USPS <jacob.m.mcc...@usps.gov>
wrote:
>
>
>
>
>
>
>
>
>
> > I am pretty new to puppet and am trying to POC different scenarios I
> > would envision we would use this product for.  One scenario I ran into
> > is setting a default kernel.sem value into /etc/sysctl.conf on linux
> > systems that can be overridden by a custom kernel.sem value for
> > systems that have specific applications installed that require a
> > modified value for application tuning.
>
> > So, as an example, I am setting the following as the default:
>
> > kernel.sem = 16384 262144 128 1023
>
> > So all systems should have the above value if they have no
> > applications installed that would modify that.  On a system that would
> > have Oracle DB installed, the value needs to be updated to:
>
> > kernel.sem = 16384 262144 128 1024
>
> > So the last value 1023 becomes 1024.
>
> > So to define a system as being a DB system I created a fact that lists
> > the applications a system will be used for:
>
> > service => OracleDB, SCSP
>
> > I then have in my site.pp:
>
> > include system_defaults
> > if $service =~ /OracleDB/ {
> >  include app_oracle
> > }
>
> > system_defaults has:
>
> > aug_conf::etc_sysctl_conf {
> >      "kernel.sem": attr => "kernel.sem", value
=> "16384 262144 128
> > 1023";
> > }
>
> > app_oracle has:
>
> > aug_conf::etc_sysctl_conf {
> >      "kernel.sem": attr => "kernel.sem", value
=> "16384 262144 128
> > 1024";
> > }
>
> > The define code is:
>
> >  define etc_sysctl_conf ( $attr = "", $value = ""
) {
> >    # guid of this entry
> >    $key = "$attr"
>
> >    $context = "/files/etc/sysctl.conf"
>
> >    $path_list  = "$attr"
> >    $path_exact = "*[self::$attr=\"$value\"]"
>
> >    augeas { "etc_sysctl_conf/$key":
> >      context => "$context",
> >      onlyif  => "match $path_exact size==0",
> >      changes => [
> >        # insert/modify new node at the end of tree
> >        "set $path_list \"$value\"",
> >      ],
> >    }
> >  }
>
> Someone has developed a native sysctl type/provider, I would recommend
> trying that out.
>
> http://github.com/duritong/puppet-sysctl
>
>
>
>
>
>
>
>
>
> > So naturally this conflicts.  What I am doing as a workaround is I
> > modified system_defaults to instead look like:
>
> > # If not OracleDB
> > if $service =~ /^((?!OracleDB).)*$/ {
> >  aug_conf::etc_sysctl_conf {
> >    "kernel.sem": attr => "kernel.sem", value
=> "16384 262144 128
> > 1023";
> >  }
> > }
>
> > This then makes things not conflict.  There are some issues with the
> > solution I came up with though:
>
> > 1) Seems awkward
> > 2) I do not want to have to keep updating the list of applications
> > that will also modify kernel.sem in my system_defaults recipe as they
> > come about
>
> I agree that this approach is less than idea.
>
> Typically, if something needs to be overridable in Puppet, the best way is
> to either use class inheritance, or expose a class parameter.
>
> 1. How to model with inheritance:
>
> I have a base platform, which specifies my default settings for all
> machines:
>
> class role::base(
> ...
> ) {
>   sysctl{sem:
>     value => ''16384 262144 128 1023''
>   }
>   class {[''ntp'', ''mail'',
''monitoring'']:}
>
> }
>
> I have an oracle role that can overwrite some of the resources from its
> parent class
>
> class role::oracle(
> ...
> ) inherits role::base {
>   Sysctl[''sem''] {
>     value => ''16384 262144 128 1024''
>   }
>   class { ''oracle'':}
>
> }
>
> This approach does have some limitations (namely that a node may represent
> more than one role)
>
> 2. Managing overridable configuration settings as parameters:
>
> In my base role, I expose the ability to set sem in sysctl as a part of the
> API
>
> class role::base(
>   $sysctl_sem = ''16384 262144 128 1023''
> ) {
>   sysctl{sem:
>     value => $sysctl_sem
>   }
>
> }
>
> now, my oracle role should also declare its base configuration, and specify
> a value for sysctl_sem:
>
> class role::oracle(
> ...
> ) {
>   class {''platform'':
>     sysctl_sem => ''16384 262144 128 1024''
>   }
>
> }
>
> I would tend to use something like #2, except declare both role::oracle and
> role::base from an external node classifier (although they do not currently
> support param classes, patch pending ;) ) like the Dashboard so that I can
> specify the data overrides externally to Puppet.
>
>
>
>
>
>
>
> > I feel like I am not understanding something completely on how I
> > should be designing this for puppet.  If anyone has any suggestions
> > please let me know.  Also, if you require more information let me
> > know.
>
> > Thanks,
> > Jake
>
> > --
> > You received this message because you are subscribed to the Google
Groups
> > "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@google
groups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Jake - USPS

2011-Jan-07 18:18 UTC

head link

[Puppet Users] Re: sysctl default values overridden by custom app values

A couple updates.

First, I''ve been finding some other documentation on parameterized
classes.  It seems this was first introduced in 2.6.0, which is the
version I am using.  I''m going to start and try updating to 2.6.3 to
see if this gets addressed in a possible bugfix.  Please feel free to
comment on the chance I am running into a bug or I am in fact doing
something wrong.  More documentation is always a plus as what I found
is a very simple example and some posts of others using this feature
and running into some issues.

As for my comment on Augeas taking a while to run, I found that the
provider has ''incl'' and ''lens''.  I used
those and a run that was
taking 20sec to complete (no changes, just verifying) now takes 2-3
seconds.  :)

Thanks,
Jake

On Jan 7, 10:03 am, Jake - USPS <jacob.m.mcc...@usps.gov>
wrote:> Dan,
>
> First, thanks for the sysctl type/provider.  I haven''t tried it
yet
> but definitely plan to do so after I get this working.  Augeas can be
> a bit slow checking/editing files (about 1 sec per item I set) so this
> could definitely not only make things easier but also faster.
>
> Also, thanks for explaining a couple of solutions for me to try out.
> I sort of understand of what you are doing, but am missing something
> cause I am not able to get it to work.  Is there some additional
> documentation on doing this?  I am unfamiliar with classes that you
> parameterize and can''t find any reference to it in "Pulling
Strings
> with Puppet".  I''d like to read up on this and try to learn
more from
> it.
>
> I tried to implement sort of what you are showing me with solution #2,
> but I am doing something wrong because it won''t work.  So I think
I''m
> still missing some understanding about this.
>
> manifests/site.pp:
> node default {
>   include system_defaults
>
>   if $service =~ /OracleDB/ {
>     include app_oracle
>     notice ("I am an Oracle Server")
>   }
>
> }
>
> modules/system_defaults/manifests/init.pp
> include aug_conf
> class system_defaults ( $sysctl_sem = "16384 262144 128 1023" ) {
>   aug_conf::etc_sysctl_conf {
>     "kernel.sem": attr => "kernel.sem", value =>
$sysctl_sem;
>   }
>
> }
>
> modules/app_oracle/manifests/init.pp
> class app_oracle {
>           class { system_defaults:
>             sysctl_sem => "16384 262144 128 1024";
>           }
>
> }
>
> err: Could not find class app_oracle at /etc/puppet/manifests/site.pp:
> 5 on node test1.usps.gov
>
> If I take out
>
>           class { system_defaults:
>             sysctl_sem => "16384 262144 128 1024";
>           }
>
> from app_oracle it is able to "find" app_oracle again but
doesn''t have
> what I am trying to do (since I took it out), but it does set it to
> the value I specify as default from system_defaults.
>
> Thanks,
> Jake
>
> On Jan 6, 2:59 pm, Dan Bode <d...@puppetlabs.com> wrote:
>
>
>
>
>
>
>
> > On Thu, Jan 6, 2011 at 6:23 AM, Jake - USPS
<jacob.m.mcc...@usps.gov> wrote:
>
> > > I am pretty new to puppet and am trying to POC different
scenarios I
> > > would envision we would use this product for.  One scenario I ran
into
> > > is setting a default kernel.sem value into /etc/sysctl.conf on
linux
> > > systems that can be overridden by a custom kernel.sem value for
> > > systems that have specific applications installed that require a
> > > modified value for application tuning.
>
> > > So, as an example, I am setting the following as the default:
>
> > > kernel.sem = 16384 262144 128 1023
>
> > > So all systems should have the above value if they have no
> > > applications installed that would modify that.  On a system that
would
> > > have Oracle DB installed, the value needs to be updated to:
>
> > > kernel.sem = 16384 262144 128 1024
>
> > > So the last value 1023 becomes 1024.
>
> > > So to define a system as being a DB system I created a fact that
lists
> > > the applications a system will be used for:
>
> > > service => OracleDB, SCSP
>
> > > I then have in my site.pp:
>
> > > include system_defaults
> > > if $service =~ /OracleDB/ {
> > >  include app_oracle
> > > }
>
> > > system_defaults has:
>
> > > aug_conf::etc_sysctl_conf {
> > >      "kernel.sem": attr => "kernel.sem",
value => "16384 262144 128
> > > 1023";
> > > }
>
> > > app_oracle has:
>
> > > aug_conf::etc_sysctl_conf {
> > >      "kernel.sem": attr => "kernel.sem",
value => "16384 262144 128
> > > 1024";
> > > }
>
> > > The define code is:
>
> > >  define etc_sysctl_conf ( $attr = "", $value =
"" ) {
> > >    # guid of this entry
> > >    $key = "$attr"
>
> > >    $context = "/files/etc/sysctl.conf"
>
> > >    $path_list  = "$attr"
> > >    $path_exact = "*[self::$attr=\"$value\"]"
>
> > >    augeas { "etc_sysctl_conf/$key":
> > >      context => "$context",
> > >      onlyif  => "match $path_exact size==0",
> > >      changes => [
> > >        # insert/modify new node at the end of tree
> > >        "set $path_list \"$value\"",
> > >      ],
> > >    }
> > >  }
>
> > Someone has developed a native sysctl type/provider, I would recommend
> > trying that out.
>
> >http://github.com/duritong/puppet-sysctl
>
> > > So naturally this conflicts.  What I am doing as a workaround is
I
> > > modified system_defaults to instead look like:
>
> > > # If not OracleDB
> > > if $service =~ /^((?!OracleDB).)*$/ {
> > >  aug_conf::etc_sysctl_conf {
> > >    "kernel.sem": attr => "kernel.sem",
value => "16384 262144 128
> > > 1023";
> > >  }
> > > }
>
> > > This then makes things not conflict.  There are some issues with
the
> > > solution I came up with though:
>
> > > 1) Seems awkward
> > > 2) I do not want to have to keep updating the list of
applications
> > > that will also modify kernel.sem in my system_defaults recipe as
they
> > > come about
>
> > I agree that this approach is less than idea.
>
> > Typically, if something needs to be overridable in Puppet, the best
way is
> > to either use class inheritance, or expose a class parameter.
>
> > 1. How to model with inheritance:
>
> > I have a base platform, which specifies my default settings for all
> > machines:
>
> > class role::base(
> > ...
> > ) {
> >   sysctl{sem:
> >     value => ''16384 262144 128 1023''
> >   }
> >   class {[''ntp'', ''mail'',
''monitoring'']:}
>
> > }
>
> > I have an oracle role that can overwrite some of the resources from
its
> > parent class
>
> > class role::oracle(
> > ...
> > ) inherits role::base {
> >   Sysctl[''sem''] {
> >     value => ''16384 262144 128 1024''
> >   }
> >   class { ''oracle'':}
>
> > }
>
> > This approach does have some limitations (namely that a node may
represent
> > more than one role)
>
> > 2. Managing overridable configuration settings as parameters:
>
> > In my base role, I expose the ability to set sem in sysctl as a part
of the
> > API
>
> > class role::base(
> >   $sysctl_sem = ''16384 262144 128 1023''
> > ) {
> >   sysctl{sem:
> >     value => $sysctl_sem
> >   }
>
> > }
>
> > now, my oracle role should also declare its base configuration, and
specify
> > a value for sysctl_sem:
>
> > class role::oracle(
> > ...
> > ) {
> >   class {''platform'':
> >     sysctl_sem => ''16384 262144 128 1024''
> >   }
>
> > }
>
> > I would tend to use something like #2, except declare both
role::oracle and
> > role::base from an external node classifier (although they do not
currently
> > support param classes, patch pending ;) ) like the Dashboard so that I
can
> > specify the data overrides externally to Puppet.
>
> > > I feel like I am not understanding something completely on how I
> > > should be designing this for puppet.  If anyone has any
suggestions
> > > please let me know.  Also, if you require more information let me
> > > know.
>
> > > Thanks,
> > > Jake
>
> > > --
> > > You received this message because you are subscribed to the
Google Groups
> > > "Puppet Users" group.
> > > To post to this group, send email to
puppet-users@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@google
groups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/puppet-users?hl=en.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.

Puppet users - Jan 2011 - sysctl default values overridden by custom app values

[Puppet Users] sysctl default values overridden by custom app values

Re: [Puppet Users] sysctl default values overridden by custom app values

[Puppet Users] Re: sysctl default values overridden by custom app values

[Puppet Users] Re: sysctl default values overridden by custom app values