[Puppet Users] Re: Storing puppet info in a database

Kenneth Holter
Fri, 04 Sep 2009 07:44:03 -0000

Thanks for all the replies - this looks very promising wrt gaining more
control over our node definitions (and inventory). It seems like having the
puppetmaster fetch client info from a database is a far better approach than
having the puppetmaster push info into one.

If I understand correctly, stored configs simply store the puppet
client''s
configuration in a database, while external nodes classification allows for
generating configurations based on contents in for example a database.
In our case, we have a Satellite server that store information in an Oracle
database, so incoorporating puppet info into that database sound very
elegant. This way we''ll have one database that contains pretty much
everything.

As far as I can see, two questions still remain:

   1. What''s the most practical way of updating external
node''s database,
   i.e. how to I create new nodes? I guess having a script that queries us for
   misc info about new nodes, and updates the database accordingly, seems like
   the most neat way of doing this. How are others doing this? Does such a
   script allready exist?
   2. Can I have the puppet clients store info (for example hardware info
   and such) in the same database? Perhaps this is where exported resources
   comes in handy.

By the way, what are the arguments to choose stored configs over external
nodes classification?

Best regards,
Kenneth


On 9/2/09, Joel Krauska <jkrauska@gmail.com>
wrote:
>
>
> Puppet already stores a lot of this data in YAML.
>
> Take a look at
> /var/lib/puppet/yaml/facts/
>
> Here''s a quick and dirty perl tool.
> (it needs the YAML perl module to work)
>
> #!/usr/bin/perl
> use YAML;
>
> $facts =''/var/lib/puppet/yaml/facts/*.yaml'';
>
> foreach my $file ( glob $facts ) {
>    my $data = YAML::LoadFile($file);
>    my $values = $data->{values};
>    my $name = $data->{name};
>    $kernellist{$values->{kernelrelease}}++;
>    $memlist{$values->{memorysize}}++;
> }
>
> print "---------------------------------------------\n";
> print "Kernel Counts\n";
> print "---------------------------------------------\n";
> foreach my $item (sort keys %kernellist) {
>    print "$kernellist{$item} \t $item\n";
> }
>
>
> print "\n\n";
> print "---------------------------------------------\n";
> print "RAM Counts\n";
> print "---------------------------------------------\n";
> foreach my $item (sort keys %memlist) {
>
>    print "$memlist{$item} \t $item\n";
> }
>
>
>
>
> example output:
> ---------------------------------------------
> Kernel Counts
> ---------------------------------------------
> 1        2.6.16.27-0.9-smp
> 37       2.6.16.53-0.16-smp
> 9        2.6.16.53-0.8-smp
> 12       2.6.16.54-0.2.5-smp
> 3        2.6.20-17-server
> 8        2.6.24-16-server
> 12       2.6.24-19-server
>
> etc...
>
> You can create custom facts to grab the type of information you want
> and puppet will store that in YAML too.
>
> Happy puppeting.
>
> --joel
>
>
>
> On Tue, Sep 1, 2009 at 6:28 PM, Ohad Levy<ohadlevy@gmail.com> wrote:
> > and, I forgot, you are not forced to use store configs just for the
> > inventory.
> >
> > On Wed, Sep 2, 2009 at 9:27 AM, Ohad Levy <ohadlevy@gmail.com>
wrote:
> >>
> >> I also recommend using external nodes and facts as inventory.
> >>
> >> if you are looking for an active project (unlike puppetshow and
> iclassiy)
> >> you might consider gni.
> >> it already does what you want, and more :)
> >> http://wiki.github.com/ohadlevy/gni
> >>
> >> cheers,
> >> Ohad
> >>
> >> On Wed, Sep 2, 2009 at 1:00 AM, Kenneth Holter
<kenneho.ndu@gmail.com>
> >> wrote:
> >>>
> >>> Hi all.
> >>> We have a bunch of RHEL servers running Puppet. They are also
connected
> >>> to our Red Hat Satellite server.
> >>> Currently we don''t have any master documentation
system that stores all
> >>> relevant information (i.e. type of server, hardware info,
linux
> >>> configuration, etc) about the servers. So what I''d
like to do is
> implement
> >>> some sort of system that can hold all this information. There
are a lot
> of
> >>> asset management products that I guess could get us a long way
(and our
> >>> organization is probably going to buy one soon), but
I''m not sure how
> >>> flexible these would be with regards to storing custom
information and
> such.
> >>> The asset management product will be administred by someone
else within
> the
> >>> organisation, and having that person implement linux specific
info
> (such as
> >>> puppet) and stuff may not be very realistic.
> >>> First of all, how to other people handle this sort of thing?
Storing
> info
> >>> in excel-files is no good, since the information will be
outdated by
> the
> >>> time I press "save". We need a more or less
automatic way of doing
> this, but
> >>> is asset management products the way to go? Currently, our
node
> definition
> >>> in Puppet holds the most current infomation when it comes to
> confuguration,
> >>> while Satellite server holds a lot of other information.
I''m hoping to
> >>> integrate all our server info (including puppet) into a
database
> (preferably
> >>> the Satellite server database on Oracle), so that we can query
it for
> info.
> >>> I''ve been playing around with two ideas for
automating system
> >>> documentation:
> >>> 1) Parse the puppet manifests (and node definitions) and
organize the
> >>> info in a database
> >>> 2) Include a logging functionality in the puppet manifests, so
that
> when
> >>> the clients implement the configuration it logs a specific
text to
> syslog.
> >>> The syslog message is sent over to our log host, which can
parse it and
> >>> store the information in a database.
> >>> Maybe the first option is allready implemented? Anyway, have
anyone
> >>> implemented such mechanisms?
> >>> Best regards,
> >>> Kenneth Holter
> >>>
> >>
> >
> >
> > >
> >
>
> >