Puppet users - Jul 2007 - Getting a list of managed machines on the puppetmaster.

Is there an easy way to list which puppet daemons a puppetmaster controls?

... and from that, is there a way of getting the puppetmaster to store a 
copy of their compiled configuration somewhere?

Thanks,

mike

On Jul 5, 2007, at 12:22 PM, Mike Pountney wrote:
>
> Is there an easy way to list which puppet daemons a puppetmaster  
> controls?
>
> ... and from that, is there a way of getting the puppetmaster to  
> store a
> copy of their compiled configuration somewhere?
If your clients are reporting, then ls /var/puppet/reports is a good  
way to get a list of hosts, although it''s not necessarily up to date.

I''m working on other ways, but any recommendations would be welcome.

  --
  Don''t throw away the old bucket until you know whether the new one  
holds
  water.  -- Swedish Proverb
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

--On Thursday, July 05, 2007 1:26 PM -0500 Luke Kanies <luke@madstop.com> 
wrote:
>> Is there an easy way to list which puppet daemons a puppetmaster
>> controls?
>>
>> ... and from that, is there a way of getting the puppetmaster to
>> store a
>> copy of their compiled configuration somewhere?
>
> If your clients are reporting, then ls /var/puppet/reports is a good
> way to get a list of hosts, although it''s not necessarily up to
date.
>
> I''m working on other ways, but any recommendations would be
welcome.
Doesn''t the store report store the yaml files?

One way we could do it is to add the functionality where as when you sign a
machine, that machines info get''s inserted into a sqlite database or
some other ruby embedded db.
   
  

Digant C Kasundra <digant@stanford.edu> wrote:
  --On Thursday, July 05, 2007 1:26 PM -0500 Luke Kanies 
wrote:
>> Is there an easy way to list which puppet daemons a puppetmaster
>> controls?
>>
>> ... and from that, is there a way of getting the puppetmaster to
>> store a
>> copy of their compiled configuration somewhere?
>
> If your clients are reporting, then ls /var/puppet/reports is a good
> way to get a list of hosts, although it''s not necessarily up to
date.
>
> I''m working on other ways, but any recommendations would be
welcome.
Doesn''t the store report store the yaml files?


_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users


       
---------------------------------
Be a better Globetrotter. Get better travel answers from someone who knows.
Yahoo! Answers - Check it out.
 
---------------------------------
Bored stiff? Loosen up...
Download and play hundreds of games for free on Yahoo! Games.

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I remember from a few months back someone writing a Rails app that
parsed the reports data into a nice overview of the puppet situation.

Any word on this? It looked really nice!

Micah

Brent Clements wrote:
> One way we could do it is to add the functionality where as when you
> sign a machine, that machines info get''s inserted into a sqlite
database
> or some other ruby embedded db.
>  
> 
> 
> */Digant C Kasundra <digant@stanford.edu>/* wrote:
> 
>     --On Thursday, July 05, 2007 1:26 PM -0500 Luke Kanies
>     wrote:
> 
>     >> Is there an easy way to list which puppet daemons a
puppetmaster
>     >> controls?
>     >>
>     >> ... and from that, is there a way of getting the puppetmaster
to
>     >> store a
>     >> copy of their compiled configuration somewhere?
>     >
>     > If your clients are reporting, then ls /var/puppet/reports is a
good
>     > way to get a list of hosts, although it''s not necessarily
up to date.
>     >
>     > I''m working on other ways, but any recommendations would
be welcome.
> 
>     Doesn''t the store report store the yaml files?
> 
> 
>     _______________________________________________
>     Puppet-users mailing list
>     Puppet-users@madstop.com
>     https://mail.madstop.com/mailman/listinfo/puppet-users
> 
> 
> ------------------------------------------------------------------------
> Be a better Globetrotter. Get better travel answers
>
<http://us.rd.yahoo.com/evt=48254/*http://answers.yahoo.com/dir/_ylc=X3oDMTI5MGx2aThyBF9TAzIxMTU1MDAzNTIEX3MDMzk2NTQ1MTAzBHNlYwNCQUJwaWxsYXJfTklfMzYwBHNsawNQcm9kdWN0X3F1ZXN0aW9uX3BhZ2U-?link=list&sid=396545469>from
> someone who knows.
> Yahoo! Answers - Check it out.
> 
> ------------------------------------------------------------------------
> Bored stiff?
<http://us.rd.yahoo.com/evt=49935/*http://games.yahoo.com>
> Loosen up...
> Download and play hundreds of games for free
> <http://us.rd.yahoo.com/evt=49935/*http://games.yahoo.com> on Yahoo!
Games.
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGj8gL9n4qXRzy1ioRApLWAKCCBSfLCg05+LeCfPR1ApZXbGepmgCfe9zT
+Sm6+9j73ehiSfkvosGxqGA=9BkT
-----END PGP SIGNATURE-----

On Jul 7, 2007, at 3:11 AM, Digant C Kasundra wrote:
> Doesn''t the store report store the yaml files?
Yep, and it would certainly be easy to use that to build a list of  
hosts.  It''s just that no such list-builder exists today.  It would  
be excellent if someone built an html report that would autogenerate  
an html list of hosts, along with pointers to the most recent report  
and maybe to rrd graphs if the graphs are enabled.

  --
  No matter how rich you become, how famous or powerful, when you die
  the size of your funeral will still pretty much depend on the
  weather.        -- Michael Pritchard
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Jul 7, 2007, at 7:11 AM, Brent Clements wrote:
> One way we could do it is to add the functionality where as when  
> you sign a machine, that machines info get''s inserted into a
sqlite
> database or some other ruby embedded db.
While it''s important to know which machines currently have a valid  
signed cert, people are probably generally more interested in knowing  
the current status of clients -- when they last contacted, what their  
run status was, etc.  In other words, it makes more sense to key off  
of reports, rather than certificates.

As to whether the information should be in a database, I''m not really  
sure.  I''ve been kind of wrestling with how much I''m willing
to
require puppetmasterd have a database.  At this point, you can use  
it, but you''re not required to.  I doubt I''m willing to have
db
integration except through an ORM like ActiveRecord, which makes it  
likely that requiring a db would require ActiveRecord, which is a  
helluva prerequisite.  This is what PuppetShow is supposed to be --  
the nice web front end and db repository for all Puppet data -- but  
none of us have apparently had the time to really develop it,  
unfortunately.

Heck, I can see a lot of need for a database on the client, too, but  
I''m in the same prereq conundrom.  That, and just not enough time to  
develop it all.

  --
  Nature and nature''s laws lay hid in night,
  God said, "Let Newton be," and all was light.

  It did not last; the devil howling "Ho!
  Let Einstein be!" restored the status quo.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On Jul 7, 2007, at 12:06 PM, Micah Anderson wrote:
> I remember from a few months back someone writing a Rails app that
> parsed the reports data into a nice overview of the puppet situation.
>
> Any word on this? It looked really nice!
It seems you must mean PuppetShow, but I don''t remember it  
integrating with reports.  That''s been my goal for ages -- actually,  
I really want the reports themselves in the ActiveRecord db -- but I  
haven''t had the time to do it.

  --
  A complex system that works is invariably found to have evolved from a
  simple system that works.       -- John Gaule
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

So is feature development on the OpenSource version of Puppet pretty much
stopped at this point(beyond bug fixes) unless others join in the development?
  

Luke Kanies <luke@madstop.com> wrote:
  On Jul 7, 2007, at 7:11 AM, Brent Clements wrote:
> One way we could do it is to add the functionality where as when 
> you sign a machine, that machines info get''s inserted into a
sqlite
> database or some other ruby embedded db.
While it''s important to know which machines currently have a valid 
signed cert, people are probably generally more interested in knowing 
the current status of clients -- when they last contacted, what their 
run status was, etc. In other words, it makes more sense to key off 
of reports, rather than certificates.

As to whether the information should be in a database, I''m not really 
sure. I''ve been kind of wrestling with how much I''m willing to
require puppetmasterd have a database. At this point, you can use 
it, but you''re not required to. I doubt I''m willing to have db
integration except through an ORM like ActiveRecord, which makes it 
likely that requiring a db would require ActiveRecord, which is a 
helluva prerequisite. This is what PuppetShow is supposed to be -- 
the nice web front end and db repository for all Puppet data -- but 
none of us have apparently had the time to really develop it, 
unfortunately.

Heck, I can see a lot of need for a database on the client, too, but 
I''m in the same prereq conundrom. That, and just not enough time to 
develop it all.

--
Nature and nature''s laws lay hid in night,
God said, "Let Newton be," and all was light.

It did not last; the devil howling "Ho!
Let Einstein be!" restored the status quo.
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com


_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users


       
---------------------------------
Be a better Globetrotter. Get better travel answers from someone who knows.
Yahoo! Answers - Check it out.

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Jul 8, 2007, at 5:12 PM, Brent Clements wrote:
> So is feature development on the OpenSource version of Puppet  
> pretty much stopped at this point(beyond bug fixes) unless others  
> join in the development?
First, there is only one version of Puppet, the publicly available  
GPL''ed version.  This has always been the case, and I''ve
stated
publicly multiple times that this will always be the case.

Second, no, development has not stopped at all.  Check the changelog  
for 0.23.0 if you doubt me. It''s just that my development priorities  
are not those of the community, necessarily, and I generally focus on  
work I''m being paid for, then work I want done, then work the  
community wants done.  Often, if the community wants something enough  
it will get pushed higher on my list, but not always.

The real problem here is that there is just too much work for even  
two full time developers, yet I''m currently only a part-time  
developer on Puppet because I spend at least half my time building  
community, interacting with clients, and working on documentation and  
the like.  Fortunately I''ve just hired a full-time developer, so work  
should speed up some, but even one more developer won''t make a huge  
dent, I fear, just because the problem is so large.

The reason I keep making comments about agreeing that a feature is  
good but that I can''t get to it is because I just don''t have
the time
to develop all of the features that the community wants.  If you want  
it badly enough, there are three very good ways for you to get a  
feature:  Develop it yourself, pay me (or someone else in the  
community) to develop it, or convince someone else in the community  
to develop it.

I always do my best to get useful features in place -- witness the  
fact that I just added module autoloading on Friday -- but my  
priorities and yours can never quite perfectly mesh.

  --
  America believes in education: the average professor earns more money
  in a year than a professional athlete earns in a whole week.
          -- Evan Esar
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

For anyone interested, here is a report I created off of the store report 
that I call "lastcheck."  It is very, very simple but useful until a 
larger, all-encompassing report can be created.

require ''puppet''

Puppet::Network::Handler.report.newreport(:lastcheck) do
    Puppet.config.use(:reporting)

    desc "Record the last time a host checked in."

    def process
        # We don''t want any tracking back in the fs.  Unlikely, but
there
        # you go.
        client = self.host.gsub("..",".")

        file = File.join(Puppet[:reportdir], client)

        now = Time.now.getlocal
        begin
            File.open(file, "w", 0640) do |f|
                f.print now.year.to_s + "-" + now.month.to_s +
"-" +
                    now.day.to_s + " " + now.hour.to_s + ":"
+
                    now.min.to_s + "\n"
            end
        rescue => detail
            if Puppet[:trace]
                puts detail.backtrace
            end
            Puppet.warning "Could not write report for %s at %s: %s" %
                [client, file, detail]
        end

        # Only testing cares about the return value
        return file
    end
end

* Luke Kanies <luke@madstop.com> [070708 11:29]:
> On Jul 7, 2007, at 12:06 PM, Micah Anderson wrote:
> 
> > I remember from a few months back someone writing a Rails app that
> > parsed the reports data into a nice overview of the puppet situation.
> >
> > Any word on this? It looked really nice!
> 
> It seems you must mean PuppetShow, but I don''t remember it  
> integrating with reports.  That''s been my goal for ages --
actually,
> I really want the reports themselves in the ActiveRecord db -- but I  
> haven''t had the time to do it.

For some reason I thought it was not PuppetShow, but something else that
someone at an organization or company was working on. There was a
mock-up image that was passed around on IRC that looked really great,
and it sounded like it was loosely based on PuppetShow but was a
complete re-write. All of this I am pulling out of my really fuzzy
memory.

micah

You probably mean this: http://dre.gs/puppetstore.png

It still works on my development box, but sadly I had to abandon it because
of the time required to make using a webservice a viable option in puppet.
(Pluggable abstraction layer for storage sources, etc.)  I may be able to
get back to it sometime in the future, but unfortunately my real job is very
unpredictably distracting.

-Blake

On 7/12/07, Micah Anderson <micah@riseup.net>
wrote:
>
> * Luke Kanies <luke@madstop.com> [070708 11:29]:
> > On Jul 7, 2007, at 12:06 PM, Micah Anderson wrote:
> >
> > > I remember from a few months back someone writing a Rails app
that
> > > parsed the reports data into a nice overview of the puppet
situation.
> > >
> > > Any word on this? It looked really nice!
> >
> > It seems you must mean PuppetShow, but I don''t remember it
> > integrating with reports.  That''s been my goal for ages --
actually,
> > I really want the reports themselves in the ActiveRecord db -- but I
> > haven''t had the time to do it.
>
>
> For some reason I thought it was not PuppetShow, but something else that
> someone at an organization or company was working on. There was a
> mock-up image that was passed around on IRC that looked really great,
> and it sounded like it was loosely based on PuppetShow but was a
> complete re-write. All of this I am pulling out of my really fuzzy
> memory.
>
> micah
> _______________________________________________
> Puppet-users mailing list
> Puppet-users@madstop.com
> https://mail.madstop.com/mailman/listinfo/puppet-users
>

_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Wed, 25 Jul 2007 15:25:00 -0700, Blake B wrote:
> You probably mean this: http://dre.gs/puppetstore.png
Aha! That *is* it!
 
> It still works on my development box, but sadly I had to abandon it
> because of the time required to make using a webservice a viable option
> in puppet. (Pluggable abstraction layer for storage sources, etc.)  I
> may be able to get back to it sometime in the future, but unfortunately
> my real job is very unpredictably distracting.
Hmm, thanks for the update, but I''m sad to hear that it got stuck. I 
would help if I had the skills, but my ruby is not even close yet.

Anyone else want to pick this up, dust it off, polish it up and make it 
happen?

Micah
> On 7/12/07, Micah Anderson <micah@riseup.net> wrote:
>>
>> * Luke Kanies <luke@madstop.com> [070708 11:29]:
>> > On Jul 7, 2007, at 12:06 PM, Micah Anderson wrote:
>> >
>> > > I remember from a few months back someone writing a Rails app
that
>> > > parsed the reports data into a nice overview of the puppet
>> > > situation.
>> > >
>> > > Any word on this? It looked really nice!
>> >
>> > It seems you must mean PuppetShow, but I don''t remember
it
>> > integrating with reports.  That''s been my goal for ages
-- actually,
>> > I really want the reports themselves in the ActiveRecord db -- but
I
>> > haven''t had the time to do it.
>>
>>
>> For some reason I thought it was not PuppetShow, but something else
>> that someone at an organization or company was working on. There was a
>> mock-up image that was passed around on IRC that looked really great,
>> and it sounded like it was loosely based on PuppetShow but was a
>> complete re-write. All of this I am pulling out of my really fuzzy
>> memory.
>>

On Jul 27, 2007, at 11:33 AM, micah wrote:
>
> Hmm, thanks for the update, but I''m sad to hear that it got stuck.
I
> would help if I had the skills, but my ruby is not even close yet.
>
> Anyone else want to pick this up, dust it off, polish it up and  
> make it
> happen?
This is very high on my priority list, and if I had any spare time, I  
would definitely get it done.

I''m hoping to at least find the time to get the interface done that  
Blake needs, so he could then take up the Rails part, which he is  
clearly better at and more motivated about.

I mean, really, who needs sleep?

  --
  Like frozen sentries of the serengeti, the century-old termite mounds
  had withstood all tests of time and foe - all tests, that is, except
  the one involving drunken aardvarks and a stolen wrecking ball."
                  -- Gary Larson
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com

On 7/27/07, Luke Kanies <luke@madstop.com> wrote:

This is very high on my priority list, and if I had any spare time,
I
> would definitely get it done.
>
> I''m hoping to at least find the time to get the interface done
that
> Blake needs, so he could then take up the Rails part, which he is
> clearly better at and more motivated about.
>
> I mean, really, who needs sleep?


Yeah I can do the rails interface quite quickly.  If I actually felt like I
knew what I was doing with the abstraction layer I''m sure it''d
be done by
now, but I tend to just stare blankly at the screen when I look at it now.
I guess the main thing that I need is to have the configuration available as
Puppet::TransObjects IIRC, it''s been a while since I looked at it.

-Blake


_______________________________________________
Puppet-users mailing list
Puppet-users@madstop.com
https://mail.madstop.com/mailman/listinfo/puppet-users

On Jul 30, 2007, at 7:33 PM, Blake B wrote:
> Yeah I can do the rails interface quite quickly.  If I actually  
> felt like I knew what I was doing with the abstraction layer I''m  
> sure it''d be done by now, but I tend to just stare blankly at the
> screen when I look at it now.  I guess the main thing that I need  
> is to have the configuration available as Puppet::TransObjects  
> IIRC, it''s been a while since I looked at it.
So, I went through all of the code last night, and I was able to move  
nearly all of the ActiveRecord interactions into the ConfigStore  
API.  The only one I didn''t get to (because it was 3am) was the bit  
for comparing resource parameters.

However, I question whether this basic strategy really makes sense.   
That is, I question whether it makes sense to use a Web Services API,  
and if it does, I question whether it makes sense to retain direct  
ActiveRecord interfaces.

I can already see two more data types that will be stored in the  
database, reports and node configurations (basically, the information  
that will be in Kinial).  Given that, we would need either a big,  
ugly WS api, or we would need a bunch of smaller ones, neither of  
which is particularly appealing.

I think we need to have basic strategy planning on how we''re going to  
interface with Rails and how Puppet and Rails will grow together over  
time, and until we make decisions there, I think it makes sense to  
stick with a direct ActiveRecord interface.

Blake, can you ping me on IRC so we can talk about this?

In the meantime, I''m going to start a thread on puppet-dev detailing  
what I see in this area in the near future.

  --
  It''s very hard to predict things . . . Especially the future.
                  -- Prof. Charles Kelemen, Swarthmore CS Dept.
  ---------------------------------------------------------------------
  Luke Kanies | http://reductivelabs.com | http://madstop.com