Puppet users - Nov 2013 - get a *structured* version of the puppet agent output

puppet agent --verbose shows a verbose output of the changes done by 
puppet, such as:

notice: 
/Stage[main]/Logstash::Config/Logstash::Configdir[agent]/File[/etc/logstash/agent/config]/owner:
owner changed ''root'' to ''logstash''
notice: /Stage[main]/Varnish/Service[varnish]/ensure: ensure changed 
''stopped'' to ''running''

I''d need to make some analysis about these changes, but it is not
trivial
to correctly parse this output.
is there an option to get an structured version of this output (such as in 
xml or yaml)?
so that I can make queries such as:
- list all the files that have changed
- list all the services that changed from stopped to running.
...

yes, i can use sed and the like as a temporal workaround, but I''d need
to
have a robust approach.

option1: maybe there is a tool that correctly parses this output?
option2: or maybe there is a "puppet agent" equivalent command that 
directly produces a structured output?

otherwise, how much time would it required to implement option2 (i am a 
software developer, and a puppet-user (not currently a puppet-developer)?

Regards,
David

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/7e4a2aa2-2bd2-4311-81c2-8f7e4bd12a5b%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
> From: "David Portabella" <david.portabella@gmail.com>
> To: puppet-users@googlegroups.com
> Sent: Tuesday, November 26, 2013 4:54:03 PM
> Subject: [Puppet Users] get a *structured* version of the puppet agent
output
> 
> 
> 
> puppet agent --verbose shows a verbose output of the changes done by
> puppet, such as:
> 
> notice:
>
/Stage[main]/Logstash::Config/Logstash::Configdir[agent]/File[/etc/logstash/agent/config]/owner:
> owner changed ''root'' to ''logstash''
> notice: /Stage[main]/Varnish/Service[varnish]/ensure: ensure changed
> ''stopped'' to ''running''
> 
> I''d need to make some analysis about these changes, but it is not
trivial
> to correctly parse this output.
The same data is in the reports, a bit better there.
>> require ''puppet''
>>
YAML.load_file("/var/lib/puppet/state/last_run_report.yaml").logs.first
=> #<Puppet::Util::Log:0x7f2e3dcbcc68 @source="Puppet",
@level=:notice, @message="Finished catalog run in 0.18 seconds",
@time=Thu Nov 21 16:38:42 +0000 2013, @tags=["notice"]>

> is there an option to get an structured version of this output (such as in
> xml or yaml)?
> so that I can make queries such as:
> - list all the files that have changed
> - list all the services that changed from stopped to running.
> ...
> 
> yes, i can use sed and the like as a temporal workaround, but I''d
need to
> have a robust approach.
> 
> option1: maybe there is a tool that correctly parses this output?
> option2: or maybe there is a "puppet agent" equivalent command
that
> directly produces a structured output?
> 
> otherwise, how much time would it required to implement option2 (i am a
> software developer, and a puppet-user (not currently a puppet-developer)?
> 
> Regards,
> David
> 
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/puppet-users/7e4a2aa2-2bd2-4311-81c2-8f7e4bd12a5b%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
> 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/465611968.409.1385485068417.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/groups/opt_out.

I am not sure if this will answer your question, but have you looked at the yaml
files to be found under /var/lib/puppet/reports/ on the Puppet Master ?

Changes can be found by searching for "changed: true" ,
"out_of_sync: true", and/or "change_count: ?" where the
"?" is NOT zero.


“Sometimes I think the surest sign that intelligent life exists elsewhere in the
universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes) 

----- Original Message -----
From: "David Portabella" <david.portabella@gmail.com> 
To: puppet-users@googlegroups.com 
Sent: Tuesday, November 26, 2013 11:54:03 AM 
Subject: [Puppet Users] get a *structured* version of the puppet agent output 





puppet agent --verbose shows a verbose output of the changes done by puppet,
such as:

notice:
/Stage[main]/Logstash::Config/Logstash::Configdir[agent]/File[/etc/logstash/agent/config]/owner:
owner changed ''root'' to ''logstash''
notice: /Stage[main]/Varnish/Service[varnish]/ensure: ensure changed
''stopped'' to ''running''

I''d need to make some analysis about these changes, but it is not
trivial to correctly parse this output.
is there an option to get an structured version of this output (such as in xml
or yaml)?
so that I can make queries such as: 
- list all the files that have changed 
- list all the services that changed from stopped to running. 
... 

yes, i can use sed and the like as a temporal workaround, but I''d need
to have a robust approach.

option1: maybe there is a tool that correctly parses this output? 
option2: or maybe there is a "puppet agent" equivalent command that
directly produces a structured output?

otherwise, how much time would it required to implement option2 (i am a software
developer, and a puppet-user (not currently a puppet-developer)?



Regards, 
David 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/7e4a2aa2-2bd2-4311-81c2-8f7e4bd12a5b%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out . 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1032688657.4748204.1385485436549.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net.
For more options, visit https://groups.google.com/groups/opt_out.

We have this exact same requirement.

Enforce a non-changing policy on the cloud, avoid automatic 
drift-prevention, await errors and reports,
track down cause, fix root cause, apply puppet manifests to right things 
back, etc.

Using Puppet to bandaid things without root-cause drill-down/post-mortem, 
is unwise!!!!

We need a simple report-formatting mechanism like David says!

On Tuesday, November 26, 2013 8:54:03 AM UTC-8, David Portabella wrote:
> puppet agent --verbose shows a verbose output of the changes done by 
> puppet, such as:
>
> notice: 
>
/Stage[main]/Logstash::Config/Logstash::Configdir[agent]/File[/etc/logstash/agent/config]/owner:
> owner changed ''root'' to ''logstash''
> notice: /Stage[main]/Varnish/Service[varnish]/ensure: ensure changed 
> ''stopped'' to ''running''
>
> I''d need to make some analysis about these changes, but it is not
trivial
> to correctly parse this output.
> is there an option to get an structured version of this output (such as in 
> xml or yaml)?
> so that I can make queries such as:
> - list all the files that have changed
> - list all the services that changed from stopped to running.
> ...
>
> yes, i can use sed and the like as a temporal workaround, but I''d
need to
> have a robust approach.
>
> option1: maybe there is a tool that correctly parses this output?
> option2: or maybe there is a "puppet agent" equivalent command
that
> directly produces a structured output?
>
> otherwise, how much time would it required to implement option2 (i am a 
> software developer, and a puppet-user (not currently a puppet-developer)?
>
> Regards,
> David
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/5907f644-2285-4298-9a07-3d5e8de63a24%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

What we want is not more complexity, but more simplicity!

I could go into 

  puppet config print reportdir

and then to its

  /var/lib/puppet/reports

then to the host directories of interest and grep out message.

But that seems a very sorry state of affairs.

Puppetmasters speak UP!


On Tuesday, November 26, 2013 8:57:48 AM UTC-8, R.I. Pienaar wrote:
>
>
> ----- Original Message ----- 
> > From: "David Portabella" <david.po...@gmail.com
<javascript:>>
> > To: puppet...@googlegroups.com <javascript:> 
> > Sent: Tuesday, November 26, 2013 4:54:03 PM 
> > Subject: [Puppet Users] get a *structured* version of the puppet agent
> output 
> > 
> > 
> > 
> > puppet agent --verbose shows a verbose output of the changes done by 
> > puppet, such as: 
> > 
> > notice: 
> > 
>
/Stage[main]/Logstash::Config/Logstash::Configdir[agent]/File[/etc/logstash/agent/config]/owner:
>
> > owner changed ''root'' to ''logstash''
> > notice: /Stage[main]/Varnish/Service[varnish]/ensure: ensure changed 
> > ''stopped'' to ''running'' 
> > 
> > I''d need to make some analysis about these changes, but it is
not
> trivial 
> > to correctly parse this output. 
>
> The same data is in the reports, a bit better there. 
>
> >> require ''puppet'' 
> >>
YAML.load_file("/var/lib/puppet/state/last_run_report.yaml").logs.first
> => #<Puppet::Util::Log:0x7f2e3dcbcc68 @source="Puppet",
@level=:notice,
> @message="Finished catalog run in 0.18 seconds", @time=Thu Nov 21
16:38:42
> +0000 2013, @tags=["notice"]> 
>
>
> > is there an option to get an structured version of this output (such
as
> in 
> > xml or yaml)? 
> > so that I can make queries such as: 
> > - list all the files that have changed 
> > - list all the services that changed from stopped to running. 
> > ... 
> > 
> > yes, i can use sed and the like as a temporal workaround, but
I''d need
> to 
> > have a robust approach. 
> > 
> > option1: maybe there is a tool that correctly parses this output? 
> > option2: or maybe there is a "puppet agent" equivalent
command that
> > directly produces a structured output? 
> > 
> > otherwise, how much time would it required to implement option2 (i am
a
> > software developer, and a puppet-user (not currently a 
> puppet-developer)? 
> > 
> > Regards, 
> > David 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "Puppet Users" group. 
> > To unsubscribe from this group and stop receiving emails from it, send
> an 
> > email to puppet-users...@googlegroups.com <javascript:>. 
> > To view this discussion on the web visit 
> > 
>
https://groups.google.com/d/msgid/puppet-users/7e4a2aa2-2bd2-4311-81c2-8f7e4bd12a5b%40googlegroups.com.
>
> > For more options, visit https://groups.google.com/groups/opt_out. 
> > 
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/512658ed-2f0d-42fc-b049-cd3c41b98aca%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
> From: "Stuart Cracraft" <smcracraft@gmail.com>
> To: puppet-users@googlegroups.com
> Sent: Tuesday, November 26, 2013 7:02:42 PM
> Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output
> 
> What we want is not more complexity, but more simplicity!
> 
> I could go into
> 
>   puppet config print reportdir
> 
> and then to its
> 
>   /var/lib/puppet/reports
> 
> then to the host directories of interest and grep out message.
> 
> But that seems a very sorry state of affairs.
> 
> Puppetmasters speak UP!
you do not need to grep out anything, as I showed you there is a ruby API
for accessing this data.

If you use PuppetDB it will also be able to store this information and it has
APIs for extracting these logs in a structured manner.

API access is about as much as you can hope for I think.  Your alternatives are
to write a logger plugin for Puppet that outputs JSON, I''ve done this
and it
was not clear sailing.


-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1024951816.609.1385492893475.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/groups/opt_out.

Who will share a report generator for the yaml reports generated by puppet so
that we do not have to reinvent wheel after wheel after wheel!!!!!!
> On Nov 26, 2013, at 11:08 AM, "R.I.Pienaar" <rip@devco.net>
wrote:
> 
> 
> 
> ----- Original Message -----
>> From: "Stuart Cracraft" <smcracraft@gmail.com>
>> To: puppet-users@googlegroups.com
>> Sent: Tuesday, November 26, 2013 7:02:42 PM
>> Subject: Re: [Puppet Users] get a *structured* version of the puppet
agent output
>> 
>> What we want is not more complexity, but more simplicity!
>> 
>> I could go into
>> 
>>  puppet config print reportdir
>> 
>> and then to its
>> 
>>  /var/lib/puppet/reports
>> 
>> then to the host directories of interest and grep out message.
>> 
>> But that seems a very sorry state of affairs.
>> 
>> Puppetmasters speak UP!
> 
> you do not need to grep out anything, as I showed you there is a ruby API
> for accessing this data.
> 
> If you use PuppetDB it will also be able to store this information and it
has
> APIs for extracting these logs in a structured manner.
> 
> API access is about as much as you can hope for I think.  Your alternatives
are
> to write a logger plugin for Puppet that outputs JSON, I''ve done
this and it
> was not clear sailing.
> 
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1024951816.609.1385492893475.JavaMail.zimbra%40devco.net.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/6E1AA223-0401-485C-8E4C-AABF12FB2291%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

----- Original Message -----
> From: "Stuart Cracraft" <smcracraft@me.com>
> To: puppet-users@googlegroups.com
> Sent: Tuesday, November 26, 2013 7:10:53 PM
> Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output
> 
> Who will share a report generator for the yaml reports generated by puppet
so
> that we do not have to reinvent wheel after wheel after wheel!!!!!!
not everyone shares the same goals so everyone tend to write them, but here you
go this is one I wrote

http://www.devco.net/archives/2013/10/10/cli-report-viewer-for-puppet.php

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/216012641.610.1385493314105.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/groups/opt_out.

Puppet Labs:

What report generators do you
have and provide at your website
so that we can report intelligently?

Stuart
> On Nov 26, 2013, at 11:15 AM, "R.I.Pienaar" <rip@devco.net>
wrote:
> 
> 
> 
> ----- Original Message -----
>> From: "Stuart Cracraft" <smcracraft@me.com>
>> To: puppet-users@googlegroups.com
>> Sent: Tuesday, November 26, 2013 7:10:53 PM
>> Subject: Re: [Puppet Users] get a *structured* version of the puppet
agent output
>> 
>> Who will share a report generator for the yaml reports generated by
puppet so
>> that we do not have to reinvent wheel after wheel after wheel!!!!!!
> 
> not everyone shares the same goals so everyone tend to write them, but here
you go this is one I wrote
> 
> http://www.devco.net/archives/2013/10/10/cli-report-viewer-for-puppet.php
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/216012641.610.1385493314105.JavaMail.zimbra%40devco.net.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/41DE4370-2957-41D5-9DAC-6AFA8B99C0B7%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

Reference: http://docs.puppetlabs.com/puppet/3/reference/format_report.html

Start with this:
----------------------------------------------------------------------------------------
#!/usr/bin/ruby 

require ''puppet''
if defined?(ARGV)
    filename = ARGV.first

    if defined?(filename)
        report = YAML.load_file(filename)
        print "Report for : "
        puts report.host
        print "Started : "
        puts report.time
        print "Log count: "
        puts report.logs.size
        print "Metrics count: "
        puts report.metrics.size
        print "Resource Status count: "
        puts report.resource_statuses.size
        print "Status: "
        puts report.status

        report.logs.each do |logg|
            puts "LOG ---------------------------------------------"
            if logg.file
                print "File: "
                puts logg.file
            end

            if logg.line
                print "Line: "
                puts logg.line
            end

            print "Level: "
            puts logg.level
            print "Message: "
            puts logg.message
            print "Source: "
            puts logg.source
            print "Tags: "
            puts logg.tags.join(" ")
            print "Time: "
            puts logg.time
        end

        report.resource_statuses.keys.each do |kk|
            if report.resource_statuses["#{kk}"].change_count > 0
                puts "RESOURCE STATUS
===================================="
                puts "#{kk} "
                report.resource_statuses["#{kk}"].events.each do
|line|
                    print "property: "
                    puts line.property
                    print "message: "
                    puts line.message
                    print "name: "
                    puts line.name
                    print "status: "
                    puts line.status
                    print "when: "
                    puts line.time
                end
            end
        end
        puts "-------------------------------------------------"
    end
end
------------------------------------------------------------------------------------------

Good luck.


“Sometimes I think the surest sign that intelligent life exists elsewhere in the
universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

----- Original Message -----
From: "Stuart Cracraft" <smcracraft@me.com>
To: puppet-users@googlegroups.com
Sent: Tuesday, November 26, 2013 2:10:53 PM
Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output

Who will share a report generator for the yaml reports generated by puppet so
that we do not have to reinvent wheel after wheel after wheel!!!!!!
> On Nov 26, 2013, at 11:08 AM, "R.I.Pienaar" <rip@devco.net>
wrote:
> 
> 
> 
> ----- Original Message -----
>> From: "Stuart Cracraft" <smcracraft@gmail.com>
>> To: puppet-users@googlegroups.com
>> Sent: Tuesday, November 26, 2013 7:02:42 PM
>> Subject: Re: [Puppet Users] get a *structured* version of the puppet
agent output
>> 
>> What we want is not more complexity, but more simplicity!
>> 
>> I could go into
>> 
>>  puppet config print reportdir
>> 
>> and then to its
>> 
>>  /var/lib/puppet/reports
>> 
>> then to the host directories of interest and grep out message.
>> 
>> But that seems a very sorry state of affairs.
>> 
>> Puppetmasters speak UP!
> 
> you do not need to grep out anything, as I showed you there is a ruby API
> for accessing this data.
> 
> If you use PuppetDB it will also be able to store this information and it
has
> APIs for extracting these logs in a structured manner.
> 
> API access is about as much as you can hope for I think.  Your alternatives
are
> to write a logger plugin for Puppet that outputs JSON, I''ve done
this and it
> was not clear sailing.
> 
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1024951816.609.1385492893475.JavaMail.zimbra%40devco.net.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/6E1AA223-0401-485C-8E4C-AABF12FB2291%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/877739462.4751612.1385494040183.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net.
For more options, visit https://groups.google.com/groups/opt_out.

I''ll try it after exiting this meeting. 
> On Nov 26, 2013, at 11:27 AM, Dan White <ygor@comcast.net> wrote:
> 
> Reference: http://docs.puppetlabs.com/puppet/3/reference/format_report.html
> 
> Start with this:
>
----------------------------------------------------------------------------------------
> #!/usr/bin/ruby 
> 
> require ''puppet''
> if defined?(ARGV)
>    filename = ARGV.first
> 
>    if defined?(filename)
>        report = YAML.load_file(filename)
>        print "Report for : "
>        puts report.host
>        print "Started : "
>        puts report.time
>        print "Log count: "
>        puts report.logs.size
>        print "Metrics count: "
>        puts report.metrics.size
>        print "Resource Status count: "
>        puts report.resource_statuses.size
>        print "Status: "
>        puts report.status
> 
>        report.logs.each do |logg|
>            puts "LOG
---------------------------------------------"
>            if logg.file
>                print "File: "
>                puts logg.file
>            end
> 
>            if logg.line
>                print "Line: "
>                puts logg.line
>            end
> 
>            print "Level: "
>            puts logg.level
>            print "Message: "
>            puts logg.message
>            print "Source: "
>            puts logg.source
>            print "Tags: "
>            puts logg.tags.join(" ")
>            print "Time: "
>            puts logg.time
>        end
> 
>        report.resource_statuses.keys.each do |kk|
>            if report.resource_statuses["#{kk}"].change_count >
0
>                puts "RESOURCE STATUS
===================================="
>                puts "#{kk} "
>                report.resource_statuses["#{kk}"].events.each do
|line|
>                    print "property: "
>                    puts line.property
>                    print "message: "
>                    puts line.message
>                    print "name: "
>                    puts line.name
>                    print "status: "
>                    puts line.status
>                    print "when: "
>                    puts line.time
>                end
>            end
>        end
>        puts "-------------------------------------------------"
>    end
> end
>
------------------------------------------------------------------------------------------
> 
> Good luck.
> 
> 
> “Sometimes I think the surest sign that intelligent life exists elsewhere
in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin & Hobbes)
> 
> ----- Original Message -----
> From: "Stuart Cracraft" <smcracraft@me.com>
> To: puppet-users@googlegroups.com
> Sent: Tuesday, November 26, 2013 2:10:53 PM
> Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output
> 
> Who will share a report generator for the yaml reports generated by puppet
so that we do not have to reinvent wheel after wheel after wheel!!!!!!
> 
>> On Nov 26, 2013, at 11:08 AM, "R.I.Pienaar"
<rip@devco.net> wrote:
>> 
>> 
>> 
>> ----- Original Message -----
>>> From: "Stuart Cracraft" <smcracraft@gmail.com>
>>> To: puppet-users@googlegroups.com
>>> Sent: Tuesday, November 26, 2013 7:02:42 PM
>>> Subject: Re: [Puppet Users] get a *structured* version of the
puppet agent output
>>> 
>>> What we want is not more complexity, but more simplicity!
>>> 
>>> I could go into
>>> 
>>> puppet config print reportdir
>>> 
>>> and then to its
>>> 
>>> /var/lib/puppet/reports
>>> 
>>> then to the host directories of interest and grep out message.
>>> 
>>> But that seems a very sorry state of affairs.
>>> 
>>> Puppetmasters speak UP!
>> 
>> you do not need to grep out anything, as I showed you there is a ruby
API
>> for accessing this data.
>> 
>> If you use PuppetDB it will also be able to store this information and
it has
>> APIs for extracting these logs in a structured manner.
>> 
>> API access is about as much as you can hope for I think.  Your
alternatives are
>> to write a logger plugin for Puppet that outputs JSON, I''ve
done this and it
>> was not clear sailing.
>> 
>> 
>> -- 
>> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
>> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1024951816.609.1385492893475.JavaMail.zimbra%40devco.net.
>> For more options, visit https://groups.google.com/groups/opt_out.
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/6E1AA223-0401-485C-8E4C-AABF12FB2291%40me.com.
> For more options, visit https://groups.google.com/groups/opt_out.
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/877739462.4751612.1385494040183.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/FE3E7F42-D57E-4DEC-A12C-A940481933B5%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

(inline)

On Tue, Nov 26, 2013 at 10:43:31AM -0800, Stuart Cracraft
wrote:
>    We have this exact same requirement.
>    Enforce a non-changing policy on the cloud, avoid automatic
>    drift-prevention, await errors and reports,
>    track down cause, fix root cause, apply puppet manifests to right things
>    back, etc.
>    Using Puppet to bandaid things without root-cause
drill-down/post-mortem,
>    is unwise!!!!
I don''t see how it''s wise to let configuration deviations
loiter in your systems until a human has the time to manually fix them. This
applies even more if you''re working at scale. Surely the ticket
generated from (whatever is monitoring agent runs that unpredictably change
things) will still be there once your configs are back in conformance?

Short form would be that I think you''ll experience fewer pain points if
you let puppet enforce your configuration like it''s designed to.
>    We need a simple report-formatting mechanism like David says!
> 
>    On Tuesday, November 26, 2013 8:54:03 AM UTC-8, David Portabella wrote:
> 
>      puppet agent --verbose shows a verbose output of the changes done by
>      puppet, such as:
> 
>      notice:
>     
/Stage[main]/Logstash::Config/Logstash::Configdir[agent]/File[/etc/logstash/agent/config]/owner:
>      owner changed ''root'' to ''logstash''
>      notice: /Stage[main]/Varnish/Service[varnish]/ensure: ensure changed
>      ''stopped'' to ''running''
> 
>      I''d need to make some analysis about these changes, but it is
not
>      trivial to correctly parse this output.
>      is there an option to get an structured version of this output (such
as
>      in xml or yaml)?
>      so that I can make queries such as:
>      - list all the files that have changed
>      - list all the services that changed from stopped to running.
>      ...
> 
>      yes, i can use sed and the like as a temporal workaround, but
I''d need
>      to have a robust approach.
> 
>      option1: maybe there is a tool that correctly parses this output?
>      option2: or maybe there is a "puppet agent" equivalent
command that
>      directly produces a structured output?
> 
>      otherwise, how much time would it required to implement option2 (i am
a
>      software developer, and a puppet-user (not currently a
>      puppet-developer)?
> 
>      Regards,
>      David
> 
>    --
>    You received this message because you are subscribed to the Google
Groups
>    "Puppet Users" group.
>    To unsubscribe from this group and stop receiving emails from it, send
an
>    email to puppet-users+unsubscribe@googlegroups.com.
>    To view this discussion on the web visit
>   
[1]https://groups.google.com/d/msgid/puppet-users/5907f644-2285-4298-9a07-3d5e8de63a24%40googlegroups.com.
>    For more options, visit [2]https://groups.google.com/groups/opt_out.
> 
> References
> 
>    Visible links
>    1.
https://groups.google.com/d/msgid/puppet-users/5907f644-2285-4298-9a07-3d5e8de63a24%40googlegroups.com
>    2. https://groups.google.com/groups/opt_out
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/20131126204924.GA4749%40iniquitous.heresiarch.ca.
For more options, visit https://groups.google.com/groups/opt_out.

Understood. 

But, we have some real good troubleshooters here. 

And we want to get at core fixes. 

Not just log-examination, esoteric
thought and bandaids. 
> On Nov 26, 2013, at 12:49 PM, Christopher Wood
<christopher_wood@pobox.com> wrote:
> 
> (inline)
> 
>> On Tue, Nov 26, 2013 at 10:43:31AM -0800, Stuart Cracraft wrote:
>>   We have this exact same requirement.
>>   Enforce a non-changing policy on the cloud, avoid automatic
>>   drift-prevention, await errors and reports,
>>   track down cause, fix root cause, apply puppet manifests to right
things
>>   back, etc.
>>   Using Puppet to bandaid things without root-cause
drill-down/post-mortem,
>>   is unwise!!!!
> 
> I don''t see how it''s wise to let configuration deviations
loiter in your systems until a human has the time to manually fix them. This
applies even more if you''re working at scale. Surely the ticket
generated from (whatever is monitoring agent runs that unpredictably change
things) will still be there once your configs are back in conformance?
> 
> Short form would be that I think you''ll experience fewer pain
points if you let puppet enforce your configuration like it''s designed
to.
> 
>>   We need a simple report-formatting mechanism like David says!
>> 
>>   On Tuesday, November 26, 2013 8:54:03 AM UTC-8, David Portabella
wrote:
>> 
>>     puppet agent --verbose shows a verbose output of the changes done
by
>>     puppet, such as:
>> 
>>     notice:
>>    
/Stage[main]/Logstash::Config/Logstash::Configdir[agent]/File[/etc/logstash/agent/config]/owner:
>>     owner changed ''root'' to
''logstash''
>>     notice: /Stage[main]/Varnish/Service[varnish]/ensure: ensure
changed
>>     ''stopped'' to ''running''
>> 
>>     I''d need to make some analysis about these changes, but it
is not
>>     trivial to correctly parse this output.
>>     is there an option to get an structured version of this output
(such as
>>     in xml or yaml)?
>>     so that I can make queries such as:
>>     - list all the files that have changed
>>     - list all the services that changed from stopped to running.
>>     ...
>> 
>>     yes, i can use sed and the like as a temporal workaround, but
I''d need
>>     to have a robust approach.
>> 
>>     option1: maybe there is a tool that correctly parses this output?
>>     option2: or maybe there is a "puppet agent" equivalent
command that
>>     directly produces a structured output?
>> 
>>     otherwise, how much time would it required to implement option2 (i
am a
>>     software developer, and a puppet-user (not currently a
>>     puppet-developer)?
>> 
>>     Regards,
>>     David
>> 
>>   --
>>   You received this message because you are subscribed to the Google
Groups
>>   "Puppet Users" group.
>>   To unsubscribe from this group and stop receiving emails from it,
send an
>>   email to puppet-users+unsubscribe@googlegroups.com.
>>   To view this discussion on the web visit
>>  
[1]https://groups.google.com/d/msgid/puppet-users/5907f644-2285-4298-9a07-3d5e8de63a24%40googlegroups.com.
>>   For more options, visit [2]https://groups.google.com/groups/opt_out.
>> 
>> References
>> 
>>   Visible links
>>   1.
https://groups.google.com/d/msgid/puppet-users/5907f644-2285-4298-9a07-3d5e8de63a24%40googlegroups.com
>>   2. https://groups.google.com/groups/opt_out
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/20131126204924.GA4749%40iniquitous.heresiarch.ca.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/696AE1AA-FE9D-4788-AE1E-0EC6FEBB03A7%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks Ygor and R.I. Pienaar.
that''s exactly what i was looking for!

Regards,
David


On Tuesday, November 26, 2013 8:27:20 PM UTC+1, Ygor
wrote:
>
> Reference: 
> http://docs.puppetlabs.com/puppet/3/reference/format_report.html 
>
> Start with this: 
>
----------------------------------------------------------------------------------------
>
> #!/usr/bin/ruby 
>
> require ''puppet'' 
> if defined?(ARGV) 
>     filename = ARGV.first 
>
>     if defined?(filename) 
>         report = YAML.load_file(filename) 
>         print "Report for : " 
>         puts report.host 
>         print "Started : " 
>         puts report.time 
>         print "Log count: " 
>         puts report.logs.size 
>         print "Metrics count: " 
>         puts report.metrics.size 
>         print "Resource Status count: " 
>         puts report.resource_statuses.size 
>         print "Status: " 
>         puts report.status 
>
>         report.logs.each do |logg| 
>             puts "LOG
---------------------------------------------"
>             if logg.file 
>                 print "File: " 
>                 puts logg.file 
>             end 
>
>             if logg.line 
>                 print "Line: " 
>                 puts logg.line 
>             end 
>
>             print "Level: " 
>             puts logg.level 
>             print "Message: " 
>             puts logg.message 
>             print "Source: " 
>             puts logg.source 
>             print "Tags: " 
>             puts logg.tags.join(" ") 
>             print "Time: " 
>             puts logg.time 
>         end 
>
>         report.resource_statuses.keys.each do |kk| 
>             if report.resource_statuses["#{kk}"].change_count
> 0
>                 puts "RESOURCE STATUS 
> ====================================" 
>                 puts "#{kk} " 
>                 report.resource_statuses["#{kk}"].events.each do
|line|
>                     print "property: " 
>                     puts line.property 
>                     print "message: " 
>                     puts line.message 
>                     print "name: " 
>                     puts line.name 
>                     print "status: " 
>                     puts line.status 
>                     print "when: " 
>                     puts line.time 
>                 end 
>             end 
>         end 
>         puts "-------------------------------------------------" 
>     end 
> end 
>
------------------------------------------------------------------------------------------
>
>
> Good luck. 
>
>
> “Sometimes I think the surest sign that intelligent life exists elsewhere 
> in the universe is that none of it has tried to contact us.” 
> Bill Waterson (Calvin & Hobbes) 
>
> ----- Original Message ----- 
> From: "Stuart Cracraft" <smcra...@me.com
<javascript:>>
> To: puppet...@googlegroups.com <javascript:> 
> Sent: Tuesday, November 26, 2013 2:10:53 PM 
> Subject: Re: [Puppet Users] get a *structured* version of the puppet agent 
> output 
>
> Who will share a report generator for the yaml reports generated by puppet 
> so that we do not have to reinvent wheel after wheel after wheel!!!!!! 
>
> > On Nov 26, 2013, at 11:08 AM, "R.I.Pienaar"
<r...@devco.net<javascript:>>
> wrote: 
> > 
> > 
> > 
> > ----- Original Message ----- 
> >> From: "Stuart Cracraft" <smcra...@gmail.com
<javascript:>>
> >> To: puppet...@googlegroups.com <javascript:> 
> >> Sent: Tuesday, November 26, 2013 7:02:42 PM 
> >> Subject: Re: [Puppet Users] get a *structured* version of the
puppet
> agent output 
> >> 
> >> What we want is not more complexity, but more simplicity! 
> >> 
> >> I could go into 
> >> 
> >>  puppet config print reportdir 
> >> 
> >> and then to its 
> >> 
> >>  /var/lib/puppet/reports 
> >> 
> >> then to the host directories of interest and grep out message. 
> >> 
> >> But that seems a very sorry state of affairs. 
> >> 
> >> Puppetmasters speak UP! 
> > 
> > you do not need to grep out anything, as I showed you there is a ruby 
> API 
> > for accessing this data. 
> > 
> > If you use PuppetDB it will also be able to store this information and
> it has 
> > APIs for extracting these logs in a structured manner. 
> > 
> > API access is about as much as you can hope for I think.  Your 
> alternatives are 
> > to write a logger plugin for Puppet that outputs JSON, I''ve
done this
> and it 
> > was not clear sailing. 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group. 
> > To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe. 
> > To unsubscribe from this group and all its topics, send an email to 
> puppet-users...@googlegroups.com <javascript:>. 
> > To view this discussion on the web visit 
>
https://groups.google.com/d/msgid/puppet-users/1024951816.609.1385492893475.JavaMail.zimbra%40devco.net.
>
> > For more options, visit https://groups.google.com/groups/opt_out. 
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Puppet Users" group. 
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to puppet-users...@googlegroups.com <javascript:>. 
> To view this discussion on the web visit 
>
https://groups.google.com/d/msgid/puppet-users/6E1AA223-0401-485C-8E4C-AABF12FB2291%40me.com.
>
> For more options, visit https://groups.google.com/groups/opt_out. 
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/ed0eb173-36d8-4771-b253-79593b4dacb6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,

On 11/26/2013 07:43 PM, Stuart Cracraft wrote:
> 
> Using Puppet to bandaid things without root-cause
> drill-down/post-mortem, is unwise!!!!
you are entitled to your opinion, but it''s not exactly common in the
community and, frankly, I believe it''s not a good fit with
puppet''s
philosophy. In other words, puppet may in fact not be the best tool for
the job.

But then, you can probably bend it towards doing what you need. Please
keep in mind that the ruby code that RI Pienaar supplied is just a very,
very basic example. The YAML in puppet reports grants you great power
and flexibility, but do not step into either of the traps of

a) grepping through and manually parsing the YAML
or
b) loading the YAML using ruby or irb, but then just converting all key
datapoints to string representations to grep through these

Instead, find out which data structures are of interest to you (i.e.,
which attributes denote a resource you want to highlight from the
report) and write a ruby script that uses the common library tools (e.g.
http://www.ruby-doc.org/core-1.8.7/Array.html#method-i-select) to fetch
those objects and generate output that is suited to your auditing needs.

HTH,
Felix

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/5295CC7D.8080902%40alumni.tu-berlin.de.
For more options, visit https://groups.google.com/groups/opt_out.

...aaaand then I saw that Dan had already shared it in what my mail
client considers a different thread for some unfathomable reason.

Sorry for the noise.

On 11/27/2013 11:42 AM, Felix Frank wrote:
> Instead, find out which data structures are of interest to you (i.e.,
> which attributes denote a resource you want to highlight from the
> report) and write a ruby script that uses the common library tools (e.g.
> http://www.ruby-doc.org/core-1.8.7/Array.html#method-i-select) to fetch
> those objects and generate output that is suited to your auditing needs.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/5295CCEE.5080007%40alumni.tu-berlin.de.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks again! It is great to use the puppet report yaml files!
here there is an example script that prints all resources statuses, 
then it filters them by taking only the services,
then it filters them by taking only the services changed to running.

test.ruby
#!/usr/bin/ruby 

require ''puppet'' 
filename = ARGV.first 
report = YAML.load_file(filename) 
rs = report.resource_statuses.values.sort_by{|r| r.resource}

puts "PRINT ALL RESOURCES STATUSES."
rs.each do |r| 
  puts "#{r.change_count} #{r.events.map {|e| e.previous_value}} 
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 

puts "PRINT ALL SERVICES."
services = rs.select{|r| r.resource_type == ''Service''}
services.each do |r| 
  puts "#{r.change_count} #{r.events.map {|e| e.previous_value}} 
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 

puts "PRINT ALL SERVICES CHANGED TO RUNNING."
services_changed_to_running = rs.select{|r| r.resource_type ==
''Service'' &&
r.change_count > 0 && r.events.index { |e| e.desired_value ==
:running } !=
nil}
services_changed_to_running.each do |r| 
  puts "#{r.change_count} #{r.events.map {|e| e.previous_value}} 
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 
 
example output:
$ test.ruby /var/opt/lib/pe-puppet/reports/test/201311261645.yaml 
PRINT ALL RESOURCES STATUSES.
1 [:absent] [:directory] File[/var/app]
1 [:absent] [:file] File[/var/app/app.conf]
2 [0, "755"] [91, "775"] File[/var/log/tomcat6]
0 [] [] Filebucket[puppet]
1 [:absent] [:present] Group[release]
1 [:absent] [:present] Package[varnish]
0 [] [] Schedule[weekly]
1 [:stopped] [:running] Service[varnish]
0 [] [] Tidy[/etc/collectd.d/]
1 ["/sbin/nologin"] ["/bin/bash"] User[tomcat]
...

PRINT ALL SERVICES.
1 [:stopped] [:running] Service[collectd]
1 [:stopped] [:running] Service[logstash-agent]
1 [:true] [:false] Service[logstash]
1 [:stopped] [:running] Service[nginx]
0 [] [] Service[rsyslog]
1 [:stopped] [:running] Service[statsd]
1 [:false] [:true] Service[supervisord]
1 [:stopped] [:running] Service[varnish]

PRINT ALL SERVICES CHANGED TO RUNNING
1 [:stopped] [:running] Service[collectd]
1 [:stopped] [:running] Service[logstash-agent]
1 [:stopped] [:running] Service[nginx]
1 [:stopped] [:running] Service[statsd]
1 [:stopped] [:running] Service[varnish]

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/c70db1f6-b133-4290-a8e1-827ea239028b%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks for stating the obvious Felix. It''s always good and I
don''t take it
any way except in which it was intended: educational.

Puppet has been running here in the desired scenario since last night near COB
which means simple monitoring and reporting of all actions which would be done,
but not done (noop = true and reports = true on node with classes enabled on
puppetmaster in puppet.conf) with daemon on node being --verbose in the
/etc/init.d/puppet
script.

While I would still like more central control of the nodes from the Puppet
Master, the above
does fit the requirement here so we are good on PM!

On Nov 27, 2013, at 2:42 AM, Felix Frank <felix.frank@alumni.tu-berlin.de>
wrote:
> Hi,
> 
> On 11/26/2013 07:43 PM, Stuart Cracraft wrote:
>> 
>> Using Puppet to bandaid things without root-cause
>> drill-down/post-mortem, is unwise!!!!
> 
> you are entitled to your opinion, but it''s not exactly common in
the
> community and, frankly, I believe it''s not a good fit with
puppet''s
> philosophy. In other words, puppet may in fact not be the best tool for
> the job.
> 
> But then, you can probably bend it towards doing what you need. Please
> keep in mind that the ruby code that RI Pienaar supplied is just a very,
> very basic example. The YAML in puppet reports grants you great power
> and flexibility, but do not step into either of the traps of
> 
> a) grepping through and manually parsing the YAML
> or
> b) loading the YAML using ruby or irb, but then just converting all key
> datapoints to string representations to grep through these
> 
> Instead, find out which data structures are of interest to you (i.e.,
> which attributes denote a resource you want to highlight from the
> report) and write a ruby script that uses the common library tools (e.g.
> http://www.ruby-doc.org/core-1.8.7/Array.html#method-i-select) to fetch
> those objects and generate output that is suited to your auditing needs.
> 
> HTH,
> Felix
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/5295CC7D.8080902%40alumni.tu-berlin.de.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/D785C0DC-D9A4-46D5-AF4D-859774A5B8DF%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

Very nice. 
I am still learning Ruby and this will help 

Thanks. 


“Sometimes I think the surest sign that intelligent life exists elsewhere in the
universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes) 

----- Original Message -----
From: "David Portabella" <david.portabella@gmail.com> 
To: puppet-users@googlegroups.com 
Sent: Wednesday, November 27, 2013 10:12:48 AM 
Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output



Thanks again! It is great to use the puppet report yaml files! 
here there is an example script that prints all resources statuses, 

then it filters them by taking only the services, 
then it filters them by taking only the services changed to running. 


test.ruby 
#!/usr/bin/ruby 


require ''puppet'' 
filename = ARGV.first 
report = YAML.load_file(filename) 
rs = report.resource_statuses.values.sort_by{|r| r.resource} 


puts "PRINT ALL RESOURCES STATUSES." 
rs.each do |r| 
puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 


puts "PRINT ALL SERVICES." 
services = rs.select{|r| r.resource_type == ''Service''} 
services.each do |r| 
puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 


puts "PRINT ALL SERVICES CHANGED TO RUNNING." 
services_changed_to_running = rs.select{|r| r.resource_type ==
''Service'' && r.change_count > 0 &&
r.events.index { |e| e.desired_value == :running } != nil}
services_changed_to_running.each do |r| 
puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 

example output: 
$ test.ruby /var/opt/lib/pe-puppet/reports/test/201311261645.yaml 
PRINT ALL RESOURCES STATUSES. 
1 [:absent] [:directory] File[/var/app] 
1 [:absent] [:file] File[/var/app/app.conf] 
2 [0, "755"] [91, "775"] File[/var/log/tomcat6] 
0 [] [] Filebucket[puppet] 
1 [:absent] [:present] Group[release] 
1 [:absent] [:present] Package[varnish] 
0 [] [] Schedule[weekly] 
1 [:stopped] [:running] Service[varnish] 
0 [] [] Tidy[/etc/collectd.d/] 
1 ["/sbin/nologin"] ["/bin/bash"] User[tomcat] 
... 


PRINT ALL SERVICES. 
1 [:stopped] [:running] Service[collectd] 
1 [:stopped] [:running] Service[logstash-agent] 
1 [:true] [:false] Service[logstash] 
1 [:stopped] [:running] Service[nginx] 
0 [] [] Service[rsyslog] 
1 [:stopped] [:running] Service[statsd] 
1 [:false] [:true] Service[supervisord] 
1 [:stopped] [:running] Service[varnish] 


PRINT ALL SERVICES CHANGED TO RUNNING 
1 [:stopped] [:running] Service[collectd] 
1 [:stopped] [:running] Service[logstash-agent] 
1 [:stopped] [:running] Service[nginx] 
1 [:stopped] [:running] Service[statsd] 
1 [:stopped] [:running] Service[varnish] 



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/c70db1f6-b133-4290-a8e1-827ea239028b%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out . 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1253465309.4764508.1385567092577.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net.
For more options, visit https://groups.google.com/groups/opt_out.

+1

I''ve been using this script for the last month to view the "last
run"
details, and it works very well.  Here at my shop I am not worried about
changes, but I am worried about errors.  I wrote a small Python script that
queries PuppetDB (if you''re not using it, use it) and reports if there
were
any errors on any nodes during their last run of the night (we do not run
Puppet during trading hours).  I also use Puppet Dashboard for at-a-glance
sanity checking and as an easy way to drill down on unresponsive hosts and
to easily view reports.

The combination of these three tools work very well for me.

Jeffrey.

On Tue, Nov 26, 2013 at 1:15 PM, R.I.Pienaar <rip@devco.net> wrote:
>
> not everyone shares the same goals so everyone tend to write them, but
> here you go this is one I wrote
>
> http://www.devco.net/archives/2013/10/10/cli-report-viewer-for-puppet.php
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAMvPdm17zvpCCJfbxp%2BB_CkNGeY-QYfmdgpZoYoc0TnntAG3eQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Get the book Learning Ruby. 

Worth it. 
> On Nov 27, 2013, at 7:44 AM, Dan White <ygor@comcast.net> wrote:
> 
> Very nice.
> I am still learning Ruby and this will help
> 
> Thanks.
> 
> “Sometimes I think the surest sign that intelligent life exists elsewhere
in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin & Hobbes)
> 
> From: "David Portabella" <david.portabella@gmail.com>
> To: puppet-users@googlegroups.com
> Sent: Wednesday, November 27, 2013 10:12:48 AM
> Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output
> 
> Thanks again! It is great to use the puppet report yaml files!
> here there is an example script that prints all resources statuses, 
> then it filters them by taking only the services,
> then it filters them by taking only the services changed to running.
> 
> test.ruby
> #!/usr/bin/ruby 
> 
> require ''puppet'' 
> filename = ARGV.first 
> report = YAML.load_file(filename) 
> rs = report.resource_statuses.values.sort_by{|r| r.resource}
> 
> puts "PRINT ALL RESOURCES STATUSES."
> rs.each do |r| 
>   puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
> end 
> 
> puts "PRINT ALL SERVICES."
> services = rs.select{|r| r.resource_type == ''Service''}
> services.each do |r| 
>   puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
> end 
> 
> puts "PRINT ALL SERVICES CHANGED TO RUNNING."
> services_changed_to_running = rs.select{|r| r.resource_type ==
''Service'' && r.change_count > 0 &&
r.events.index { |e| e.desired_value == :running } != nil}
> services_changed_to_running.each do |r| 
>   puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
> end 
>  
> example output:
> $ test.ruby /var/opt/lib/pe-puppet/reports/test/201311261645.yaml 
> PRINT ALL RESOURCES STATUSES.
> 1 [:absent] [:directory] File[/var/app]
> 1 [:absent] [:file] File[/var/app/app.conf]
> 2 [0, "755"] [91, "775"] File[/var/log/tomcat6]
> 0 [] [] Filebucket[puppet]
> 1 [:absent] [:present] Group[release]
> 1 [:absent] [:present] Package[varnish]
> 0 [] [] Schedule[weekly]
> 1 [:stopped] [:running] Service[varnish]
> 0 [] [] Tidy[/etc/collectd.d/]
> 1 ["/sbin/nologin"] ["/bin/bash"] User[tomcat]
> ...
> 
> PRINT ALL SERVICES.
> 1 [:stopped] [:running] Service[collectd]
> 1 [:stopped] [:running] Service[logstash-agent]
> 1 [:true] [:false] Service[logstash]
> 1 [:stopped] [:running] Service[nginx]
> 0 [] [] Service[rsyslog]
> 1 [:stopped] [:running] Service[statsd]
> 1 [:false] [:true] Service[supervisord]
> 1 [:stopped] [:running] Service[varnish]
> 
> PRINT ALL SERVICES CHANGED TO RUNNING
> 1 [:stopped] [:running] Service[collectd]
> 1 [:stopped] [:running] Service[logstash-agent]
> 1 [:stopped] [:running] Service[nginx]
> 1 [:stopped] [:running] Service[statsd]
> 1 [:stopped] [:running] Service[varnish]
> 
> -- 
> You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/c70db1f6-b133-4290-a8e1-827ea239028b%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1253465309.4764508.1385567092577.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/36085646-13B6-4457-A36B-B2C8F97A603E%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

This one: http://shop.oreilly.com/product/9780596529864.do ? 


“Sometimes I think the surest sign that intelligent life exists elsewhere in the
universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes) 

----- Original Message -----
From: "Stuart Cracraft" <smcracraft@me.com> 
To: puppet-users@googlegroups.com 
Sent: Wednesday, November 27, 2013 11:00:00 AM 
Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output


Get the book Learning Ruby. 


Worth it. 

On Nov 27, 2013, at 7:44 AM, Dan White < ygor@comcast.net > wrote: 





Very nice. 
I am still learning Ruby and this will help 

Thanks. 


“Sometimes I think the surest sign that intelligent life exists elsewhere in the
universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes) 

----- Original Message -----
From: "David Portabella" < david.portabella@gmail.com > 
To: puppet-users@googlegroups.com 
Sent: Wednesday, November 27, 2013 10:12:48 AM 
Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output



Thanks again! It is great to use the puppet report yaml files! 
here there is an example script that prints all resources statuses, 

then it filters them by taking only the services, 
then it filters them by taking only the services changed to running. 


test.ruby 
#!/usr/bin/ruby 


require ''puppet'' 
filename = ARGV.first 
report = YAML.load_file(filename) 
rs = report.resource_statuses.values.sort_by{|r| r.resource} 


puts "PRINT ALL RESOURCES STATUSES." 
rs.each do |r| 
puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 


puts "PRINT ALL SERVICES." 
services = rs.select{|r| r.resource_type == ''Service''} 
services.each do |r| 
puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 


puts "PRINT ALL SERVICES CHANGED TO RUNNING." 
services_changed_to_running = rs.select{|r| r.resource_type ==
''Service'' && r.change_count > 0 &&
r.events.index { |e| e.desired_value == :running } != nil}
services_changed_to_running.each do |r| 
puts "#{r.change_count} #{r.events.map {|e| e.previous_value}}
#{r.events.map {|e| e.desired_value}} #{r.resource}"
end 

example output: 
$ test.ruby /var/opt/lib/pe-puppet/reports/test/201311261645.yaml 
PRINT ALL RESOURCES STATUSES. 
1 [:absent] [:directory] File[/var/app] 
1 [:absent] [:file] File[/var/app/app.conf] 
2 [0, "755"] [91, "775"] File[/var/log/tomcat6] 
0 [] [] Filebucket[puppet] 
1 [:absent] [:present] Group[release] 
1 [:absent] [:present] Package[varnish] 
0 [] [] Schedule[weekly] 
1 [:stopped] [:running] Service[varnish] 
0 [] [] Tidy[/etc/collectd.d/] 
1 ["/sbin/nologin"] ["/bin/bash"] User[tomcat] 
... 


PRINT ALL SERVICES. 
1 [:stopped] [:running] Service[collectd] 
1 [:stopped] [:running] Service[logstash-agent] 
1 [:true] [:false] Service[logstash] 
1 [:stopped] [:running] Service[nginx] 
0 [] [] Service[rsyslog] 
1 [:stopped] [:running] Service[statsd] 
1 [:false] [:true] Service[supervisord] 
1 [:stopped] [:running] Service[varnish] 


PRINT ALL SERVICES CHANGED TO RUNNING 
1 [:stopped] [:running] Service[collectd] 
1 [:stopped] [:running] Service[logstash-agent] 
1 [:stopped] [:running] Service[nginx] 
1 [:stopped] [:running] Service[statsd] 
1 [:stopped] [:running] Service[varnish] 



-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com .
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/c70db1f6-b133-4290-a8e1-827ea239028b%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out . 


-- 
You received this message because you are subscribed to a topic in the Google
Groups "Puppet Users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe .
To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com .
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/1253465309.4764508.1385567092577.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net
.
For more options, visit https://groups.google.com/groups/opt_out . 




-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/36085646-13B6-4457-A36B-B2C8F97A603E%40me.com
.
For more options, visit https://groups.google.com/groups/opt_out . 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/133023313.4765018.1385568524350.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net.
For more options, visit https://groups.google.com/groups/opt_out.

Given this puppet manifest (test.pp):
$dir = ''/tmp''
file {''myfile'':
  path => "$dir/myfile.txt",
  content => ''hello''
}

puppet produces this report:
[...]
    File[myfile]: !ruby/object:Puppet::Resource::Status
      resource: File[myfile]
      file: /Users/david/test.pp
      line: 4
      evaluation_time: 0.001354
      change_count: 1
      out_of_sync_count: 1
      tags: 
        - file
        - myfile
        - class
      time: 2013-12-13 11:44:59.092716 +01:00
      events: 
        - !ruby/object:Puppet::Transaction::Event
          audited: false
          property: ensure
          previous_value: !ruby/sym absent
          desired_value: !ruby/sym file
          historical_value: 
          message: "defined content as 
''{md5}5d41402abc4b2a76b9719d911017c592''"
          name: !ruby/sym file_created
          status: success
          time: 2013-12-13 11:44:59.093067 +01:00
      out_of_sync: true
      changed: true
      resource_type: File
      title: myfile
      skipped: false
      failed: false
      containment_path: 
        - Stage[main]
        - ""
        - File[myfile]
[...]

so, the report tells that it has created File[myfile],
but it does not contain the path of the file /tmp/myfile.txt.

that''s a pity, because it means that I cannot get a list of all the
files
updated by puppet.

is there a way to get the final path of all File resources created by 
puppet?
(not "$dir/myfile.txt", but "/tmp/myfile.txt")

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/ff8d4db1-29b2-4c0e-aca8-a9d9faf4e02f%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

You ask a great question.

Also, it would be great if we could get the actual Unix cli equivalent of 
what Puppet is proposing to do.

On Friday, December 13, 2013 5:30:19 AM UTC-8, David Portabella wrote:
> Given this puppet manifest (test.pp):
> $dir = ''/tmp''
> file {''myfile'':
>   path => "$dir/myfile.txt",
>   content => ''hello''
> }
>
> puppet produces this report:
> [...]
>     File[myfile]: !ruby/object:Puppet::Resource::Status
>       resource: File[myfile]
>       file: /Users/david/test.pp
>       line: 4
>       evaluation_time: 0.001354
>       change_count: 1
>       out_of_sync_count: 1
>       tags: 
>         - file
>         - myfile
>         - class
>       time: 2013-12-13 11:44:59.092716 +01:00
>       events: 
>         - !ruby/object:Puppet::Transaction::Event
>           audited: false
>           property: ensure
>           previous_value: !ruby/sym absent
>           desired_value: !ruby/sym file
>           historical_value: 
>           message: "defined content as 
> ''{md5}5d41402abc4b2a76b9719d911017c592''"
>           name: !ruby/sym file_created
>           status: success
>           time: 2013-12-13 11:44:59.093067 +01:00
>       out_of_sync: true
>       changed: true
>       resource_type: File
>       title: myfile
>       skipped: false
>       failed: false
>       containment_path: 
>         - Stage[main]
>         - ""
>         - File[myfile]
> [...]
>
> so, the report tells that it has created File[myfile],
> but it does not contain the path of the file /tmp/myfile.txt.
>
> that''s a pity, because it means that I cannot get a list of all
the files
> updated by puppet.
>
> is there a way to get the final path of all File resources created by 
> puppet?
> (not "$dir/myfile.txt", but "/tmp/myfile.txt")
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/8b0f3577-d5e4-4a55-abdb-cc15cbe949de%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On Friday, December 13, 2013 5:30:19 AM UTC-8, David Portabella wrote:

Given this puppet manifest (test.pp):
> $dir = ''/tmp''
> file {''myfile'':
>   path => "$dir/myfile.txt",
>   content => ''hello''
> }
>
> puppet produces this report:
> [...]
>     File[myfile]: !ruby/object:Puppet::Resource::Status
>       resource: File[myfile]
>       file: /Users/david/test.pp
>       line: 4
>       evaluation_time: 0.001354
>       change_count: 1
>       out_of_sync_count: 1
>       tags: 
>         - file
>         - myfile
>         - class
>       time: 2013-12-13 11:44:59.092716 +01:00
>       events: 
>         - !ruby/object:Puppet::Transaction::Event
>           audited: false
>           property: ensure
>           previous_value: !ruby/sym absent
>           desired_value: !ruby/sym file
>           historical_value: 
>           message: "defined content as 
> ''{md5}5d41402abc4b2a76b9719d911017c592''"
>           name: !ruby/sym file_created
>           status: success
>           time: 2013-12-13 11:44:59.093067 +01:00
>       out_of_sync: true
>       changed: true
>       resource_type: File
>       title: myfile
>       skipped: false
>       failed: false
>       containment_path: 
>         - Stage[main]
>         - ""
>         - File[myfile]
> [...]
>
> so, the report tells that it has created File[myfile],
> but it does not contain the path of the file /tmp/myfile.txt.
>
> that''s a pity, because it means that I cannot get a list of all
the files
> updated by puppet.
>
> is there a way to get the final path of all File resources created by 
> puppet?
> (not "$dir/myfile.txt", but "/tmp/myfile.txt")
>
 The report is focused on describing the changes that occurred during that 
run. To that end, only the title is included as that information is 
sufficient to uniquely identify the reporting resource when combined with 
resource_type. Information concerning properties that were not changed 
during the run is omitted as this data can be obtained from the catalog 
that the agent was processing.

In this specific situation, you can modify your file resource such that the 
path is used as the title:

$dir = ''/tmp''
file { "$dir/myfile.txt":
  content => ''hello''
}

However, the general problem of matching changes to resources can be solved 
by loading the report and then loading the catalog and joining the two 
datasets using the resource_type and title. However, this particular task 
is one of the problems PuppetDB was designed to solve. PuppetDB stores both 
the catalogs and reports and provides an API that can be used to query the 
data.

For example, the changes related to myfile can be retrieved by querying the
events
endpoint
<https://docs.puppetlabs.com/puppetdb/1.5/api/query/v3/events.html>of the
PuppetDB API:

curl -G ''http://localhost:8080/v3/events'' --data-urlencode
query=\''["and", ["=", "resource-type",
"File"],
  ["=", "containing-class", "Testfile"],
  ["=", "certname",
"pe-310-agent.puppetdebug.vlan"]]''

Which gives results similar to the following:

[ {

  "status" : "success",
  "timestamp" : "2013-12-13T17:11:39.144Z",
  "certname" : "pe-310-agent.puppetdebug.vlan",
  "containing-class" : "Testfile",
  "containment-path" : [ "Stage[main]",
"Testfile", "File[myfile]" ],
  "report" : "6bed5163b50b5857921b5ec27d9147b428c684f8",
  "run-start-time" : "2013-12-13T17:11:29.382Z",
  "resource-title" : "myfile",
  "configuration-version" : "1386954691",
  "run-end-time" : "2013-12-13T17:11:36.527Z",
  "property" : "ensure",
  "message" : "defined content as
''{md5}5d41402abc4b2a76b9719d911017c592''",
  "new-value" : "file",
  "old-value" : "absent",
  "line" : 44,
  "file" :
"/etc/puppetlabs/puppet/modules/testfile/manifests/init.pp",
  "report-receive-time" : "2013-12-13T17:11:41.334Z",
  "resource-type" : "File"} ]

The resource contained in the catalog delivered to that node, which 
contains properties such as the path, can be retrieved by passing the 
resource Type and title to the resources
endpoint<https://docs.puppetlabs.com/puppetdb/1.5/api/query/v3/resources.html>
:

curl -G ''http://localhost:8080/v3/resources/File/myfile''
--data-urlencode ''query=["=", "certname",
"pe-310-agent.puppetdebug.vlan"]''

This returns all the parameters of interest:

[ {
  "parameters" : {
    "path" : "/tmp/myfile.txt",
    "mode" : "0755",
    "content" : "hello",
    "backup" : "main",
    "alias" : [ "/tmp/myfile.txt" ]
  },
  "line" : 45,
  "file" :
"/etc/puppetlabs/puppet/modules/testfile/manifests/init.pp",
  "exported" : false,
  "tags" : [ "default", "node",
"myfile", "testfile", "class", "file" ],
  "title" : "myfile",
  "type" : "File",
  "resource" : "93f90701c8f54a485246a9e3725040f1992fd90b",
  "certname" : "pe-310-agent.puppetdebug.vlan"} ]

The power of PuppetDB in this situation is that your reporting script can 
now focus on analyzing the data instead of finding and then filtering it.

Hope this helps!
-- 
Charlie Sharpsteen
Open Source Support Engineer
Puppet Labs

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/a4dba39f-5d95-437f-b252-57a44d1e4f03%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

wow, thanks for this very exhaustive answer!

is there a way to get the list of resources created when using a puppet 
apply (instead of puppet agent)?

the point is that we need this when refactoring puppet modules, in order to 
test the modules in a vagrant machine and check that there are not 
regression issues.
so, i run our current puppet modules in a vagrant machine, get the list of 
all puppet resources created,
then i refactor the puppet modules, i run again the refactored puppet 
modules in a new vagrant machine,
and i compare all the files and other resources.


On Friday, December 13, 2013 7:32:39 PM UTC+1, Charlie Sharpsteen
wrote:
>
> On Friday, December 13, 2013 5:30:19 AM UTC-8, David Portabella wrote:
>
> Given this puppet manifest (test.pp):
>> $dir = ''/tmp''
>> file {''myfile'':
>>   path => "$dir/myfile.txt",
>>   content => ''hello''
>> }
>>
>> puppet produces this report:
>> [...]
>>     File[myfile]: !ruby/object:Puppet::Resource::Status
>>       resource: File[myfile]
>>       file: /Users/david/test.pp
>>       line: 4
>>       evaluation_time: 0.001354
>>       change_count: 1
>>       out_of_sync_count: 1
>>       tags: 
>>         - file
>>         - myfile
>>         - class
>>       time: 2013-12-13 11:44:59.092716 +01:00
>>       events: 
>>         - !ruby/object:Puppet::Transaction::Event
>>           audited: false
>>           property: ensure
>>           previous_value: !ruby/sym absent
>>           desired_value: !ruby/sym file
>>           historical_value: 
>>           message: "defined content as 
>> ''{md5}5d41402abc4b2a76b9719d911017c592''"
>>           name: !ruby/sym file_created
>>           status: success
>>           time: 2013-12-13 11:44:59.093067 +01:00
>>       out_of_sync: true
>>       changed: true
>>       resource_type: File
>>       title: myfile
>>       skipped: false
>>       failed: false
>>       containment_path: 
>>         - Stage[main]
>>         - ""
>>         - File[myfile]
>> [...]
>>
>> so, the report tells that it has created File[myfile],
>> but it does not contain the path of the file /tmp/myfile.txt.
>>
>> that''s a pity, because it means that I cannot get a list of
all the files
>> updated by puppet.
>>
>> is there a way to get the final path of all File resources created by 
>> puppet?
>> (not "$dir/myfile.txt", but "/tmp/myfile.txt")
>>
>  The report is focused on describing the changes that occurred during that 
> run. To that end, only the title is included as that information is 
> sufficient to uniquely identify the reporting resource when combined with 
> resource_type. Information concerning properties that were not changed 
> during the run is omitted as this data can be obtained from the catalog 
> that the agent was processing.
>
> In this specific situation, you can modify your file resource such that 
> the path is used as the title:
>
> $dir = ''/tmp''
> file { "$dir/myfile.txt":
>   content => ''hello''
> }
>
> However, the general problem of matching changes to resources can be 
> solved by loading the report and then loading the catalog and joining the 
> two datasets using the resource_type and title. However, this particular 
> task is one of the problems PuppetDB was designed to solve. PuppetDB stores
> both the catalogs and reports and provides an API that can be used to query
> the data.
>
> For example, the changes related to myfile can be retrieved by querying 
> the events
endpoint<https://docs.puppetlabs.com/puppetdb/1.5/api/query/v3/events.html>of
the PuppetDB API:
>
> curl -G ''http://localhost:8080/v3/events''
--data-urlencode query=\''["and", ["=",
"resource-type", "File"],
>   ["=", "containing-class", "Testfile"],
>   ["=", "certname",
"pe-310-agent.puppetdebug.vlan"]]''
>
> Which gives results similar to the following:
>
> [ {
>
>   "status" : "success",
>   "timestamp" : "2013-12-13T17:11:39.144Z",
>   "certname" : "pe-310-agent.puppetdebug.vlan",
>   "containing-class" : "Testfile",
>   "containment-path" : [ "Stage[main]",
"Testfile", "File[myfile]" ],
>   "report" :
"6bed5163b50b5857921b5ec27d9147b428c684f8",
>   "run-start-time" : "2013-12-13T17:11:29.382Z",
>   "resource-title" : "myfile",
>   "configuration-version" : "1386954691",
>   "run-end-time" : "2013-12-13T17:11:36.527Z",
>   "property" : "ensure",
>   "message" : "defined content as
''{md5}5d41402abc4b2a76b9719d911017c592''",
>   "new-value" : "file",
>   "old-value" : "absent",
>   "line" : 44,
>   "file" :
"/etc/puppetlabs/puppet/modules/testfile/manifests/init.pp",
>   "report-receive-time" : "2013-12-13T17:11:41.334Z",
>   "resource-type" : "File"} ]
>
> The resource contained in the catalog delivered to that node, which 
> contains properties such as the path, can be retrieved by passing the 
> resource Type and title to the resources
endpoint<https://docs.puppetlabs.com/puppetdb/1.5/api/query/v3/resources.html>
> :
>
> curl -G ''http://localhost:8080/v3/resources/File/myfile''
--data-urlencode ''query=["=", "certname",
"pe-310-agent.puppetdebug.vlan"]''
>
> This returns all the parameters of interest:
>
> [ {
>   "parameters" : {
>     "path" : "/tmp/myfile.txt",
>     "mode" : "0755",
>     "content" : "hello",
>     "backup" : "main",
>     "alias" : [ "/tmp/myfile.txt" ]
>   },
>   "line" : 45,
>   "file" :
"/etc/puppetlabs/puppet/modules/testfile/manifests/init.pp",
>   "exported" : false,
>   "tags" : [ "default", "node",
"myfile", "testfile", "class", "file" ],
>   "title" : "myfile",
>   "type" : "File",
>   "resource" :
"93f90701c8f54a485246a9e3725040f1992fd90b",
>   "certname" : "pe-310-agent.puppetdebug.vlan"} ]
>
> The power of PuppetDB in this situation is that your reporting script can 
> now focus on analyzing the data instead of finding and then filtering it.
>
> Hope this helps!
> -- 
> Charlie Sharpsteen
> Open Source Support Engineer
> Puppet Labs
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/d2a0ca3e-cb85-439c-a908-7efdd80f9b43%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

On Friday, December 13, 2013 2:43:41 PM UTC-8, David Portabella wrote:

> is there a way to get the list of resources created when using a puppet 
> apply (instead of puppet agent)?
>
> the point is that we need this when refactoring puppet modules, in order 
> to test the modules in a vagrant machine and check that there are not 
> regression issues.
> so, i run our current puppet modules in a vagrant machine, get the list of 
> all puppet resources created,
> then i refactor the puppet modules, i run again the refactored puppet 
> modules in a new vagrant machine,
> and i compare all the files and other resources.
>
 If you are looking to keep track of resources while refactoring, then 
there are a few approaches you can take:

   1. Add another VM to your Vagrant environment that runs a Puppet Master 
   and do the refactoring there. This gives you access to tools such as 
   PuppetDB and the Dashboard while also simulating how your refactor will 
   affect an agent/master setup. 
   2. Set up your VM such that puppet apply submits the catalog and reports 
   to PuppetDB: 
   https://docs.puppetlabs.com/puppetdb/1.5/connect_puppet_apply.html 
   3. Configure configure YAML caching for puppet apply in 
   /etc/puppet/routes.yaml: 


---
apply:
  catalog:
    cache: yaml

This will cause puppet apply to save a copy of the last compiled catalog to 
/var/lib/puppet/state/client_yaml/catalog/<certname>.yaml. The information
in this file can be combined with the report in 
/var/lib/puppet/state/last_run_report.yaml to create a list of applied 
resources along with their properties and resulting changes.

The first two approaches will cut down on the amount of custom code you 
need to write as the task of parsing and storing the reports and catalogs 
is handled by PuppetDB or the dashboard. Approach 3 works as well and 
offers a lot of control, but you will need to write a bit of code that 
extracts data from the catalog and report files before you can focus on the 
analysis.
-- 
Charlie Sharpsteen
Open Source Support Engineer
Puppet Labs

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/6892e0dd-61a1-4728-b78f-2407e1124962%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi, thanks again for the info.

This seems an overcomplicated issue. Just to step back and look at the 
global picture:
we are just asking to get the list of all resources updated when executing 
"puppet apply" in a non master/agent puppet environment.
this should be a basic functionality; we are not asking something weird.

after Christmas I''ll take a look at installing PuppetDB in the same
virtual
machine, connecting puppet apply to it, and try to get the list of 
resources updated by puppet apply.


Regards,
David

On Sunday, December 15, 2013 8:59:37 PM UTC+1, Charlie Sharpsteen
wrote:
>
> On Friday, December 13, 2013 2:43:41 PM UTC-8, David Portabella wrote:
>
>
>> is there a way to get the list of resources created when using a puppet
>> apply (instead of puppet agent)?
>>
>> the point is that we need this when refactoring puppet modules, in
order
>> to test the modules in a vagrant machine and check that there are not 
>> regression issues.
>> so, i run our current puppet modules in a vagrant machine, get the list
>> of all puppet resources created,
>> then i refactor the puppet modules, i run again the refactored puppet 
>> modules in a new vagrant machine,
>> and i compare all the files and other resources.
>>
>  If you are looking to keep track of resources while refactoring, then 
> there are a few approaches you can take:
>
>    1. Add another VM to your Vagrant environment that runs a Puppet 
>    Master and do the refactoring there. This gives you access to tools such
as
>    PuppetDB and the Dashboard while also simulating how your refactor will 
>    affect an agent/master setup. 
>    2. Set up your VM such that puppet apply submits the catalog and 
>    reports to PuppetDB: 
>    https://docs.puppetlabs.com/puppetdb/1.5/connect_puppet_apply.html 
>    3. Configure configure YAML caching for puppet apply in 
>    /etc/puppet/routes.yaml: 
>
>
> ---
> apply:
>   catalog:
>     cache: yaml
>
> This will cause puppet apply to save a copy of the last compiled catalog 
> to /var/lib/puppet/state/client_yaml/catalog/<certname>.yaml. The 
> information in this file can be combined with the report in 
> /var/lib/puppet/state/last_run_report.yaml to create a list of applied 
> resources along with their properties and resulting changes.
>
> The first two approaches will cut down on the amount of custom code you 
> need to write as the task of parsing and storing the reports and catalogs 
> is handled by PuppetDB or the dashboard. Approach 3 works as well and 
> offers a lot of control, but you will need to write a bit of code that 
> extracts data from the catalog and report files before you can focus on the
> analysis.
> -- 
> Charlie Sharpsteen
> Open Source Support Engineer
> Puppet Labs
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/6d2793ce-0b9f-468b-bc35-76b149f31f46%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi,

this thread has been going on for a while, and you may have already
commented on this, in this case please humor me:

Couldn''t you collect and interpret the output of puppet apply?

On 12/16/2013 02:14 PM, David Portabella wrote:
> This seems an overcomplicated issue. Just to step back and look at the
> global picture:
> we are just asking to get the list of all resources updated when
> executing "puppet apply" in a non master/agent puppet
environment.
> this should be a basic functionality; we are not asking something weird.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/52AF0787.8040904%40alumni.tu-berlin.de.
For more options, visit https://groups.google.com/groups/opt_out.

hi,

yes, this script works quite well! :)
https://groups.google.com/d/msg/puppet-users/cHpZlKkPmr4/RRQ6VWsGdMQJ

However, we are currently using the "Oracle WebLogic / Fusion
Middleware"
puppet module,
http://forge.puppetlabs.com/biemond/wls/1.3.3

which declares some file resources with the path parameter, so the script 
no longer works because
the file''s $name and $path differ, and the report does not contain the 
information about the path of the resource.

example: orawls/manifests/domain.pp:

# the domain.py used by the wlst
file { "domain.py ${domain_name} ${title}":
  path    => "${download_dir}/domain_${domain_name}.py",
  ...


On Monday, December 16, 2013 3:00:39 PM UTC+1, Felix.Frank
wrote:
>
> Hi, 
>
> this thread has been going on for a while, and you may have already 
> commented on this, in this case please humor me: 
>
> Couldn''t you collect and interpret the output of puppet apply? 
>
> On 12/16/2013 02:14 PM, David Portabella wrote: 
> > This seems an overcomplicated issue. Just to step back and look at the
> > global picture: 
> > we are just asking to get the list of all resources updated when 
> > executing "puppet apply" in a non master/agent puppet
environment.
> > this should be a basic functionality; we are not asking something
weird.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/51d77a7e-ed0c-45cf-831b-26a8bdfe0cd1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

If that is all you want, run it with --write-catalog-summary and examine
the resourcefile (puppet apply --configprint resourcefile)


On 16 December 2013 14:14, David Portabella
<david.portabella@gmail.com>wrote:
> Hi, thanks again for the info.
>
> This seems an overcomplicated issue. Just to step back and look at the
> global picture:
> we are just asking to get the list of all resources updated when executing
> "puppet apply" in a non master/agent puppet environment.
> this should be a basic functionality; we are not asking something weird.
>
> after Christmas I''ll take a look at installing PuppetDB in the
same
> virtual machine, connecting puppet apply to it, and try to get the list of
> resources updated by puppet apply.
>
>
> Regards,
> David
>
> On Sunday, December 15, 2013 8:59:37 PM UTC+1, Charlie Sharpsteen wrote:
>>
>> On Friday, December 13, 2013 2:43:41 PM UTC-8, David Portabella wrote:
>>
>>
>>> is there a way to get the list of resources created when using a
puppet
>>> apply (instead of puppet agent)?
>>>
>>> the point is that we need this when refactoring puppet modules, in
order
>>> to test the modules in a vagrant machine and check that there are
not
>>> regression issues.
>>> so, i run our current puppet modules in a vagrant machine, get the
list
>>> of all puppet resources created,
>>> then i refactor the puppet modules, i run again the refactored
puppet
>>> modules in a new vagrant machine,
>>> and i compare all the files and other resources.
>>>
>>  If you are looking to keep track of resources while refactoring, then
>> there are a few approaches you can take:
>>
>>    1. Add another VM to your Vagrant environment that runs a Puppet
>>    Master and do the refactoring there. This gives you access to tools
such as
>>    PuppetDB and the Dashboard while also simulating how your refactor
will
>>    affect an agent/master setup.
>>    2. Set up your VM such that puppet apply submits the catalog and
>>    reports to PuppetDB: https://docs.puppetlabs.com/
>>   
puppetdb/1.5/connect_puppet_apply.html<https://docs.puppetlabs.com/puppetdb/1.5/connect_puppet_apply.html>
>>    3. Configure configure YAML caching for puppet apply in
>>    /etc/puppet/routes.yaml:
>>
>>
>> ---
>> apply:
>>   catalog:
>>     cache: yaml
>>
>> This will cause puppet apply to save a copy of the last compiled
catalog
>> to /var/lib/puppet/state/client_yaml/catalog/<certname>.yaml. The
>> information in this file can be combined with the report in
>> /var/lib/puppet/state/last_run_report.yaml to create a list of applied
>> resources along with their properties and resulting changes.
>>
>> The first two approaches will cut down on the amount of custom code you
>> need to write as the task of parsing and storing the reports and
catalogs
>> is handled by PuppetDB or the dashboard. Approach 3 works as well and
>> offers a lot of control, but you will need to write a bit of code that
>> extracts data from the catalog and report files before you can focus on
the
>> analysis.
>> --
>> Charlie Sharpsteen
>> Open Source Support Engineer
>> Puppet Labs
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/puppet-users/6d2793ce-0b9f-468b-bc35-76b149f31f46%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>


-- 
Erik Dalén

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAAAzDLdZKfuqaOSWzgZYuJFHCaJ6oYgnSbUQ3RcMe0hkivcvhw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Well, that is actually all resources managed rather than all resources
changed. The changed resources are in the report. In my case we use
puppetdb for report querying.


On 18 December 2013 17:41, Erik Dalén <erik.gustav.dalen@gmail.com> wrote:
> If that is all you want, run it with --write-catalog-summary and examine
> the resourcefile (puppet apply --configprint resourcefile)
>
>
> On 16 December 2013 14:14, David Portabella
<david.portabella@gmail.com>wrote:
>
>> Hi, thanks again for the info.
>>
>> This seems an overcomplicated issue. Just to step back and look at the
>> global picture:
>> we are just asking to get the list of all resources updated when
>> executing "puppet apply" in a non master/agent puppet
environment.
>> this should be a basic functionality; we are not asking something
weird.
>>
>> after Christmas I''ll take a look at installing PuppetDB in the
same
>> virtual machine, connecting puppet apply to it, and try to get the list
of
>> resources updated by puppet apply.
>>
>>
>> Regards,
>> David
>>
>> On Sunday, December 15, 2013 8:59:37 PM UTC+1, Charlie Sharpsteen
wrote:
>>>
>>> On Friday, December 13, 2013 2:43:41 PM UTC-8, David Portabella
wrote:
>>>
>>>
>>>> is there a way to get the list of resources created when using
a puppet
>>>> apply (instead of puppet agent)?
>>>>
>>>> the point is that we need this when refactoring puppet modules,
in
>>>> order to test the modules in a vagrant machine and check that
there are not
>>>> regression issues.
>>>> so, i run our current puppet modules in a vagrant machine, get
the list
>>>> of all puppet resources created,
>>>> then i refactor the puppet modules, i run again the refactored
puppet
>>>> modules in a new vagrant machine,
>>>> and i compare all the files and other resources.
>>>>
>>>  If you are looking to keep track of resources while refactoring,
then
>>> there are a few approaches you can take:
>>>
>>>    1. Add another VM to your Vagrant environment that runs a Puppet
>>>    Master and do the refactoring there. This gives you access to
tools such as
>>>    PuppetDB and the Dashboard while also simulating how your
refactor will
>>>    affect an agent/master setup.
>>>    2. Set up your VM such that puppet apply submits the catalog and
>>>    reports to PuppetDB: https://docs.puppetlabs.com/
>>>   
puppetdb/1.5/connect_puppet_apply.html<https://docs.puppetlabs.com/puppetdb/1.5/connect_puppet_apply.html>
>>>    3. Configure configure YAML caching for puppet apply in
>>>    /etc/puppet/routes.yaml:
>>>
>>>
>>> ---
>>> apply:
>>>   catalog:
>>>     cache: yaml
>>>
>>> This will cause puppet apply to save a copy of the last compiled
>>> catalog to
/var/lib/puppet/state/client_yaml/catalog/<certname>.yaml.
>>> The information in this file can be combined with the report in
>>> /var/lib/puppet/state/last_run_report.yaml to create a list of
applied
>>> resources along with their properties and resulting changes.
>>>
>>> The first two approaches will cut down on the amount of custom code
you
>>> need to write as the task of parsing and storing the reports and
catalogs
>>> is handled by PuppetDB or the dashboard. Approach 3 works as well
and
>>> offers a lot of control, but you will need to write a bit of code
that
>>> extracts data from the catalog and report files before you can
focus on the
>>> analysis.
>>> --
>>> Charlie Sharpsteen
>>> Open Source Support Engineer
>>> Puppet Labs
>>>
>>  --
>> You received this message because you are subscribed to the Google
Groups
>> "Puppet Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send
an
>> email to puppet-users+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>>
https://groups.google.com/d/msgid/puppet-users/6d2793ce-0b9f-468b-bc35-76b149f31f46%40googlegroups.com
>> .
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>
>
> --
> Erik Dalén
>


-- 
Erik Dalén

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CAAAzDLdTcE0nAObFJC0ZgPSEiXA2ST1hOz8g8LYkXEh1PC61EQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

What we are looking for is a Ruby program which takes the contents of

  /var/lib/puppet/reports/*/*.yaml

and reports in detail on everything changed or proposed for change if in 
noop mode
(file permissions, modes, user creates, etc.)

Stuart

On Wednesday, November 27, 2013 7:12:48 AM UTC-8, David Portabella wrote:
> Thanks again! It is great to use the puppet report yaml files!
> here there is an example script that prints all resources statuses, 
> then it filters them by taking only the services,
> then it filters them by taking only the services changed to running.
>
> test.ruby
> #!/usr/bin/ruby 
>
> require ''puppet'' 
> filename = ARGV.first 
> report = YAML.load_file(filename) 
> rs = report.resource_statuses.values.sort_by{|r| r.resource}
>
> puts "PRINT ALL RESOURCES STATUSES."
> rs.each do |r| 
>   puts "#{r.change_count} #{r.events.map {|e| e.previous_value}} 
> #{r.events.map {|e| e.desired_value}} #{r.resource}"
> end 
>
> puts "PRINT ALL SERVICES."
> services = rs.select{|r| r.resource_type == ''Service''}
> services.each do |r| 
>   puts "#{r.change_count} #{r.events.map {|e| e.previous_value}} 
> #{r.events.map {|e| e.desired_value}} #{r.resource}"
> end 
>
> puts "PRINT ALL SERVICES CHANGED TO RUNNING."
> services_changed_to_running = rs.select{|r| r.resource_type ==
''Service''
> && r.change_count > 0 && r.events.index { |e|
e.desired_value == :running }
> != nil}
> services_changed_to_running.each do |r| 
>   puts "#{r.change_count} #{r.events.map {|e| e.previous_value}} 
> #{r.events.map {|e| e.desired_value}} #{r.resource}"
> end 
>  
> example output:
> $ test.ruby /var/opt/lib/pe-puppet/reports/test/201311261645.yaml 
> PRINT ALL RESOURCES STATUSES.
> 1 [:absent] [:directory] File[/var/app]
> 1 [:absent] [:file] File[/var/app/app.conf]
> 2 [0, "755"] [91, "775"] File[/var/log/tomcat6]
> 0 [] [] Filebucket[puppet]
> 1 [:absent] [:present] Group[release]
> 1 [:absent] [:present] Package[varnish]
> 0 [] [] Schedule[weekly]
> 1 [:stopped] [:running] Service[varnish]
> 0 [] [] Tidy[/etc/collectd.d/]
> 1 ["/sbin/nologin"] ["/bin/bash"] User[tomcat]
> ...
>
> PRINT ALL SERVICES.
> 1 [:stopped] [:running] Service[collectd]
> 1 [:stopped] [:running] Service[logstash-agent]
> 1 [:true] [:false] Service[logstash]
> 1 [:stopped] [:running] Service[nginx]
> 0 [] [] Service[rsyslog]
> 1 [:stopped] [:running] Service[statsd]
> 1 [:false] [:true] Service[supervisord]
> 1 [:stopped] [:running] Service[varnish]
>
> PRINT ALL SERVICES CHANGED TO RUNNING
> 1 [:stopped] [:running] Service[collectd]
> 1 [:stopped] [:running] Service[logstash-agent]
> 1 [:stopped] [:running] Service[nginx]
> 1 [:stopped] [:running] Service[statsd]
> 1 [:stopped] [:running] Service[varnish]
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/37dc9802-5090-4502-aac2-c3f6421236fa%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Stuart,

Puppet Labs has a large professional service department that you might 
want to engage with these very specific requests.  I''m sure they can 
give you a hand with whatever you need done.

Jason

On 12/18/2013 12:55 PM, Stuart Cracraft wrote:
> What we are looking for is a Ruby program which takes the contents of
>
>    /var/lib/puppet/reports/*/*.yaml
>
> and reports in detail on everything changed or proposed for change if in
> noop mode
> (file permissions, modes, user creates, etc.)
>
> Stuart
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/52B1E2AB.7060909%40tacorp.net.
For more options, visit https://groups.google.com/groups/opt_out.

thanks. 

who is your contact?

I am not getting the help I need. 
> On Dec 18, 2013, at 10:00 AM, Jason Slagle <raistlin@tacorp.net>
wrote:
> 
> Hi Stuart,
> 
> Puppet Labs has a large professional service department that you might want
to engage with these very specific requests.  I''m sure they can give
you a hand with whatever you need done.
> 
> Jason
> 
>> On 12/18/2013 12:55 PM, Stuart Cracraft wrote:
>> What we are looking for is a Ruby program which takes the contents of
>> 
>>   /var/lib/puppet/reports/*/*.yaml
>> 
>> and reports in detail on everything changed or proposed for change if
in
>> noop mode
>> (file permissions, modes, user creates, etc.)
>> 
>> Stuart
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/52B1E2AB.7060909%40tacorp.net.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/36C6B660-92B8-4056-B82D-789C1B0AE7ED%40me.com.
For more options, visit https://groups.google.com/groups/opt_out.

Try this link: 

http://puppetlabs.com/services/consulting 


“Sometimes I think the surest sign that intelligent life exists elsewhere in the
universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes) 

----- Original Message -----
From: "Stuart Cracraft" <smcracraft@me.com> 
To: puppet-users@googlegroups.com 
Sent: Wednesday, December 18, 2013 1:01:42 PM 
Subject: Re: [Puppet Users] get a *structured* version of the puppet agent
output


thanks. 

who is your contact? 

I am not getting the help I need. 
> On Dec 18, 2013, at 10:00 AM, Jason Slagle <raistlin@tacorp.net>
wrote:
> 
> Hi Stuart, 
> 
> Puppet Labs has a large professional service department that you might want
to engage with these very specific requests. I''m sure they can give you
a hand with whatever you need done.
> 
> Jason 
> 
>> On 12/18/2013 12:55 PM, Stuart Cracraft wrote: 
>> What we are looking for is a Ruby program which takes the contents of 
>> 
>> /var/lib/puppet/reports/*/*.yaml 
>> 
>> and reports in detail on everything changed or proposed for change if
in
>> noop mode 
>> (file permissions, modes, user creates, etc.) 
>> 
>> Stuart 
> 
> -- 
> You received this message because you are subscribed to a topic in the
Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/52B1E2AB.7060909%40tacorp.net.
> For more options, visit https://groups.google.com/groups/opt_out. 
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/36C6B660-92B8-4056-B82D-789C1B0AE7ED%40me.com.
For more options, visit https://groups.google.com/groups/opt_out. 

-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/184556643.104256.1387394731423.JavaMail.root%40sz0126a.westchester.pa.mail.comcast.net.
For more options, visit https://groups.google.com/groups/opt_out.

It''s too expensive given our small company.

On Wednesday, December 18, 2013 11:25:31 AM UTC-8, Ygor
wrote:
>
> Try this link:
>
> http://puppetlabs.com/services/consulting
>
> “Sometimes I think the surest sign that intelligent life exists elsewhere 
> in the universe is that none of it has tried to contact us.”
> Bill Waterson (Calvin & Hobbes)
>
> ------------------------------
> *From: *"Stuart Cracraft" <smcra...@me.com
<javascript:>>
> *To: *puppet...@googlegroups.com <javascript:>
> *Sent: *Wednesday, December 18, 2013 1:01:42 PM
> *Subject: *Re: [Puppet Users] get a *structured* version of the puppet 
> agent output
>
>
> thanks. 
>
> who is your contact?
>
> I am not getting the help I need. 
>
> > On Dec 18, 2013, at 10:00 AM, Jason Slagle
<rais...@tacorp.net<javascript:>>
> wrote:
> > 
> > Hi Stuart,
> > 
> > Puppet Labs has a large professional service department that you might
> want to engage with these very specific requests.  I''m sure they
can give
> you a hand with whatever you need done.
> > 
> > Jason
> > 
> >> On 12/18/2013 12:55 PM, Stuart Cracraft wrote:
> >> What we are looking for is a Ruby program which takes the contents
of
> >> 
> >>   /var/lib/puppet/reports/*/*.yaml
> >> 
> >> and reports in detail on everything changed or proposed for change
if in
> >> noop mode
> >> (file permissions, modes, user creates, etc.)
> >> 
> >> Stuart
> > 
> > -- 
> > You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> > To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to 
> puppet-users...@googlegroups.com <javascript:>.
> > To view this discussion on the web visit 
>
https://groups.google.com/d/msgid/puppet-users/52B1E2AB.7060909%40tacorp.net
> .
> > For more options, visit https://groups.google.com/groups/opt_out.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to puppet-users...@googlegroups.com <javascript:>.
> To view this discussion on the web visit 
>
https://groups.google.com/d/msgid/puppet-users/36C6B660-92B8-4056-B82D-789C1B0AE7ED%40me.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/65902161-3831-4b46-8828-d80f2f3355f1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Stuart,

You have *very* specific requirements here. In these situations, there
are generally three options:
1) find someone else who''s done it and use theirs
2) learn how to do it yourself
3) pay someone else to do it
4) don''t do it

You posted your question (which I assume you researched before posting,
and did not find an appropriate solution) and nobody has yet stepped
forward to say "I have that," though you have been given many
examples,
some very detailed, and haven''t given back any indication that you
tried
them. If the prices that PL charges for consulting are too expensive,
and the same is true for hiring some random ruby coder, I think either
you (or someone at your company) need to learn ruby, or reassess how you
are using Puppet and your requirements. From your many previous posts,
it seems like you''re asking things of puppet that are relatively
uncommon.

There have been some very good responses in this thread, including the
(IMO) best, which is to use PuppetDB and its API, in which case you also
wouldn''t be restricted to Ruby. What have you tried so far? What issues
do you have with the various options?

This list has over 6,600 members, and you alone are responsible for a
whopping 10% (yes, one in ten) of the posts in the last month. If you
feel that the responses you receive are not adequate, perhaps you should
be more judicious in how quickly you appeal to the community for an
answer, how much detail you give (specifically "I tried this <link to
source code> and seem to be experiencing this <very specific
problem>"),
and give a better indication that you''ve actually tried the suggested
solutions, and what the results were.

On 12/18/2013 02:46 PM, Stuart Cracraft wrote:
> It''s too expensive given our small company.
>
> On Wednesday, December 18, 2013 11:25:31 AM UTC-8, Ygor wrote:
>
>     Try this link:
>
>     http://puppetlabs.com/services/consulting
>     <http://puppetlabs.com/services/consulting>
>
>     “Sometimes I think the surest sign that intelligent life exists
>     elsewhere in the universe is that none of it has tried to contact us.”
>     Bill Waterson (Calvin & Hobbes)
>
>    
------------------------------------------------------------------------
>     *From: *"Stuart Cracraft" <smcra...@me.com
<javascript:>>
>     *To: *puppet...@googlegroups.com <javascript:>
>     *Sent: *Wednesday, December 18, 2013 1:01:42 PM
>     *Subject: *Re: [Puppet Users] get a *structured* version of the
>     puppet agent output
>
>
>     thanks.
>
>     who is your contact?
>
>     I am not getting the help I need.
>
>     > On Dec 18, 2013, at 10:00 AM, Jason Slagle <rais...@tacorp.net
>     <javascript:>> wrote:
>     >
>     > Hi Stuart,
>     >
>     > Puppet Labs has a large professional service department that you
>     might want to engage with these very specific requests.  I''m
sure
>     they can give you a hand with whatever you need done.
>     >
>     > Jason
>     >
>     >> On 12/18/2013 12:55 PM, Stuart Cracraft wrote:
>     >> What we are looking for is a Ruby program which takes the
>     contents of
>     >>
>     >>   /var/lib/puppet/reports/*/*.yaml
>     >>
>     >> and reports in detail on everything changed or proposed for
>     change if in
>     >> noop mode
>     >> (file permissions, modes, user creates, etc.)
>     >>
>     >> Stuart
>     >
>     > --
>     > You received this message because you are subscribed to a topic
>     in the Google Groups "Puppet Users" group.
>     > To unsubscribe from this topic, visit
>     https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe
>    
<https://groups.google.com/d/topic/puppet-users/cHpZlKkPmr4/unsubscribe>.
>     > To unsubscribe from this group and all its topics, send an email
>     to puppet-users...@googlegroups.com <javascript:>.
>     > To view this discussion on the web visit
>    
https://groups.google.com/d/msgid/puppet-users/52B1E2AB.7060909%40tacorp.net
>    
<https://groups.google.com/d/msgid/puppet-users/52B1E2AB.7060909%40tacorp.net>.
>     > For more options, visit https://groups.google.com/groups/opt_out
>     <https://groups.google.com/groups/opt_out>.
>
>     -- 
>     You received this message because you are subscribed to the Google
>     Groups "Puppet Users" group.
>     To unsubscribe from this group and stop receiving emails from it,
>     send an email to puppet-users...@googlegroups.com <javascript:>.
>     To view this discussion on the web visit
>    
https://groups.google.com/d/msgid/puppet-users/36C6B660-92B8-4056-B82D-789C1B0AE7ED%40me.com
>    
<https://groups.google.com/d/msgid/puppet-users/36C6B660-92B8-4056-B82D-789C1B0AE7ED%40me.com>.
>     For more options, visit https://groups.google.com/groups/opt_out
>     <https://groups.google.com/groups/opt_out>.
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users+unsubscribe@googlegroups.com.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/puppet-users/65902161-3831-4b46-8828-d80f2f3355f1%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
-- 
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/52B20A78.4040700%40jasonantman.com.
For more options, visit https://groups.google.com/groups/opt_out.