Hi all, Can anyone recommend any tools for Puppet report aggregation? I''m interested in something that can take a given set of Puppet reports and summarise to me what resources have changed across all hosts. If nothing exists I will look to write one myself. In that case, is Puppet report format 3 valid for Puppet 3.0? http://projects.puppetlabs.com/projects/puppet/wiki/Report_Format_3 Thanks, -Luke -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Fcx6zByYGPQJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
You could use foreman for that? filtering the hosts via search should allow you to find the exact resources you are looking for? Ohad On Tue, Dec 4, 2012 at 11:00 PM, Luke Bigum <Luke.Bigum@lmax.com> wrote:> Hi all, > > Can anyone recommend any tools for Puppet report aggregation? I''m > interested in something that can take a given set of Puppet reports and > summarise to me what resources have changed across all hosts. > > If nothing exists I will look to write one myself. In that case, is Puppet > report format 3 valid for Puppet 3.0? > > http://projects.puppetlabs.com/projects/puppet/wiki/Report_Format_3 > > Thanks, > > -Luke > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/puppet-users/-/Fcx6zByYGPQJ. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wednesday, December 5, 2012 12:50:43 PM UTC, ohad wrote:> You could use foreman for that? filtering the hosts via search should > allow you to find the exact resources you are looking for? > > Ohad > >Hi Ohad, I haven''t looked at The Foreman in a while but in my mind it''s more like Puppet Dashboard - correct me if I''m wrong. What I''m aiming for is a tool that can aid change / release management where we run Puppet --noop across the estate, gather all the reports, then summarise what changes will be applied (resolv.conf changes on all hosts, fstab changes on 20 hosts, service X refreshes on Y hosts). I don''t really want to be searching for explicit resources changing across hosts, it''s the resources I don''t know about that worry me ;-) Is the foreman worth a look in this case? -Luke> > On Tue, Dec 4, 2012 at 11:00 PM, Luke Bigum <Luke....@lmax.com<javascript:> > > wrote: > >> Hi all, >> >> Can anyone recommend any tools for Puppet report aggregation? I''m >> interested in something that can take a given set of Puppet reports and >> summarise to me what resources have changed across all hosts. >> >> If nothing exists I will look to write one myself. In that case, is >> Puppet report format 3 valid for Puppet 3.0? >> >> http://projects.puppetlabs.com/projects/puppet/wiki/Report_Format_3 >> >> Thanks, >> >> -Luke >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/puppet-users/-/Fcx6zByYGPQJ. >> To post to this group, send email to puppet...@googlegroups.com<javascript:> >> . >> To unsubscribe from this group, send email to >> puppet-users...@googlegroups.com <javascript:>. >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/zk_ospPVmYkJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Thu, Dec 6, 2012 at 11:29 AM, Luke Bigum <Luke.Bigum@lmax.com> wrote:> On Wednesday, December 5, 2012 12:50:43 PM UTC, ohad wrote: > >> You could use foreman for that? filtering the hosts via search should >> allow you to find the exact resources you are looking for? >> >> Ohad >> >> > Hi Ohad, > > I haven''t looked at The Foreman in a while but in my mind it''s more like > Puppet Dashboard - correct me if I''m wrong. What I''m aiming for is a tool > that can aid change / release management where we run Puppet --noop across > the estate, gather all the reports, then summarise what changes will be > applied (resolv.conf changes on all hosts, fstab changes on 20 hosts, > service X refreshes on Y hosts). > >You should be able to setup a search term that would match your requirement it might be best to use the API for that (http://bit.ly/Uq033I), since you would get all of your reports already parsed and you could cross reference the info with a puppet env, fact etc. Ohad> I don''t really want to be searching for explicit resources changing across > hosts, it''s the resources I don''t know about that worry me ;-) Is the > foreman worth a look in this case? > > -Luke > > >> >> On Tue, Dec 4, 2012 at 11:00 PM, Luke Bigum <Luke....@lmax.com> wrote: >> >>> Hi all, >>> >>> Can anyone recommend any tools for Puppet report aggregation? I''m >>> interested in something that can take a given set of Puppet reports and >>> summarise to me what resources have changed across all hosts. >>> >>> If nothing exists I will look to write one myself. In that case, is >>> Puppet report format 3 valid for Puppet 3.0? >>> >>> http://projects.puppetlabs.**com/projects/puppet/wiki/**Report_Format_3<http://projects.puppetlabs.com/projects/puppet/wiki/Report_Format_3> >>> >>> Thanks, >>> >>> -Luke >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Users" group. >>> To view this discussion on the web visit https://groups.google.com/d/** >>> msg/puppet-users/-/**Fcx6zByYGPQJ<https://groups.google.com/d/msg/puppet-users/-/Fcx6zByYGPQJ> >>> . >>> To post to this group, send email to puppet...@googlegroups.com. >>> To unsubscribe from this group, send email to puppet-users...@** >>> googlegroups.com. >>> >>> For more options, visit this group at http://groups.google.com/** >>> group/puppet-users?hl=en<http://groups.google.com/group/puppet-users?hl=en> >>> . >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/puppet-users/-/zk_ospPVmYkJ. > > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 6 December 2012 20:29, Luke Bigum <Luke.Bigum@lmax.com> wrote:> I haven''t looked at The Foreman in a while but in my mind it''s more like > Puppet Dashboard - correct me if I''m wrong. What I''m aiming for is a tool > that can aid change / release management where we run Puppet --noop across > the estate, gather all the reports, then summarise what changes will be > applied (resolv.conf changes on all hosts, fstab changes on 20 hosts, > service X refreshes on Y hosts). > > I don''t really want to be searching for explicit resources changing across > hosts, it''s the resources I don''t know about that worry me ;-) Is the > foreman worth a look in this case? > > Luke, we use the puppet dashboard which aggregates all the reports andthen lets us suck down a CSV ("Export nodes as CSV" on front page) which contains a status of all resources on all machine reporting. We run puppet in noop all the time, so need similar reports you are requesting. It is just a matter of slicing & dicing the csv to get what you want % wget http://localhost:3000/nodes.csv # Omit all the resources in sync % egrep -v '',0,0,false,false'' nodes.csv | wc -l 146057 % wc -l nodes.csv 612762 nodes.csv % echo "scale=2; 146057*100/612762" | bc 23.83 Almost 24% of all resources (600K mind you) out of sync We can check our security module compliance: % egrep -v '',0,0,false,false'' nodes.csv | grep modules/security|wc -l 6992 % grep modules/security nodes.csv|wc -l 30433 % echo "scale=2; 6992*100/130433" | bc 5.36 5% *resources* out of sync from our security module. Not bad. We can dice it to give percentages of hostnames too Top 10 resources out of sync: % egrep -v '',0,0,false,false'' nodes.csv | awk -F, ''{print $7, $8}'' | sort | uniq -c | sort -rn | head 898 File /etc/sudoers 801 File /opt/local/sbin/run-puppet.sh 792 File ntp.conf 754 File vposend-test.pl 752 File /opt/local/sbin/puppet-register.sh 751 File /var/puppet/etc/local_puppet_info 745 File /etc/default/passwd 711 File /local/apps 710 File /etc/default/nfs 705 File /etc/ssh/sshd_config John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Thursday, December 6, 2012 10:07:43 PM UTC, John Warburton wrote:> On 6 December 2012 20:29, Luke Bigum <Luke....@lmax.com <javascript:>>wrote: > >> I haven''t looked at The Foreman in a while but in my mind it''s more like >> Puppet Dashboard - correct me if I''m wrong. What I''m aiming for is a tool >> that can aid change / release management where we run Puppet --noop across >> the estate, gather all the reports, then summarise what changes will be >> applied (resolv.conf changes on all hosts, fstab changes on 20 hosts, >> service X refreshes on Y hosts). >> >> I don''t really want to be searching for explicit resources changing >> across hosts, it''s the resources I don''t know about that worry me ;-) Is >> the foreman worth a look in this case? >> >> Luke, we use the puppet dashboard which aggregates all the reports and > then lets us suck down a CSV ("Export nodes as CSV" on front page) which > contains a status of all resources on all machine reporting. We run puppet > in noop all the time, so need similar reports you are requesting. It is > just a matter of slicing & dicing the csv to get what you want > > % wget http://localhost:3000/nodes.csv > >Thanks John and Ohad, I use Puppet Dashboard but I''ve never tried that control before ;-) That should do as a very good start. Cheers, -Luke -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/VIHTWkFeoQQJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.