hi I''ve read about PuppetReporting at the Trac website. Apparently you are working on integrating the benchmark tests for every type in ruby. Do you have any idea when this will all get implemented? This is a crucial part (and actually a necessary feature) in the configuration management system we have set up using Puppet. grtz Koen Vereeken _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
On Jan 30, 2007, at 11:00 AM, Koen Vereeken wrote:> hi > > I''ve read about PuppetReporting at the Trac website. Apparently you > are working on integrating the benchmark tests for every type in ruby. > Do you have any idea when this will all get implemented? > This is a crucial part (and actually a necessary feature) in the > configuration management system we have set up using Puppet.Hmm. That document is quite old (probably a year old now). There is already some reporting in Puppet: https://reductivelabs.com/cgi-bin/puppet.cgi/wiki/ReportsAndReporting I won''t say it''s the best or anything, but it''s a start. We''ll be working on integrating it into PuppetShow once the next release (kermit) comes out, which will be this week. What are you hoping to do with reporting? One of the reasons I''ve made so little progress is that I can''t seem to get anyone to express any interest in reporting, which seems a bit weird to me. -- You only have to be open minded if you''re wrong. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
2007/1/30, Luke Kanies <luke@madstop.com>:> > On Jan 30, 2007, at 11:00 AM, Koen Vereeken wrote: > > > hi > > > > I''ve read about PuppetReporting at the Trac website. Apparently you > > are working on integrating the benchmark tests for every type in ruby. > > Do you have any idea when this will all get implemented? > > This is a crucial part (and actually a necessary feature) in the > > configuration management system we have set up using Puppet. > > Hmm. That document is quite old (probably a year old now). There is > already some reporting in Puppet: > > https://reductivelabs.com/cgi-bin/puppet.cgi/wiki/ReportsAndReportingI''ve read about this, but the RRDtool doesn''t give enough information for me. Also I''m not convinced to keep track of machines with email reporting (email depends on a service that''s currently not installed on the puppetmaster, I don''t want to install it just for Puppet). I won''t say it''s the best or anything, but it''s a start. We''ll be> working on integrating it into PuppetShow once the next release > (kermit) comes out, which will be this week. > > What are you hoping to do with reporting? One of the reasons I''ve > made so little progress is that I can''t seem to get anyone to express > any interest in reporting, which seems a bit weird to me.In fact there are some separate processes here: development, release management, deployment, configuration management, .... Currently we use systemimager to deploy our configuration and software packages on machines, but we have no idea about the current status of machines. Most of the development/testing happens on these machines too. Anyhow, the development work must be committed into CVS. These reports can give track about what changed on which machine, so we know what to bring back to CVS. (when bootstrapping these machines from a systemimager source, everything is lost and we surely don''t want to lose any development work) These reports can help us creating new releases of updated software and configuration files (depends on how many and the impact of these changes ; things you can learn easily from puppet) So I want to know with these reports: what (will) change on which machine, problems when deploying new configuration with puppet (packages that don''t install nicely due to dependencies), when was the last time the machines applied their configuration (to know whether development on these machines are already lost), etc..) and that is something that was described in that Reporting page on Trac. --> You only have to be open minded if you''re wrong. > --------------------------------------------------------------------- > Luke Kanies | http://reductivelabs.com | http://madstop.com > > >_______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
Koen Vereeken wrote:> 2007/1/30, Luke Kanies <luke@madstop.com <mailto:luke@madstop.com>>: > What are you hoping to do with reporting? One of the reasons I''ve > made so little progress is that I can''t seem to get anyone to express > any interest in reporting, which seems a bit weird to me. > > So I want to know with these reports: what (will) change on which > machine, problems when deploying new configuration with puppet (packages > that don''t install nicely due to dependencies), when was the last time > the machines applied their configuration (to know whether development on > these machines are already lost), etc..) and that is something that was > described in that Reporting page on Trac.Let me just second this. I don''t think a configuration management system can be complete unless it tells you which systems are up to date, which are not, and why. One of the applications I''ve been daydreaming about is a side-by-side diff-style comparison of packages installed on all my systems. It would need to know a fair bit about the particular systems: I want to know about differences in package versions, SLOTs, and (most importantly) USE flags. Maybe something similar for specific directories and their contents? I really haven''t looked into the puppet reporting stuff yet, so I''m just hoping that it could one day support something like this. Peter
> One of the applications I''ve been daydreaming about is a side-by-side > diff-style comparison of packages installed on all my systems. It would > need to know a fair bit about the particular systems: I want to know > about differences in package versions, SLOTs, and (most importantly) USE > flags. Maybe something similar for specific directories and their > contents? > > I really haven''t looked into the puppet reporting stuff yet, so I''m just > hoping that it could one day support something like this.We use Red Hat Satellite to build our Red Hat boxes off of and it gives us this functionality, but through a bloated WebApp that we hate. We''re also looking to use Puppet in this sort of information gathering fashion. -- Digant C Kasundra <digant@stanford.edu> Technical Lead, ITS Unix Systems and Applications, Stanford University
2007/1/31, Peter Abrahamsen <peter@betcha.com>:> > Koen Vereeken wrote: > > 2007/1/30, Luke Kanies <luke@madstop.com <mailto:luke@madstop.com>>: > > What are you hoping to do with reporting? One of the reasons I''ve > > made so little progress is that I can''t seem to get anyone to > express > > any interest in reporting, which seems a bit weird to me. > > > > So I want to know with these reports: what (will) change on which > > machine, problems when deploying new configuration with puppet (packages > > that don''t install nicely due to dependencies), when was the last time > > the machines applied their configuration (to know whether development on > > these machines are already lost), etc..) and that is something that was > > described in that Reporting page on Trac. > > Let me just second this. I don''t think a configuration management system > can be complete unless it tells you which systems are up to date, which > are not, and why. > > One of the applications I''ve been daydreaming about is a side-by-side > diff-style comparison of packages installed on all my systems. It would > need to know a fair bit about the particular systems: I want to know > about differences in package versions, SLOTs, and (most importantly) USE > flags. Maybe something similar for specific directories and their > contents? >I guess taking into account slots and use flags would require changing the package type to handle this. This is something I''ve been thinking about, and my solution (still not tested, I''m about to do it) is to handle global (/etc/make.conf) and package (/etc/portage/package.use) use flags using puppet managed files (downloaded from puppet server). I would then control compilation / installation of packages with EMERGE_DEFAULT_OPTS in /etc/make.conf, so I can make deep/newuse emerges. I think this is all we can do regarding Gentoo with puppet in its current state, what do you think? Any other approach/solution? Best regards Jose _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
2007/1/31, Peter Abrahamsen <peter@betcha.com>:> > Koen Vereeken wrote: > > 2007/1/30, Luke Kanies <luke@madstop.com <mailto:luke@madstop.com>>: > > What are you hoping to do with reporting? One of the reasons I''ve > > made so little progress is that I can''t seem to get anyone to > express > > any interest in reporting, which seems a bit weird to me. > > > > So I want to know with these reports: what (will) change on which > > machine, problems when deploying new configuration with puppet (packages > > that don''t install nicely due to dependencies), when was the last time > > the machines applied their configuration (to know whether development on > > these machines are already lost), etc..) and that is something that was > > described in that Reporting page on Trac. > > Let me just second this. I don''t think a configuration management system > can be complete unless it tells you which systems are up to date, which > are not, and why. > > One of the applications I''ve been daydreaming about is a side-by-side > diff-style comparison of packages installed on all my systems. It would > need to know a fair bit about the particular systems: I want to know > about differences in package versions, SLOTs, and (most importantly) USE > flags. Maybe something similar for specific directories and their > contents? >I guess taking into account slots and use flags would require changing the package type to handle this. This is something I''ve been thinking about, and my solution (still not tested, I''m about to do it) is to handle global (/etc/make.conf) and package (/etc/portage/package.use) use flags using puppet managed files (downloaded from puppet server). I would then control compilation / installation of packages with EMERGE_DEFAULT_OPTS in /etc/make.conf, so I can make deep/newuse emerges. I think this is all we can do regarding Gentoo with puppet in its current state, what do you think? Any other approach/solution? Best regards Jose _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
José González Gómez wrote:> I guess taking into account slots and use flags would require changing > the package type to handle this. This is something I''ve been thinking > about, and my solution (still not tested, I''m about to do it) is to > handle global (/etc/make.conf) and package (/etc/portage/package.use) > use flags using puppet managed files (downloaded from puppet server). I > would then control compilation / installation of packages with > EMERGE_DEFAULT_OPTS in /etc/make.conf, so I can make deep/newuse > emerges. I think this is all we can do regarding Gentoo with puppet in > its current state, what do you think? Any other approach/solution?Hmm. I don''t know what the best way to implement it is, but I''d really like hardware type classes to be able to push hardware-related make.conf options, while any class is able to push other options. Wouldn''t this be difficult to do with ERB, if that''s what you''re suggesting? It''d be totally awesome if the implementation also meant that USE flags and keywords were grouped logically into files in /etc/portage/package.keywords/ and /etc/portage/package.use/. This will also make it a lot easier for administrators to have hybrid managed (puppet) and manual configuration files on the same machine. So and I guess there''s a portage replacement being worked on that will be much faster, more intelligent, overcome some other limitations of portage, and /come with Ruby bindings/. Have you looked at it at all? Peter
On Jan 31, 2007, at 1:01 PM, Peter Abrahamsen wrote:> > Let me just second this. I don''t think a configuration management > system > can be complete unless it tells you which systems are up to date, > which > are not, and why.I agree.> One of the applications I''ve been daydreaming about is a side-by-side > diff-style comparison of packages installed on all my systems. It > would > need to know a fair bit about the particular systems: I want to know > about differences in package versions, SLOTs, and (most > importantly) USE > flags. Maybe something similar for specific directories and their > contents?Just for the record, you could write a web-based app that did most of this today. Puppet''s ''resource'' client-side handler (previously named ''pelement'') allows you to get lists of resources and then describe individual resources. Thus, you could write a web application that could connect to two servers, list all available packages, and then provide a page showing the differences. Everything is in place but the web pages; the client-side stuff is already done. PuppetShow was originally written to be used for this kind of thing, and it would be downright straightforward to start adding this functionality back into it. There''s currently no RBAC system, which is kinda scary and should be fixed before we get too far, and I''m not making any promises about how efficient the APIs are right now (e.g., you might have to make a separate call for the details on each package, which would be bad), but the basic structure is in place. Note that RBAC will always be very difficult -- you''ll want to give users access to read some files (e.g., /etc/passwd) but maybe not others (/etc/shadow). Yuck. Heck, you could even write a web application that could sync two machines, as long as the machines used package tools that know how to download their packages. You could copy just about any resource from one host to another -- it''s a query on the first host, and uploading a config on the second one. Puppet is a web application short of replacing Webmin, in other words; I just haven''t had the time to spend on it. -- Brand''s Shortcut: The only way to predict the future is to make sure it stays exactly the same as the present. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Feb 1, 2007, at 3:58 AM, José González Gómez wrote:> > I guess taking into account slots and use flags would require > changing the package type to handle this. This is something I''ve > been thinking about, and my solution (still not tested, I''m about > to do it) is to handle global (/etc/make.conf) and package (/etc/ > portage/package.use) use flags using puppet managed files > (downloaded from puppet server). I would then control compilation / > installation of packages with EMERGE_DEFAULT_OPTS in /etc/ > make.conf, so I can make deep/newuse emerges. I think this is all > we can do regarding Gentoo with puppet in its current state, what > do you think? Any other approach/solution?Yeah, sorry, I forgot; of course, if Puppet resources don''t know about it, then you can''t use Puppet to view or compare it. We should probably add the ability to have states that can only retrieve information (like USE flags) but can''t change them. Or would you consider that a modifiable aspect of a package? -- Millions long for immortality who do not know what to do with themselves on a rainy Sunday afternoon. -- Susan Ertz --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
2007/2/2, Luke Kanies <luke@madstop.com>:> > On Feb 1, 2007, at 3:58 AM, José González Gómez wrote: > > > > > I guess taking into account slots and use flags would require > > changing the package type to handle this. This is something I''ve > > been thinking about, and my solution (still not tested, I''m about > > to do it) is to handle global (/etc/make.conf) and package (/etc/ > > portage/package.use) use flags using puppet managed files > > (downloaded from puppet server). I would then control compilation / > > installation of packages with EMERGE_DEFAULT_OPTS in /etc/ > > make.conf, so I can make deep/newuse emerges. I think this is all > > we can do regarding Gentoo with puppet in its current state, what > > do you think? Any other approach/solution? > > Yeah, sorry, I forgot; of course, if Puppet resources don''t know > about it, then you can''t use Puppet to view or compare it. > > We should probably add the ability to have states that can only > retrieve information (like USE flags) but can''t change them. Or > would you consider that a modifiable aspect of a package? > >Portage may have global and per package use flags, and they change the way a package is installed. For example, you may install heimdal kerberos with or without support for OpenLDAP just activating the ldap use flag for that package (in /etc/portage/package.use). If you want to have ldap functionality in the whole system, not just in heimdal, you may activate the ldap use flag globally (in /etc/make.conf). So yes, I think regarding puppet this should be a modifiable aspect of a package, or even a modifiable aspect of the package type (for global use flags), but I haven''t suggested adding this to puppet, as I thought this was a portage only functionality (don''t know any other package manager with anything similar), it could be solved with a workaround (handling /etc/make.conf and /etc/portage/package.use with puppet) and I hadn''t enough knowledge about puppet internals to implement it myself and post the patch. Related to this you have the concept of slots, IIRC we talked about that in a prior thread. Do you think it would be a good idea to add this kind of support to the package type / provider taking into account that is something only applicable to portage? I would be willing to implement it with a bit of assistance. Best regards Jose _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
2007/2/1, Peter Abrahamsen <peter@betcha.com>:> > José González Gómez wrote: > > I guess taking into account slots and use flags would require changing > > the package type to handle this. This is something I''ve been thinking > > about, and my solution (still not tested, I''m about to do it) is to > > handle global (/etc/make.conf) and package (/etc/portage/package.use) > > use flags using puppet managed files (downloaded from puppet server). I > > would then control compilation / installation of packages with > > EMERGE_DEFAULT_OPTS in /etc/make.conf, so I can make deep/newuse > > emerges. I think this is all we can do regarding Gentoo with puppet in > > its current state, what do you think? Any other approach/solution? > > Hmm. I don''t know what the best way to implement it is, but I''d really > like hardware type classes to be able to push hardware-related make.conf > options, while any class is able to push other options. Wouldn''t this be > difficult to do with ERB, if that''s what you''re suggesting?Yes. In the simplest approach you could have an ERB template and the define some variables in selected classes with some sensible defaults so your make.conf and /etc/portage files get populated with those values. Some of those variables could even take its values based on hardware related facts (architecture, hardwaremodel, processorcount,...) It''d be totally awesome if the implementation also meant that USE flags> and keywords were grouped logically into files in > /etc/portage/package.keywords/ and /etc/portage/package.use/. This will > also make it a lot easier for administrators to have hybrid managed > (puppet) and manual configuration files on the same machine.I understand you''re talking about having a .d kind of directory with a file per package? Is this correct? Or do you mean managing entries in the package.keywords and package.use files? So and I guess there''s a portage replacement being worked on that will> be much faster, more intelligent, overcome some other limitations of > portage, and /come with Ruby bindings/. Have you looked at it at all? >I guess you''re talking about paludis [1]. Unfortunately there doesn''t seem to be a clear successor to portage, as there is another package manager under development called pkgcore [2], and from what I''ve read in forums / mailing lists, the Gentoo developers doesn''t seem to agree on which is the best. I guess we''ll have to wait until the dust settles, and then write a new provider... well, you could write a provider right now for pkgcore and paludis, but I wouldn''t do it until you know if they''re widely used. Best regards Jose [1] http://www.paludis.org/ [2] http://www.pkgcore.org/ _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
José González Gómez wrote:> Yes. In the simplest approach you could have an ERB template and the > define some variables in selected classes with some sensible defaults so > your make.conf and /etc/portage files get populated with those values. > Some of those variables could even take its values based on hardware > related facts (architecture, hardwaremodel, processorcount,...)Huh, OK. Yeah, I guess with some good usage patterns/examples, ERB should be easy enough to use for this.> I understand you''re talking about having a .d kind of directory with a > file per package? Is this correct? Or do you mean managing entries in > the package.keywords and package.use files?I have /etc/portage organized like: /etc/portage/package.keywords: apache mysql pdsh php tripwire /etc/portage/package.use: apache exim mysql php ruby svn which allows me to logically group package USE flags and keywords by how I think about them. The package.use/php file, for example, mentions not just PHP itself, but also packages it depends on, and packages related to having PHP on a system.> I guess you''re talking about paludis [1]. Unfortunately there doesn''t > seem to be a clear successor to portage, as there is another package > manager under development called pkgcore [2], and from what I''ve read in > forums / mailing lists, the Gentoo developers doesn''t seem to agree on > which is the best. I guess we''ll have to wait until the dust settles, > and then write a new provider... well, you could write a provider right > now for pkgcore and paludis, but I wouldn''t do it until you know if > they''re widely used.Yes, I was referring to paludis; I wasn''t aware of pkgcore. I hope that the developers are able to combine the best aspects of each into a clear portage successor relatively soon. Humm. I guess we''ll be supporting portage for a while to come in any case. Peter
2007/2/2, Peter Abrahamsen <peter@betcha.com>:> > José González Gómez wrote: > > Yes. In the simplest approach you could have an ERB template and the > > define some variables in selected classes with some sensible defaults so > > your make.conf and /etc/portage files get populated with those values. > > Some of those variables could even take its values based on hardware > > related facts (architecture, hardwaremodel, processorcount,...) > > Huh, OK. Yeah, I guess with some good usage patterns/examples, ERB > should be easy enough to use for this. > > > I understand you''re talking about having a .d kind of directory with a > > file per package? Is this correct? Or do you mean managing entries in > > the package.keywords and package.use files? > > I have /etc/portage organized like: > > /etc/portage/package.keywords: > apache mysql pdsh php tripwire > > /etc/portage/package.use: > apache exim mysql php ruby svn > > which allows me to logically group package USE flags and keywords by how > I think about them. The package.use/php file, for example, mentions not > just PHP itself, but also packages it depends on, and packages related > to having PHP on a system.Umm, I didn''t know you could have use files arranged like this... you learn something new everyday :o) Regarding all this stuff, if the per-provider parameters proposal [1] goes ahead all of this could be handled transparently by the portage provider... it would be great. [1] https://reductivelabs.com/trac/puppet/wiki/LanguageEvolution#id6 Best regards Jose _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
> > Let me just second this. I don''t think a configuration management > > system can be complete unless it tells you which systems are up > > to date, which are not, and why. > > I agree.(Late to the party, but anyways). I think this is the crux of it for us. (up till last week) I though this was something that bcfg2 had going for it well ahead of puppet - that was until I saw the puppet reporting hooks. I''ll admit that it''s still got a lot to do, but it is starting with a reasonable basis that we can use for now. Our approach for deployment at this point is to have puppet run in noop and report mode in a daily cron job, and then a report processor run a couple hours after the daily cron jobs. This gives us a sense of how consistent the environment is to our manifests, and from that we can decide how to fix the inconsistencies(either through puppet or manually if need be based on the policy for that box) I worry that a model like this won''t scale, but right now it''s the model that I think will give us better confidence in rolling out the environment. If anyone''s interested, the report from our test environment looks like(script attached - and yes, it''s dirty). ------------------------------------------------------------------ ---------------- HOSTS BY CHECKIN ------------------ Fri Feb 02 04:02:04 -0800 2007 host1 Fri Feb 02 04:02:05 -0800 2007 host2 Fri Feb 02 04:02:05 -0800 2007 host3 Fri Feb 02 04:02:13 -0800 2007 host4 Fri Feb 02 04:02:32 -0800 2007 host5 Fri Feb 02 15:14:44 -0800 2007 host6 Fri Feb 02 15:33:35 -0800 2007 host7 Fri Feb 02 15:50:06 -0800 2007 host8 ----------- HOSTS BY INCONSISTENCIES --------------- 22/24 host2 22/24 host8 22/24 host1 23/24 host5 24/25 host4 26/27 host3 39/40 host6 24/24 host7 ----------------- LOGS BY TOTALS ------------------- 5 FILE[/etc/ntp.conf]/source host2 host1 host5 host3 host8 4 FILE[/etc/resolv.conf]/source host2 host1 host4 host8 1 FILE[/etc/ntp.conf]/mode host2 1 FILE[/etc/ntp.conf]/checksum host5 1 FILE[/etc/mail/submit.cf]/content host6 ------------------------------------------------------------------ This avoids us getting email reports and does some consolidation of the highest priority(well, scale) of issues. This is the only report we''re looking at each morning. We used to have the philosophy of using cfengine to just blast out changes. Since it wasn''t as friendly with collecting messages(and I just may never have seen a good way), we didn''t have the chance to audit, then fix. And that''s come back to bite us. So, yeah, it''s all about good reporting to us and we''ve got a vested interest in seeing that happen. The icing is that the same inputs can be used to report or fix the problems. Of course, the caveat is that we''re just starting to our deployment so it''s kinda hard to say the exact features of what we''re lookging for in reporting. It''s small now so scaling this is going to be interesting. Since we all know that log churn is an issue, I think it''s about keeping the critical items at the top of the report. -- Just my thoughts off the top of my head. --mac _______________________________________________ Puppet-users mailing list Puppet-users@madstop.com https://mail.madstop.com/mailman/listinfo/puppet-users
On Feb 2, 2007, at 6:34 PM, Chris McEniry wrote:> > (Late to the party, but anyways). I think this is the crux of it for > us. (up till last week) I though this was something that bcfg2 had > going for it well ahead of puppet - that was until I saw the puppet > reporting hooks. I''ll admit that it''s still got a lot to do, but it > is starting with a reasonable basis that we can use for now.That''s good to hear. Since you''re one of the few people who appears to be doing much outside the bounds with reporting right now: Is there anything specific you are looking for in addition to what''s there? Any comments on the document I wrote?> Our approach for deployment at this point is to have puppet run in > noop and report mode in a daily cron job, and then a report > processor run a couple hours after the daily cron jobs. This gives > us a sense of how consistent the environment is to our manifests, > and from that we can decide how to fix the inconsistencies(either > through puppet or manually if need be based on the policy for > that box) I worry that a model like this won''t scale, but right > now it''s the model that I think will give us better confidence in > rolling out the environment.If you''d be interested in writing a short description on the wiki of this process you use, I expect quite a few people would be interested in hearing about it.> If anyone''s interested, the report from our test environment > looks like(script attached - and yes, it''s dirty).I''m not sure it could really help you in your current process, nor if it''s worth it since we''re planning on such significant enhancements to reporting in the near future, but it''s worth noting that you can write a Puppet report module and drop it into lib/puppet/reports, tell puppetmasterd to start using it, and it will. Your process.rb script shows that you understand the Report format, but I wasn''t sure if you know you could tell puppetmasterd to do real-time processing with your own report. [snip]> This avoids us getting email reports and does some consolidation > of the highest priority(well, scale) of issues. This is the only > report we''re looking at each morning. > > We used to have the philosophy of using cfengine to just blast out > changes. Since it wasn''t as friendly with collecting messages(and > I just may never have seen a good way), we didn''t have the chance > to audit, then fix. And that''s come back to bite us.This is a process I''d really like to enable, so if there''s anything I can change in Puppet to make this easier, I''d really like to hear about it.> So, yeah, it''s all about good reporting to us and we''ve got a > vested interest in seeing that happen. The icing is that the same > inputs can be used to report or fix the problems.That''s always been a key design goal for me.> Of course, the caveat is that we''re just starting to our deployment > so it''s kinda hard to say the exact features of what we''re lookging > for in reporting. It''s small now so scaling this is going to be > interesting. Since we all know that log churn is an issue, I think > it''s about keeping the critical items at the top of the report.That''s an interesting point -- how to make sure failures don''t scroll past, even as more events show up. Hopefully the resource-based reporting should make this more clear, since a given resource on a given machine will normally either be correct or incorrect, and its last attempt at syncing will either have failed or succeeded. Once the data is in place (current state, whether the last sync attempt failed or succeeded) it should be pretty easy to write a report that shows all resources whose last modification failed. It might be worth keeping track of which (using Ben''s newly- recommended terms) property failed, so you wouldn''t want a file to show no failures if the source failed and then the mode succeeded. -- This space intentionally has nothing but text explaining why this space has nothing but text explaining that this space would otherwise have been left blank, and would otherwise have been left blank. --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
> Is there anything > specific you are looking for in addition to what''s there?I think a lot of it is intelligence built on top of the basic report mechanisms. What''s there seems to be a very solid basis, but it doesn''t do much consolidation of failed properties over the entire environment - at least, not in what we''ve seen. I think one vision would be to have a management console which you can drill down into on a resource, node, or class basis and it can show you the failed properties. There''s pro''s and con''s with this, but it''s one direction to shot for.> Any > comments on the document I wrote?Well, it tells you how to enable, but doesn''t show you how useful it can be. Examples of output was the biggest thing I''d say it''s missing. I''m still going through the buitlin reports - so I''ll have to see that part. The other piece that missed was the dependency on outside models. Turning on rrdgraphs on puppetmasterd without having the rrd library around silently fails. Makes sense, but it should at least throw an error for missing dependencies.> I''m not sure it could really help you in your current process, nor if > it''s worth it since we''re planning on such significant enhancements > to reporting in the near future, but it''s worth noting that you can > write a Puppet report module and drop it into lib/puppet/reports, > tell puppetmasterd to start using it, and it will. Your process.rb > script shows that you understand the Report format, but I wasn''t sure > if you know you could tell puppetmasterd to do real-time processing > with your own report.Sadly - I overlooked that part. I have to go back, take a swing at it, and get back to you on that one. Off the cuff - not sure where the changes are going, but I would say to definitely leave in the ability to drop in custom reports.> > We used to have the philosophy of using cfengine to just blast out > > changes. Since it wasn''t as friendly with collecting messages(and > > I just may never have seen a good way), we didn''t have the chance > > to audit, then fix. And that''s come back to bite us. > > This is a process I''d really like to enable, so if there''s anything I > can change in Puppet to make this easier, I''d really like to hear > about it.Part of what we''re tripping over is being able to select between audit and change modes. I think the best way to do this right now is to have something in the manifests, but that takes some doing to make all of the host groupings correct. Ideally, I want to say "push changes to $thesehosts at $time" - which has us manage the host groupings outside of puppet(debatable if that''s good or bad) and control the timing outside of puppet(again, debatable). Having hooks for that kind of control would give us an option(not sure how much it''s worth it though). We''ve looked at doing it via puppetrun and --listen, but we still have to work around groupings and timings outside of puppet, and general practices keeps us from running another listening daemon. We use sshkeys for every control piece, so it''d be nice if we could choose the transport for puppetrun between say "native" and "ssh" or so.> That''s an interesting point -- how to make sure failures don''t scroll > past, even as more events show up. > > Hopefully the resource-based reporting should make this more clear, > since a given resource on a given machine will normally either be > correct or incorrect, and its last attempt at syncing will either > have failed or succeeded. Once the data is in place (current state, > whether the last sync attempt failed or succeeded) it should be > pretty easy to write a report that shows all resources whose last > modification failed.I may have mis-represented this with the log analogy. If we''re to watch the consolidated logs of every node''s run output, we''re going to get lost in the details. With some allowances for individual exceptions - I don''t care as much if ntp.conf is out of sync on one or two nodes, when resolv.conf is out of sync on every node. I want to address resolv.conf first. You can gleam some of this from watching reports, but that''s not very quantitative. Taking it to a class level, I may care that one class(say "prod") is out of sync at all, versus if another class(say "desktop") is out of sync with a few. There''s obviously can be a lot of tweaking here, but if it''s important, people will definitely put in the right thresholds and organizations that make sense for them. --mac
> Is there anything > specific you are looking for in addition to what''s there?One other thing that I think might be useful - expanding (or writing something new for) puppetdoc so that it can use commenting in the manifests. Not really sure how this should be structured normally; but if I were using a management console to drill down into a node and see its failed properties, it should be able to pull comments from the manifests out and present them there. Just a thought. --mac
On Feb 5, 2007, at 12:17 PM, Chris McEniry wrote:>> Is there anything >> specific you are looking for in addition to what''s there? > > I think a lot of it is intelligence built on top of the basic report > mechanisms. What''s there seems to be a very solid basis, but it > doesn''t do much consolidation of failed properties over the entire > environment - at least, not in what we''ve seen. > > I think one vision would be to have a management console which you > can drill down into on a resource, node, or class basis and it > can show you the failed properties. There''s pro''s and con''s with > this, but it''s one direction to shot for.That''s my primary goal with the next stage of reporting work. It sounds like you''ll probably want the ability to define your own web-based reports, too, so I''ll add that to the document.>> Any >> comments on the document I wrote? > > Well, it tells you how to enable, but doesn''t show you how useful it > can be. Examples of output was the biggest thing I''d say it''s > missing. I''m still going through the buitlin reports - so I''ll have > to see that part. The other piece that missed was the dependency on > outside models. Turning on rrdgraphs on puppetmasterd without > having the rrd library around silently fails. Makes sense, but it > should at least throw an error for missing dependencies.We don''t really have a plan for the interface yet, which is why no output was described. You could look at the database model (in lib/puppet/rails/database/ schema.rb) to get an idea of the data that will be available; Rails makes it really easy to display each of these record types with their relationships (e.g., it''ll be easy to list hosts, then list the resources associated with that host), but anything else will be a bit tougher. Note also that we don''t currently explicitly model classes or tags, so it''ll be a bit more difficult (but still not "hard") to show, for instance, all resources in a given class. Clearly that''s an important feature, though, especially the ability to show all failed resources in a given class, or all classes with failures, etc.> Sadly - I overlooked that part. I have to go back, take a swing at > it, > and get back to you on that one. > > Off the cuff - not sure where the changes are going, but I would say > to definitely leave in the ability to drop in custom reports.Definitely.> Part of what we''re tripping over is being able to select between audit > and change modes. I think the best way to do this right now is to > have > something in the manifests, but that takes some doing to make all of > the host groupings correct. Ideally, I want to say "push changes to > $thesehosts at $time" - which has us manage the host groupings outside > of puppet(debatable if that''s good or bad) and control the timing > outside of puppet(again, debatable). Having hooks for that kind of > control would give us an option(not sure how much it''s worth it > though). > > We''ve looked at doing it via puppetrun and --listen, but we still have > to work around groupings and timings outside of puppet, and general > practices keeps us from running another listening daemon. We use > sshkeys for every control piece, so it''d be nice if we could choose > the transport for puppetrun between say "native" and "ssh" or so.I actually think the right solution here is to move at least the grouping information out of puppet. If you put your grouping information into ldap, for instance, then puppetrun and puppet will both have access to the grouping information. The more I think about it, the more it seems like the long-term direction is to cleanly separate the grouping from the manifests, which would make at least part of this task easier. It''s clearly bad to have the groupings outside of Puppet if it cannot get access to that grouping information, but as long as it can, everything should work well. I almost think there should be a small tool that just handles host classing; then Puppet, bcfg2, or whatever could use this tool to handle host classing independently. I expect that if such a tool existed, lots of other tools would find it useful.> I may have mis-represented this with the log analogy. If we''re to > watch the consolidated logs of every node''s run output, we''re going > to get lost in the details. > > With some allowances for individual exceptions - I don''t care as much > if ntp.conf is out of sync on one or two nodes, when resolv.conf is > out of sync on every node. I want to address resolv.conf first. You > can gleam some of this from watching reports, but that''s not very > quantitative. > > Taking it to a class level, I may care that one class(say "prod") is > out of sync at all, versus if another class(say "desktop") is out of > sync with a few. There''s obviously can be a lot of tweaking here, > but if it''s important, people will definitely put in the right > thresholds and organizations that make sense for them.This is where flexible queries become important; we need to be able to add tags to restrict the scope of queries, so that you''re only looking for errors on production hosts, for instance. -- The first time I see a jogger smiling, I''ll consider it. --Joan Rivers --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com
On Feb 5, 2007, at 12:25 PM, Chris McEniry wrote:> One other thing that I think might be useful - expanding > (or writing something new for) puppetdoc so that it can > use commenting in the manifests. Not really sure how this > should be structured normally; but if I were using a > management console to drill down into a node and see its > failed properties, it should be able to pull comments from > the manifests out and present them there. Just a thought.You mean adding in-line documentation? I''ve thought about it a bit, but I haven''t really looked at it much. It would be pretty easy to add per-resource log messages as parameters: user { luke: log => "Never give this guy sudo access", ... } That''s clearly not sufficient beyond simple resources, though. I''m definitely open to doing this, with one caveat: I refuse to use a custom markup format. We''d have to use something like markdown or restructured text (I prefer the latter, since it''s a better markup language, but there isn''t yet a parser for it). -- I take my children everywhere, but they always find their way back home. --Robert Orben --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com