Out of curiosity, I pulled 1188 records out of /var/log/messages, and
extracted the number of seconds required to compile the manifest. Ran
the output through perl and Excel to get a graph of count vs time to
see what the puppetmaster box performance looks like. This excludes
data out to the 2200 second mark, where the virtual machine got its
knickers in a twist.
Puppetmaster runs in a VMware Server VM, 384 MB RAM, single Xeon 3.4
GHz CPU visible to the VM. Shared with 2 other VMs and the backup
service, so some of the long-time compiles are going to be when the
backups are running.
http://www.cricalix.net/~dhill/puppet/puppet-compile-graph-20080619.png
In general, on the setup I''m running, one third of the compiles fall
under 20 seconds and half under 40 seconds. That''s fairly acceptable,
but hopefully we''re getting new hardware designed to run VMs (ie, has
the right flags on the chips), and the compiles will get a bit faster.
(107 compiles in the 120 - 2189 second range).
The perl, if anyone cares, is brute, brute force:
grep Compiled /var/log/messages | perl -lne ''$_ =~ m/tld in (\d+)\./;
$counter{$1}++; END { foreach $p ( sort keys %counter) { print
"$p,$counter{$p}"; }}''
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Duncan Hill wrote:> In general, on the setup I''m running, one third of the compiles fall > under 20 seconds and half under 40 seconds. That''s fairly acceptable, > but hopefully we''re getting new hardware designed to run VMs (ie, has > the right flags on the chips), and the compiles will get a bit faster. > (107 compiles in the 120 - 2189 second range). >Is this 0.24.4? Regards James Turnbull - -- Author of: * Pulling Strings with Puppet (http://www.amazon.com/gp/product/1590599780/) * Pro Nagios 2.0 (http://www.amazon.com/gp/product/1590596099/) * Hardening Linux (http://www.amazon.com/gp/product/1590594444/) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIWmUZ9hTGvAxC30ARApyBAJ43jfvFtHEY86PrWrxnFPTYi2j1JwCgyM6u 367jUP6UdJKwh8lWhBg4Kt0=Ao3z -----END PGP SIGNATURE----- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
2008/6/19 James Turnbull <james@lovedthanlost.net>:> > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Duncan Hill wrote: >> In general, on the setup I''m running, one third of the compiles fall >> under 20 seconds and half under 40 seconds. That''s fairly acceptable, >> but hopefully we''re getting new hardware designed to run VMs (ie, has >> the right flags on the chips), and the compiles will get a bit faster. >> (107 compiles in the 120 - 2189 second range). >> > > Is this 0.24.4?Trust me to forget the really relevant details huh? CentOS 5 32-bit, and yes, Puppet 0.24.4. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Duncan Hill wrote:> 2008/6/19 James Turnbull <james@lovedthanlost.net>: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Duncan Hill wrote: >>> In general, on the setup I''m running, one third of the compiles fall >>> under 20 seconds and half under 40 seconds. That''s fairly acceptable, >>> but hopefully we''re getting new hardware designed to run VMs (ie, has >>> the right flags on the chips), and the compiles will get a bit faster. >>> (107 compiles in the 120 - 2189 second range). >>> >> Is this 0.24.4? > > Trust me to forget the really relevant details huh? CentOS 5 32-bit, > and yes, Puppet 0.24.4.There was some talk a while ago about adding additional serialisation methods once RESTfulness is achieved. I believe Nigel from Google did some work by hacking Marshall (and possibly JSON?) in to replace the YAML serialiser, which produced significant compilation time reductions, like 40s down to <5s, IIRC. Nigel? /me prods In short, wait; good things will come once we achieve a state of REST. I watch too much kung fu ;D Regards, AJ --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
2008/6/19 AJ <aj@junglist.gen.nz>:> > Duncan Hill wrote: > There was some talk a while ago about adding additional serialisation methods > once RESTfulness is achieved. > > I believe Nigel from Google did some work by hacking Marshall (and possibly > JSON?) in to replace the YAML serialiser, which produced significant compilation > time reductions, like 40s down to <5s, IIRC. Nigel? /me prods > > In short, wait; good things will come once we achieve a state of REST.Oh, I''m not complaining :) It''s just a data point for anyone who cares :> --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
On Jun 19, 2008, at 10:28 AM, Duncan Hill wrote:> > 2008/6/19 AJ <aj@junglist.gen.nz>: >> >> Duncan Hill wrote: >> There was some talk a while ago about adding additional >> serialisation methods >> once RESTfulness is achieved. >> >> I believe Nigel from Google did some work by hacking Marshall (and >> possibly >> JSON?) in to replace the YAML serialiser, which produced >> significant compilation >> time reductions, like 40s down to <5s, IIRC. Nigel? /me prods >> >> In short, wait; good things will come once we achieve a state of >> REST. > > Oh, I''m not complaining :) It''s just a data point for anyone who > cares :>I''m complaining. :/ In only a couple of days of optimizing, I was able to reduce the time the lexer took by about 70%. I expect there are similar gains to be had in the parser, and it looks like (for complicated configurations, anyway) yaml is a big part of the problem. I think it was someone else at Google who pointed out that yaml was taking 70% of the compile time, which is pretty heinous. As mentioned, we will be supporting other formats in 0.25, which should make things much better. Heck, I expect even switching to a graph, rather than a recursive tree like it is now, will be much better -- apparently yaml is particularly bad at nested structures. Thanks for the data. -- Puritanism: The haunting fear that someone, somewhere, may be happy. -- H. L. Mencken --------------------------------------------------------------------- Luke Kanies | http://reductivelabs.com | http://madstop.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users-unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Possibly Parallel Threads
- How long for changes to be pushed out?
- Module interaction and dependencies in Puppet (SNMP in particular)
- spam on this list..
- poor mans puppet, can puppet work with a offline puppetmaster?
- Could not request certificate: Certificate does not match private key