Hi everyone, I am using puppet 2.7.9 and ruby 1.8.7 on debian box. I don''t a lot of modules right now: just one module that create user + dotfile + ssh key and compiled catalog takes around 70s do I have to worry is that big. Does the compiled time scale with the module number? Do I have to make sure I don''t cross a certain limit? Regards, JM -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
70s sounds does sound a bit high for managing only three resources, but more info is required before I can really answer your question. How did you install Puppet? (OS packages, gem?) What OS are you running, and can you provide some info about the Hardware? Also, if you have a sample of the manifest that you are using that would be great as well. On Jan 27, 5:02 am, Antidot SAS <antidot...@gmail.com> wrote:> Hi everyone, > > I am using puppet 2.7.9 and ruby 1.8.7 on debian box. I don''t a lot of > modules right now: just one module that create user + dotfile + ssh key and > compiled catalog takes around 70s do I have to worry is that big. > > Does the compiled time scale with the module number? Do I have to make sure > I don''t cross a certain limit? > > Regards, > JM-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Jan 27, 4:02 am, Antidot SAS <antidot...@gmail.com> wrote:> Hi everyone, > > I am using puppet 2.7.9 and ruby 1.8.7 on debian box. I don''t a lot of > modules right now: just one module that create user + dotfile + ssh key and > compiled catalog takes around 70s do I have to worry is that big.That''s pretty excessive. My manifests are considerably more complex than you describe, and my compilation times average around 1s. Exact timing also depends on the hardware, of course, but something is wrong for you if it takes 70s to compile a simple catalog. We could be more helpful if you gave us a manifest set that exhibits the problem.> Does the compiled time scale with the module number? Do I have to make sure > I don''t cross a certain limit?Compilation time scales with the number and complexity of the manifests to compile. That is correlated to the number of modules used (not the number installed) inasmuch as using more modules implies compiling more manifests. The actual number of manifests compiled depends on the details of the modules involved and their manner of use, however. As for limits, the more work your master needs to do to compile each node''s catalog, the fewer nodes it can serve. In the extreme that compilation time approached or exceeded the delay between agent runs, effectively the master could serve only a single node. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hey thx for the answer... the puppetmaster is an old Poweredge SC1425 with 2 Go of RAM and 4 CPU (Intel Xeon 2.8Ghz) Regarding the catalog it is playing a lot with hash tables... Trying to figure out a way to show it to you... Attachment could be a way? On Fri, Jan 27, 2012 at 4:55 PM, jcbollinger <John.Bollinger@stjude.org>wrote:> > > On Jan 27, 4:02 am, Antidot SAS <antidot...@gmail.com> wrote: > > Hi everyone, > > > > I am using puppet 2.7.9 and ruby 1.8.7 on debian box. I don''t a lot of > > modules right now: just one module that create user + dotfile + ssh key > and > > compiled catalog takes around 70s do I have to worry is that big. > > > That''s pretty excessive. My manifests are considerably more complex > than you describe, and my compilation times average around 1s. Exact > timing also depends on the hardware, of course, but something is wrong > for you if it takes 70s to compile a simple catalog. We could be more > helpful if you gave us a manifest set that exhibits the problem. > > > > Does the compiled time scale with the module number? Do I have to make > sure > > I don''t cross a certain limit? > > > Compilation time scales with the number and complexity of the > manifests to compile. That is correlated to the number of modules > used (not the number installed) inasmuch as using more modules implies > compiling more manifests. The actual number of manifests compiled > depends on the details of the modules involved and their manner of > use, however. > > As for limits, the more work your master needs to do to compile each > node''s catalog, the fewer nodes it can serve. In the extreme that > compilation time approached or exceeded the delay between agent runs, > effectively the master could serve only a single node. > > > John > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Fri, 2012-01-27 at 11:02 +0100, Antidot SAS wrote:> Hi everyone, > > > > > I am using puppet 2.7.9 and ruby 1.8.7 on debian box. I don''t a lot of > modules right now: just one module that create user + dotfile + ssh > key and compiled catalog takes around 70s do I have to worry is that > big.70s is very high. My biggest node has around 2000 resources, and it takes around 12s to compile (including storeconfigs) on a 4 year old Dell 2950 (one 4C 1.18GHz processor, 4GiB RAM). Compilation time depends on two things: * the complexity (in number of classes or resources) of your manifests * the number of concurrent nodes asking for catalog (the more node asking for catalogs, the more load on your server and processes start to fight for CPU/RAM). Also, if you have storeconfigs enabled, this can take quite some time. In which case I suggest to run with thin_storeconfigs instead of the full one if you can. You might also want to check that your server is not swapping.> Does the compiled time scale with the module number? Do I have to make > sure I don''t cross a certain limit?-- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Here are the info you wanted: OS: Debian squeeze Puppet: installed from gem version 2.7.9 As I sayed the hardware is an old Poweredge SC1425 with 4 Xeon 2,8 Ghz with 2 Go of Memory Apparently the code is preety ugly since the compiled time is big, I have just share the code: https://github.com/jmauro/puppet-user The compilation time increased when I want to create about 50 employe for instance in the file admin/params.pp Let me know if you see something odd or If I misunderstand a concept. Regards, JM On Fri, Jan 27, 2012 at 1:26 PM, Kelsey Hightower <kelsey@puppetlabs.com>wrote:> 70s sounds does sound a bit high for managing only three resources, > but more info is required before I can really answer your question. > > How did you install Puppet? (OS packages, gem?) > What OS are you running, and can you provide some info about the > Hardware? > > Also, if you have a sample of the manifest that you are using that > would be great as well. > > On Jan 27, 5:02 am, Antidot SAS <antidot...@gmail.com> wrote: > > Hi everyone, > > > > I am using puppet 2.7.9 and ruby 1.8.7 on debian box. I don''t a lot of > > modules right now: just one module that create user + dotfile + ssh key > and > > compiled catalog takes around 70s do I have to worry is that big. > > > > Does the compiled time scale with the module number? Do I have to make > sure > > I don''t cross a certain limit? > > > > Regards, > > JM > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Jan 27, 11:10 am, Antidot SAS <antidot...@gmail.com> wrote:> Here are the info you wanted: > OS: Debian squeeze > Puppet: installed from gem version 2.7.9 > As I sayed the hardware is an old Poweredge SC1425 with 4 Xeon 2,8 Ghz with > 2 Go of Memory > > Apparently the code is preety ugly since the compiled time is big, I have > just share the code:https://github.com/jmauro/puppet-userThe code is pretty, but very complicated, especially for the relatively simple job it seems to be trying to do. Clearly that code is not your first cut at this, so did older versions exhibit the same poor compile times? If you go back through your revision history, perhaps you can find the change that made compile times spike. Also, Brice''s suggestion to check for swapping is a good one, especially because Puppet sometimes uses a lot of memory. Although your hardware is a bit old, it should be up to this task so long as it hasn''t too many competing demands on it. Since your catalog eventually does compile, it might be useful to look at the result, available in YAML form. You might discover that it is a lot more complicated than you thought, and that would yield clues to what needs to be cut back. If all else fails, try disabling as much of the code as possible, then watch compilation times as you re-enable it bit by bit. I suspect that you will find some particular part that consumes the lion''s share of the compilation time. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Thx for answering, I really appreciated your investment, you gave some serious lead to search, I''ll see on Monday and let you know if something comes up. As far as swapping, no sign on the server. On Fri, Jan 27, 2012 at 11:40 PM, jcbollinger <John.Bollinger@stjude.org>wrote:> > > On Jan 27, 11:10 am, Antidot SAS <antidot...@gmail.com> wrote: > > Here are the info you wanted: > > OS: Debian squeeze > > Puppet: installed from gem version 2.7.9 > > As I sayed the hardware is an old Poweredge SC1425 with 4 Xeon 2,8 Ghz > with > > 2 Go of Memory > > > > Apparently the code is preety ugly since the compiled time is big, I have > > just share the code:https://github.com/jmauro/puppet-user > > > The code is pretty, but very complicated, especially for the > relatively simple job it seems to be trying to do. Clearly that code > is not your first cut at this, so did older versions exhibit the same > poor compile times? If you go back through your revision history, > perhaps you can find the change that made compile times spike. > > Also, Brice''s suggestion to check for swapping is a good one, > especially because Puppet sometimes uses a lot of memory. Although > your hardware is a bit old, it should be up to this task so long as it > hasn''t too many competing demands on it. > > Since your catalog eventually does compile, it might be useful to look > at the result, available in YAML form. You might discover that it is > a lot more complicated than you thought, and that would yield clues to > what needs to be cut back. > > If all else fails, try disabling as much of the code as possible, then > watch compilation times as you re-enable it bit by bit. I suspect > that you will find some particular part that consumes the lion''s share > of the compilation time. > > > John > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
One more thing to keep in mind, the more templates you are compiling also adds to time. On Sat, Jan 28, 2012 at 11:11 AM, Antidot SAS <antidotsas@gmail.com> wrote:> Thx for answering, I really appreciated your investment, you gave some > serious lead to search, I''ll see on Monday and let you know if something > comes up. As far as swapping, no sign on the server. > > > > > On Fri, Jan 27, 2012 at 11:40 PM, jcbollinger <John.Bollinger@stjude.org> > wrote: >> >> >> >> On Jan 27, 11:10 am, Antidot SAS <antidot...@gmail.com> wrote: >> > Here are the info you wanted: >> > OS: Debian squeeze >> > Puppet: installed from gem version 2.7.9 >> > As I sayed the hardware is an old Poweredge SC1425 with 4 Xeon 2,8 Ghz >> > with >> > 2 Go of Memory >> > >> > Apparently the code is preety ugly since the compiled time is big, I >> > have >> > just share the code:https://github.com/jmauro/puppet-user >> >> >> The code is pretty, but very complicated, especially for the >> relatively simple job it seems to be trying to do. Clearly that code >> is not your first cut at this, so did older versions exhibit the same >> poor compile times? If you go back through your revision history, >> perhaps you can find the change that made compile times spike. >> >> Also, Brice''s suggestion to check for swapping is a good one, >> especially because Puppet sometimes uses a lot of memory. Although >> your hardware is a bit old, it should be up to this task so long as it >> hasn''t too many competing demands on it. >> >> Since your catalog eventually does compile, it might be useful to look >> at the result, available in YAML form. You might discover that it is >> a lot more complicated than you thought, and that would yield clues to >> what needs to be cut back. >> >> If all else fails, try disabling as much of the code as possible, then >> watch compilation times as you re-enable it bit by bit. I suspect >> that you will find some particular part that consumes the lion''s share >> of the compilation time. >> >> >> John >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.