Jesús Oliván
2013-Jan-22 10:18 UTC
[Puppet Users] Performance really bad when executing agents
Hi, i''ve got a performance problem when running my puppet agents. I need to apply tons of files included in my website module and i assume that executing time would be long, but i can''t understand why every time when a File is evaluated, it takes about 2-3 secs to do it. In my case i''ve got tonf of files in this module, and time raises dramatically. If file belongs to another module, evaluation time drops under 1 sec. I''m using puppet 2.7.11 under Ubuntu 12.04 LTS. My agents are running under Solaris 10. Can anybody help me to reduce this execution time, please? Thank you! info: /Stage[main]/xxxxxx::Website/xxxxxxx::Website_defs::Virtual_host[cl.xxxxxxx.com]/File[/etc/apache2/conf.d/alias_cl.xxxxxxx.com.inc]: Evaluated in 2.57 seconds info: /Stage[main]/xxxxxx::Website/xxxxxxx::Website_defs::Virtual_host[ca.xxxxxxx.com]/File[/etc/apache2/conf.d/alias_ca.xxxxxxx.com.inc]: Starting to evaluate the resource info: /Stage[main]/xxxxxx::Website/xxxxxxx::Website_defs::Virtual_host[ca.xxxxxxx.com]/File[/etc/apache2/conf.d/alias_ca.xxxxxxx.com.inc]: Evaluated in 2.57 seconds info: /Stage[main]/xxxxxx::Website/xxxxxxx::Website_defs::Virtual_host[m.za.xxxxxxx.com]/File[/etc/apache2/conf.d/alias_m.za.xxxxxxx.com.inc]: Starting to evaluate the resource info: /Stage[main]/xxxxxx::Website/xxxxxxx::Website_defs::Virtual_host[m.za.xxxxxxx.com]/File[/etc/apache2/conf.d/alias_m.za.xxxxxxx.com.inc]: Evaluated in 2.57 seconds info: /Stage[main]/xxxxxx::Website/xxxxxxx::Website_defs::Virtual_host[www.lowcost-xxxxxxx.es]/File[/etc/apache2/conf.d/alias_www.lowcost-xxxxxxx.es.inc]: Starting to evaluate the resource info: /Stage[main]/xxxxxx::Website/xxxxxxx::Website_defs::Virtual_host[www.lowcost-xxxxxxx.es]/File[/etc/apache2/conf.d/alias_www.lowcost-xxxxxxx.es.inc]: Evaluated in 2.59 seconds -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi, On a VM Windows 7 32bits, I have the bug describe here : http://projects.puppetlabs.com/issues/10261 . Reading the bug history, I understood that the bug has been accepted, corrected, and merged. Did I miss understand ? I have almost the last version of facter, I installed it middle of December 2012. Cordialement, Bernard Granier CE Plateforme Système bernard.granier@morpho.com 01 58 11 32 51 # " This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system." # -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Hi Bernard, On Tue, Jan 22, 2013 at 2:27 AM, GRANIER Bernard (MORPHO) <bernard.granier@morpho.com> wrote:> Hi, > > On a VM Windows 7 32bits, I have the bug describe here : http://projects.puppetlabs.com/issues/10261 . > > Reading the bug history, I understood that the bug has been accepted, corrected, and merged. > Did I miss understand ? > > I have almost the last version of facter, I installed it middle of December 2012. > > Cordialement, > > Bernard Granier > CE Plateforme Système > bernard.granier@morpho.com > 01 58 11 32 51 > > # > " This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system." > # > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >Ticket #10261 was that facter always reported the architecture as x86, even when running a 64-bit OS on x64 hardware. However, there is a still unresolved ticket #16948 when running a 32-bit OS on x64 hardware. Josh -- Josh Cooper Developer, Puppet Labs -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2013-Jan-22 18:34 UTC
[Puppet Users] Re: Performance really bad when executing agents
On Tuesday, January 22, 2013 4:18:51 AM UTC-6, MorSa wrote:> > Hi, > > i''ve got a performance problem when running my puppet agents. I need > to apply tons of files included in my website module and i assume that > executing time would be long, but i can''t understand why every time > when a File > is evaluated, it takes about 2-3 secs to do it. In my case i''ve got > tonf of files in this module, and time raises dramatically. If file > belongs to another module, evaluation time drops under 1 sec. > > I''m using puppet 2.7.11 under Ubuntu 12.04 LTS. My agents are running > under Solaris 10. Can anybody help me to reduce this execution time, > please? > >To determine whether a file''s content is already in sync, Puppet computes a checksum of it and of the target content, and compares. By default, it uses MD5 for the checksums, and that''s probably what take most of the 2-3 seconds per file. You might be able to speed it up a bit by setting "checksum => md5lite" on your file resources. Or you could probably speed it up a lot, at the cost of a significant risk of false positive comparisons, by setting "checksum => mtime" to use last modification time for the checksum. Really, though, those are band aids. If you have a large number of related files then you probably shouldn''t be managing them individually via Puppet at all. Instead, package them up and manage the package, or sync them directly with your version-control repository (under Puppet control, if you wish). John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/OjsI4DYgABwJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Ok thanks for your answer Cordialement, Bernard Granier CE Plateforme Système bernard.granier@morpho.com 01 58 11 32 51 -----Original Message----- From: puppet-users@googlegroups.com [mailto:puppet-users@googlegroups.com] On Behalf Of Josh Cooper Sent: Tuesday, January 22, 2013 6:23 PM To: puppet-users@googlegroups.com Subject: Re: [Puppet Users] facter : Bug #10261 Hi Bernard, On Tue, Jan 22, 2013 at 2:27 AM, GRANIER Bernard (MORPHO) <bernard.granier@morpho.com> wrote:> Hi, > > On a VM Windows 7 32bits, I have the bug describe here : http://projects.puppetlabs.com/issues/10261 . > > Reading the bug history, I understood that the bug has been accepted, corrected, and merged. > Did I miss understand ? > > I have almost the last version of facter, I installed it middle of December 2012. > > Cordialement, > > Bernard Granier > CE Plateforme Système > bernard.granier@morpho.com > 01 58 11 32 51 > > # > " This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system." > # > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. >Ticket #10261 was that facter always reported the architecture as x86, even when running a 64-bit OS on x64 hardware. However, there is a still unresolved ticket #16948 when running a 32-bit OS on x64 hardware. Josh -- Josh Cooper Developer, Puppet Labs -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. # " This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system." # -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
MorSa
2013-Jan-24 10:20 UTC
[Puppet Users] Re: Performance really bad when executing agents
Hi,
first of all, thanks for your reply.
I''ve tried what you suggest, but now i''m receiving this
message in every
file''s evaluation:
err:
/Stage[main]/xxxxxxx::Website/xxxxxxxxx::Website_defs::Virtual_host[www.xxxxxxx.com.ve]/File[/etc/apache2/conf.d/vhostfake_www.xxxxxx.com.ve.conf]/content:
change from {md5lite}1c6184966d40bdfe8d6307f1b515c79b to
{md5}c0e8f9d616134259aa712464ae258919 failed: Could not rename temporary
file /etc/apache2/conf.d/vhostfake_www.xxxxxx.com.ve.conf.puppettmp_185 to
/etc/apache2/conf.d/vhostfake_www.xxxxxx.com.ve.conf: File written to disk
did not match checksum; discarding changes
({md5lite}c0e8f9d616134259aa712464ae258919 vs
{md5lite}1c6184966d40bdfe8d6307f1b515c79b) at
/etc/puppet/modules/xxxxxx/manifests/website_defs.pp:45 at
/etc/puppet/modules/xxxxxxx/manifests/website_defs.pp:45
After a bit of investigation, it seems there was a bug related to it:
http://en.usenet.digipedia.org/thread/16576/5276/
El martes, 22 de enero de 2013 19:34:22 UTC+1, jcbollinger
escribió:>
>
>
> On Tuesday, January 22, 2013 4:18:51 AM UTC-6, MorSa wrote:
>>
>> Hi,
>>
>> i''ve got a performance problem when running my puppet agents.
I need
>> to apply tons of files included in my website module and i assume that
>> executing time would be long, but i can''t understand why every
time
>> when a File
>> is evaluated, it takes about 2-3 secs to do it. In my case
i''ve got
>> tonf of files in this module, and time raises dramatically. If file
>> belongs to another module, evaluation time drops under 1 sec.
>>
>> I''m using puppet 2.7.11 under Ubuntu 12.04 LTS. My agents are
running
>> under Solaris 10. Can anybody help me to reduce this execution time,
>> please?
>>
>>
>
> To determine whether a file''s content is already in sync, Puppet
computes
> a checksum of it and of the target content, and compares. By default, it
> uses MD5 for the checksums, and that''s probably what take most of
the 2-3
> seconds per file. You might be able to speed it up a bit by setting
> "checksum => md5lite" on your file resources. Or you could
probably speed
> it up a lot, at the cost of a significant risk of false positive
> comparisons, by setting "checksum => mtime" to use last
modification time
> for the checksum.
>
> Really, though, those are band aids. If you have a large number of
> related files then you probably shouldn''t be managing them
individually via
> Puppet at all. Instead, package them up and manage the package, or sync
> them directly with your version-control repository (under Puppet control,
> if you wish).
>
>
> John
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/ujz4okaz9WUJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
jcbollinger
2013-Jan-24 14:52 UTC
[Puppet Users] Re: Performance really bad when executing agents
On Thursday, January 24, 2013 4:20:27 AM UTC-6, MorSa wrote:> > Hi, > > first of all, thanks for your reply. > > I''ve tried what you suggest, but now i''m receiving this message in every > file''s evaluation: > > err: /Stage[main]/xxxxxxx::Website/xxxxxxxxx::Website_defs::Virtual_host[ > www.xxxxxxx.com.ve]/File[/etc/apache2/conf.d/vhostfake_www.xxxxxx.com.ve.conf]/content: > change from {md5lite}1c6184966d40bdfe8d6307f1b515c79b to > {md5}c0e8f9d616134259aa712464ae258919 failed: Could not rename temporary > file /etc/apache2/conf.d/vhostfake_www.xxxxxx.com.ve.conf.puppettmp_185 to > /etc/apache2/conf.d/vhostfake_www.xxxxxx.com.ve.conf: File written to disk > did not match checksum; discarding changes > ({md5lite}c0e8f9d616134259aa712464ae258919 vs > {md5lite}1c6184966d40bdfe8d6307f1b515c79b) at > /etc/puppet/modules/xxxxxx/manifests/website_defs.pp:45 at > /etc/puppet/modules/xxxxxxx/manifests/website_defs.pp:45 > > After a bit of investigation, it seems there was a bug related to it: > http://en.usenet.digipedia.org/thread/16576/5276/ > >I wasn''t aware of that bug, but it just means that you need to resolve the problem in one of the other ways I suggested, or by some similar means. A solution of that type would really be the best way to go even if changing the checksum algorithm were a viable alternative. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/6AHa-0hWCIQJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.