Hi!
I''m incrementally developing puppet rules on the master server and
applying them to the testing machine. For some reason puppet agent
applies changes from the master very slowly, about 3 minutes every
time. At this speed it''s very tiresome to test the new rules...
I''m running puppet agent as follows:
#puppet agent --no-daemonize --verbose --onetime --summarize --debug --
trace
Here is what it shows in the end:
Changes:
Events:
Resources:
Total: 46
Time:
Filebucket: 0.00
Exec: 0.00
Schedule: 0.00
Ssh authorized key: 0.00
Group: 0.00
Package: 0.00
User: 0.01
Service: 0.07
Config retrieval: 12.25
Last run: 1306935632
File: 84.25
Total: 96.58
During the run most of the time is spent in the following phases:
....
info: Applying configuration version ''1306935546''
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: Service[sshd](provider=redhat): Executing ''/sbin/service sshd
status''
debug: Puppet::Type::Service::ProviderRedhat: Executing ''/sbin/
chkconfig sshd''
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: Finishing transaction 23816162721700
Any ideas why this can be happening? Thanks.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
On Jun 1, 2011, at 7:04 AM, pachanga wrote:> Hi! > > I''m incrementally developing puppet rules on the master server and > applying them to the testing machine. For some reason puppet agent > applies changes from the master very slowly, about 3 minutes every > time. At this speed it''s very tiresome to test the new rules... > > I''m running puppet agent as follows: > > #puppet agent --no-daemonize --verbose --onetime --summarize --debug -- > trace > > Here is what it shows in the end: > > Changes: > Events: > Resources: > Total: 46 > Time: > Filebucket: 0.00 > Exec: 0.00 > Schedule: 0.00 > Ssh authorized key: 0.00 > Group: 0.00 > Package: 0.00 > User: 0.01 > Service: 0.07 > Config retrieval: 12.25 > Last run: 1306935632 > File: 84.25 > Total: 96.58 > > During the run most of the time is spent in the following phases: > > .... > info: Applying configuration version ''1306935546'' > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: Service[sshd](provider=redhat): Executing ''/sbin/service sshd > status'' > debug: Puppet::Type::Service::ProviderRedhat: Executing ''/sbin/ > chkconfig sshd'' > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw > yaml; using pson > debug: Finishing transaction 23816162721700 > > Any ideas why this can be happening? Thanks.This probably means you are doing one of these (all of which are slow): 1) Using "File" for big file(s) 2) Using "File" with "recurse=true" to deploy lots of little files. 3) Using "File" with recurse=true" to deploy into a directory with lots of little files, even though you are only deploying very few. ( purge=fasle ) 4) Using "File" over a connection with high ping. Of course, combining more than one is even worse. Mitigations for 1 or 2: 1) Put your files in a package and let your package manager of choice handle it for you. 2) Use rsync with a cron job or rsync with an exec Mitigation for 3: 1) Try "recurse=remote". Mitigation for 4: 1) If the combined size of the files are small, you can try using "content" instead of "source" for the files. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
> Mitigation for 4: > 1) If the combined size of the files are small, you can try using "content" > instead of "source" for the files.Wow, that did the trick.... Thanks a lot! Deployment time is now 10-12 seconds. I''m a puppet newbie and don''t quite understand why it worked but it''s great :) -- Best regards, Pavel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Wed, 01 Jun 2011 23:58:24 +0400, Pavel Shevaev wrote:> > > Mitigation for 4: > > 1) If the combined size of the files are small, you can try using "content" > > instead of "source" for the files. > > Wow, that did the trick.... Thanks a lot! Deployment time is now 10-12 > seconds. I''m a puppet newbie and don''t quite understand why it worked > but it''s great :) >So, the big difference there is that by using ''content'', the file contents are shipped down to the agent as part of the catalog. With ''source'', the agent will have to make a new HTTP request per file to get the content when it attempts to manage the resource. You end up saving a potentially large number of HTTP round-trips, which adds up. -- Jacob Helwig
> You end up saving a potentially large number of HTTP round-trips, which > adds up.Thanks for explanations -- Best regards, Pavel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Jun 1, 2011, at 1:30 PM, Jacob Helwig wrote:> On Wed, 01 Jun 2011 23:58:24 +0400, Pavel Shevaev wrote: >> >>> Mitigation for 4: >>> 1) If the combined size of the files are small, you can try using "content" >>> instead of "source" for the files. >> >> Wow, that did the trick.... Thanks a lot! Deployment time is now 10-12 >> seconds. I''m a puppet newbie and don''t quite understand why it worked >> but it''s great :) >> > > So, the big difference there is that by using ''content'', the file > contents are shipped down to the agent as part of the catalog. > > With ''source'', the agent will have to make a new HTTP request per file > to get the content when it attempts to manage the resource. > > You end up saving a potentially large number of HTTP round-trips, which > adds up.So, to expand on this, the reason why the files need to be "small" is because all the files''s contents get sent every run, instead of the normal puppet module that just sends the checksum and asks for the file if the checksum doesn''t match. (As a trivial example, the file''s checksum won''t match if the file is empty or doesn''t exist.) -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.