Kurt
2009-Aug-30 15:56 UTC
[Puppet Users] puppet 0.24.8, random problem with very slow puppetruns - no stored-configuration
Hello list, In our company we use puppet to configure our ec2 instances. we start our servers with fresh instances debian lenny, some small and some large. we install puppet like this: apt-get -y --force-yes install ruby1.8 rubygems1.8 gem install puppet --include-dependencies --version 0.24.8 --no-rdoc -- no-ri ln -s /var/lib/gems/1.8/bin/puppetd /usr/sbin/puppetd ln -s /var/lib/gems/1.8/bin/puppet /usr/sbin/puppet ln -s /var/lib/gems/1.8/bin/facter /usr/bin/facter and create a puppet client config file: echo "[puppetd]" > /etc/puppet/puppetd.conf echo "logdir=/var/log/puppet" >> /etc/puppet/puppetd.conf echo "vardir=/var/lib/puppet" >> /etc/puppet/puppetd.conf echo "rundir=/var/run" >> /etc/puppet/puppetd.conf echo "server=<%=env_puppetmaster_fqdn%>" >> /etc/puppet/puppetd.conf echo "masterport=<%=pm_apache2_lb_port%>" >> /etc/puppet/puppetd.conf echo "pluginsync=true" >> /etc/puppet/puppetd.conf echo "factpath = /var/lib/puppet/lib/facter" >> /etc/puppet/ puppetd.conf and start a first puppetrun with /usr/sbin/puppetd --test -v --waitforcert 120 The problem: sometimes (not always) the client "breaks", uses 100%CPU, runs manifest very very slowly and sometimes never finishes. Killing the Client after puppetd -v --test --debug shows: debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]/ checksum: Initializing checksum hash debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]: Creating checksum {md5}d6a4b36fea3bf1f7cd0c6e70989ca8c7 notice: Caught INT; shutting down debug: Signal caught here: debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `call'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 262:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `each'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: 260:in `walk'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb: 116:in `tree_from_vertex'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb:37:in `dependencies'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: 346:in `failed_dependencies?'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: 619:in `skip?'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: 219:in `eval_resource'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: 310:in `evaluate'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:425:in `thinmark'' debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:424:in `thinmark'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: 309:in `evaluate'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: 303:in `collect'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: 303:in `evaluate'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/node/catalog.rb: 124:in `apply'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ master.rb:256:in `run'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:180:in `benchmark'' debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:179:in `benchmark'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ master.rb:255:in `run'' debug: /usr/lib/ruby/1.8/sync.rb:230:in `synchronize'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ master.rb:237:in `run'' debug: /var/lib/gems/1.8/gems/puppet-0.24.8/bin/puppetd:417 debug: /usr/sbin/puppetd:19:in `load'' debug: /usr/sbin/puppetd:19 notice: Shutting down debug: Storing state debug: Stored state in 0.37 seconds debug: Storing state debug: Stored state in 0.52 seconds The problem: sometimes this never happens on one ec2-instance. we can start new "fresh" instances and sometimes it always works with the same manifests file. we have different "roles" in our environment with different manifests/ classes/modules and when we start a puppetrun on role1 and another on role2 it happens that role2 fails and role1 not. Starting the run again sometimes both fail, sometimes both are ok, sometimes they switch. puppet mostly "stops" when creating checksums of configuration files (template based or via fileserver-mount) and at the end of the puppetrun. Files which are checked at the beginning of the puppetrun (because this Files are first checked because of require/subscribe direcives) always gets done fast. Files which are checked at the end hangs on md5summing. The thing is that I cant reproduce this problem when I want to. Deleting /var/lib/puppet/* and running the client does not always helps. Anyone experiences the same problem sometimes or could give a hint what it could be? Many thanks, Kurt B. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Joe McDonagh
2009-Sep-01 22:49 UTC
[Puppet Users] Re: puppet 0.24.8, random problem with very slow puppetruns - no stored-configuration
Kurt wrote:> Hello list, > > In our company we use puppet to configure our ec2 instances. > we start our servers with fresh instances debian lenny, some small and > some large. > > we install puppet like this: > apt-get -y --force-yes install ruby1.8 rubygems1.8 > gem install puppet --include-dependencies --version 0.24.8 --no-rdoc -- > no-ri > ln -s /var/lib/gems/1.8/bin/puppetd /usr/sbin/puppetd > ln -s /var/lib/gems/1.8/bin/puppet /usr/sbin/puppet > ln -s /var/lib/gems/1.8/bin/facter /usr/bin/facter > > and create a puppet client config file: > echo "[puppetd]" > /etc/puppet/puppetd.conf > echo "logdir=/var/log/puppet" >> /etc/puppet/puppetd.conf > echo "vardir=/var/lib/puppet" >> /etc/puppet/puppetd.conf > echo "rundir=/var/run" >> /etc/puppet/puppetd.conf > echo "server=<%=env_puppetmaster_fqdn%>" >> /etc/puppet/puppetd.conf > echo "masterport=<%=pm_apache2_lb_port%>" >> /etc/puppet/puppetd.conf > echo "pluginsync=true" >> /etc/puppet/puppetd.conf > echo "factpath = /var/lib/puppet/lib/facter" >> /etc/puppet/ > puppetd.conf > > > and start a first puppetrun with > /usr/sbin/puppetd --test -v --waitforcert 120 > > The problem: > > sometimes (not always) the client "breaks", uses 100%CPU, runs > manifest very very slowly and sometimes never finishes. > > Killing the Client after > puppetd -v --test --debug shows: > > debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service > [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]/ > checksum: Initializing checksum hash > debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service > [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]: > Creating checksum {md5}d6a4b36fea3bf1f7cd0c6e70989ca8c7 > notice: Caught INT; shutting down > debug: Signal caught here: > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `call'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb: > 116:in `tree_from_vertex'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb:37:in > `dependencies'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 346:in `failed_dependencies?'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 619:in `skip?'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 219:in `eval_resource'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 310:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:425:in > `thinmark'' > debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:424:in > `thinmark'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 309:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 303:in `collect'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 303:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/node/catalog.rb: > 124:in `apply'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:256:in `run'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:180:in > `benchmark'' > debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:179:in > `benchmark'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:255:in `run'' > debug: /usr/lib/ruby/1.8/sync.rb:230:in `synchronize'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:237:in `run'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/bin/puppetd:417 > debug: /usr/sbin/puppetd:19:in `load'' > debug: /usr/sbin/puppetd:19 > notice: Shutting down > debug: Storing state > debug: Stored state in 0.37 seconds > debug: Storing state > debug: Stored state in 0.52 seconds > > The problem: > > sometimes this never happens on one ec2-instance. > we can start new "fresh" instances and sometimes it always works with > the same manifests file. > we have different "roles" in our environment with different manifests/ > classes/modules and when we start a puppetrun on role1 and another on > role2 it happens that role2 fails and role1 not. Starting the run > again sometimes both fail, sometimes both are ok, sometimes they > switch. > > puppet mostly "stops" when creating checksums of configuration files > (template based or via fileserver-mount) and at the end of the > puppetrun. > Files which are checked at the beginning of the puppetrun (because > this Files are first checked because of require/subscribe direcives) > always gets done fast. Files which are checked at the end hangs on > md5summing. > > The thing is that I cant reproduce this problem when I want to. > > Deleting /var/lib/puppet/* and running the client does not always > helps. > > Anyone experiences the same problem sometimes or could give a hint > what it could be? > > Many thanks, > > Kurt B. > > > >Hi Kurt, possibly some kind of weird clash between gems and packages, which is fairly common in deb/ubuntu if you mix gems with os-native dpkg. I have the .24.8 package from the pool ftp.ubuntu.com (analogous to ftp.debian.com) which seems to work pretty well in my environment and I don''t really hit any ''strange'' issues. -- Joe McDonagh Operations Engineer www.colonfail.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Jordi Funollet
2009-Sep-03 17:46 UTC
[Puppet Users] Re: puppet 0.24.8, random problem with very slow puppetruns - no stored-configuration
On Wednesday 02 September 2009 00:49:01 Joe McDonagh wrote:> Hi Kurt, possibly some kind of weird clash between gems and packages, > which is fairly common in deb/ubuntu if you mix gems with os-native > dpkg. I have the .24.8 package from the pool ftp.ubuntu.com (analogous > to ftp.debian.com) which seems to work pretty well in my environment and > I don''t really hit any ''strange'' issues.You can get it from backports.org, packaged specifically for Debian. I install just ''puppet'' and ''facter'' from Backports and everything else from the official Debian repositories. -- ############################## ### Jordi Funollet ### http://www.terraquis.net --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Bostjan Skufca
2009-Sep-04 01:39 UTC
[Puppet Users] Re: puppet 0.24.8, random problem with very slow puppetruns - no stored-configuration
I just have to acknowledge that I too am seeing random problems with puppet which have advanced to "persistent problems". In the beginning it was fine, I installed ruby from sources, then facter and then puppet and it worked like a charm. Then, 2 months ago I noticed that puppet was just not functioning anymore. Around that time I did some minor library upgrade (x.xx.YY <-- only YY changed) to the systems but I did it WITH puppet and it was working OK then. But then it died. Usually when one upgrades libraries only a recompile is required. I did recompile ruby, no difference. Now if I try to run it, it only runs once, or not even once. Then it just stucks at some "select(..." system call and that is it. Both, puppetmasterd and puppetd. If I kill and start both (the master and the client) it maybe runs once, maybe. Now, I do not have issues with things breaking down. What bothers me is that I cannot analyze this problem. Despite all the --debugging and stracing I can''t figure out what is wrong. And this, sadly, is turning me to other solutions. Why sadly? Because it worked before like a charm and I know it can work like a charm, but if a tool is unreliable it is useless. Now, if anyone (besides the original poster) is experiencing similar problems and/or (even better) has found the cause for this, please let me know. Best regards, bostjan On Aug 30, 5:56 pm, Kurt <zarniwoop...@gmail.com> wrote:> Hello list, > > In our company we use puppet to configure our ec2 instances. > we start our servers with fresh instances debian lenny, some small and > some large. > > we install puppet like this: > apt-get -y --force-yes install ruby1.8 rubygems1.8 > gem install puppet --include-dependencies --version 0.24.8 --no-rdoc -- > no-ri > ln -s /var/lib/gems/1.8/bin/puppetd /usr/sbin/puppetd > ln -s /var/lib/gems/1.8/bin/puppet /usr/sbin/puppet > ln -s /var/lib/gems/1.8/bin/facter /usr/bin/facter > > and create a puppet client config file: > echo "[puppetd]" > /etc/puppet/puppetd.conf > echo "logdir=/var/log/puppet" >> /etc/puppet/puppetd.conf > echo "vardir=/var/lib/puppet" >> /etc/puppet/puppetd.conf > echo "rundir=/var/run" >> /etc/puppet/puppetd.conf > echo "server=<%=env_puppetmaster_fqdn%>" >> /etc/puppet/puppetd.conf > echo "masterport=<%=pm_apache2_lb_port%>" >> /etc/puppet/puppetd.conf > echo "pluginsync=true" >> /etc/puppet/puppetd.conf > echo "factpath = /var/lib/puppet/lib/facter" >> /etc/puppet/ > puppetd.conf > > and start a first puppetrun with > /usr/sbin/puppetd --test -v --waitforcert 120 > > The problem: > > sometimes (not always) the client "breaks", uses 100%CPU, runs > manifest very very slowly and sometimes never finishes. > > Killing the Client after > puppetd -v --test --debug shows: > > debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service > [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]/ > checksum: Initializing checksum hash > debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service > [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]: > Creating checksum {md5}d6a4b36fea3bf1f7cd0c6e70989ca8c7 > notice: Caught INT; shutting down > debug: Signal caught here: > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `call'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb: > 116:in `tree_from_vertex'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb:37:in > `dependencies'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 346:in `failed_dependencies?'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 619:in `skip?'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 219:in `eval_resource'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 310:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:425:in > `thinmark'' > debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:424:in > `thinmark'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 309:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 303:in `collect'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 303:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/node/catalog.rb: > 124:in `apply'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:256:in `run'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:180:in > `benchmark'' > debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:179:in > `benchmark'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:255:in `run'' > debug: /usr/lib/ruby/1.8/sync.rb:230:in `synchronize'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:237:in `run'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/bin/puppetd:417 > debug: /usr/sbin/puppetd:19:in `load'' > debug: /usr/sbin/puppetd:19 > notice: Shutting down > debug: Storing state > debug: Stored state in 0.37 seconds > debug: Storing state > debug: Stored state in 0.52 seconds > > The problem: > > sometimes this never happens on one ec2-instance. > we can start new "fresh" instances and sometimes it always works with > the same manifests file. > we have different "roles" in our environment with different manifests/ > classes/modules and when we start a puppetrun on role1 and another on > role2 it happens that role2 fails and role1 not. Starting the run > again sometimes both fail, sometimes both are ok, sometimes they > switch. > > puppet mostly "stops" when creating checksums of configuration files > (template based or via fileserver-mount) and at the end of the > puppetrun. > Files which are checked at the beginning of the puppetrun (because > this Files are first checked because of require/subscribe direcives) > always gets done fast. Files which are checked at the end hangs on > md5summing. > > The thing is that I cant reproduce this problem when I want to. > > Deleting /var/lib/puppet/* and running the client does not always > helps. > > Anyone experiences the same problem sometimes or could give a hint > what it could be? > > Many thanks, > > Kurt B.--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Bostjan Skufca
2009-Sep-04 01:45 UTC
[Puppet Users] Re: puppet 0.24.8, random problem with very slow puppetruns - no stored-configuration
Sorry, I forgot to mention what I AM seeing in the logs: ---> Could not call fileserver.list: #<#<Class:0xb6e9e8d0>: execution expired> b. On Aug 30, 5:56 pm, Kurt <zarniwoop...@gmail.com> wrote:> Hello list, > > In our company we use puppet to configure our ec2 instances. > we start our servers with fresh instances debian lenny, some small and > some large. > > we install puppet like this: > apt-get -y --force-yes install ruby1.8 rubygems1.8 > gem install puppet --include-dependencies --version 0.24.8 --no-rdoc -- > no-ri > ln -s /var/lib/gems/1.8/bin/puppetd /usr/sbin/puppetd > ln -s /var/lib/gems/1.8/bin/puppet /usr/sbin/puppet > ln -s /var/lib/gems/1.8/bin/facter /usr/bin/facter > > and create a puppet client config file: > echo "[puppetd]" > /etc/puppet/puppetd.conf > echo "logdir=/var/log/puppet" >> /etc/puppet/puppetd.conf > echo "vardir=/var/lib/puppet" >> /etc/puppet/puppetd.conf > echo "rundir=/var/run" >> /etc/puppet/puppetd.conf > echo "server=<%=env_puppetmaster_fqdn%>" >> /etc/puppet/puppetd.conf > echo "masterport=<%=pm_apache2_lb_port%>" >> /etc/puppet/puppetd.conf > echo "pluginsync=true" >> /etc/puppet/puppetd.conf > echo "factpath = /var/lib/puppet/lib/facter" >> /etc/puppet/ > puppetd.conf > > and start a first puppetrun with > /usr/sbin/puppetd --test -v --waitforcert 120 > > The problem: > > sometimes (not always) the client "breaks", uses 100%CPU, runs > manifest very very slowly and sometimes never finishes. > > Killing the Client after > puppetd -v --test --debug shows: > > debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service > [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]/ > checksum: Initializing checksum hash > debug: //Node[default]/milter-sms/milter-sms::mimedefang/Runit_service > [mimedefang-multiplexor]/File[run-logfile mimedefang-multiplexor]: > Creating checksum {md5}d6a4b36fea3bf1f7cd0c6e70989ca8c7 > notice: Caught INT; shutting down > debug: Signal caught here: > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `call'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 262:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `each'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/simple_graph.rb: > 260:in `walk'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb: > 116:in `tree_from_vertex'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/pgraph.rb:37:in > `dependencies'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 346:in `failed_dependencies?'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 619:in `skip?'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 219:in `eval_resource'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 310:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:425:in > `thinmark'' > debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:424:in > `thinmark'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 309:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 303:in `collect'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/transaction.rb: > 303:in `evaluate'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/node/catalog.rb: > 124:in `apply'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:256:in `run'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:180:in > `benchmark'' > debug: /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/util.rb:179:in > `benchmark'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:255:in `run'' > debug: /usr/lib/ruby/1.8/sync.rb:230:in `synchronize'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/lib/puppet/network/client/ > master.rb:237:in `run'' > debug: /var/lib/gems/1.8/gems/puppet-0.24.8/bin/puppetd:417 > debug: /usr/sbin/puppetd:19:in `load'' > debug: /usr/sbin/puppetd:19 > notice: Shutting down > debug: Storing state > debug: Stored state in 0.37 seconds > debug: Storing state > debug: Stored state in 0.52 seconds > > The problem: > > sometimes this never happens on one ec2-instance. > we can start new "fresh" instances and sometimes it always works with > the same manifests file. > we have different "roles" in our environment with different manifests/ > classes/modules and when we start a puppetrun on role1 and another on > role2 it happens that role2 fails and role1 not. Starting the run > again sometimes both fail, sometimes both are ok, sometimes they > switch. > > puppet mostly "stops" when creating checksums of configuration files > (template based or via fileserver-mount) and at the end of the > puppetrun. > Files which are checked at the beginning of the puppetrun (because > this Files are first checked because of require/subscribe direcives) > always gets done fast. Files which are checked at the end hangs on > md5summing. > > The thing is that I cant reproduce this problem when I want to. > > Deleting /var/lib/puppet/* and running the client does not always > helps. > > Anyone experiences the same problem sometimes or could give a hint > what it could be? > > Many thanks, > > Kurt B.--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---