christian.le.flamm@gmail.com
2013-Mar-08 08:33 UTC
[Puppet Users] "Could not find class" hiccups *often* once after manifest/module changes
Hi, I’ve got no idea if this is a bug or some kind of misuse/misconfiguration. Hope somebody can help. Thanks! If I change a file in a module or manifest directory this *often* results in errors, mostly of the following kinds - Could not find class for on node - Cannot find definition Class on node These errors occur when puppet modules/manifests are changed – but it happens only once. If it happens affected puppet agents will use their cached catalog once. *The next time these agents request a catalog it is compiled flawlessly and everything will work like a charm* – all errors are gone. Problem is we have agents with large catalogs. Their catalog runs can take up to over a minute and so the time it takes for a change to be applied will then be doubled to take up to 2 or even 3 minutes. Usually changes of modules/manifests are performed by “git pull”. But it’s easy to reproduce these error messages by simple “touch” operations. Here are some interesting log observations and their [commonness]. Yes AFAICS they always appear in triplets. It rarely happens that there are no problems at all after a modification.> touch /opt/xxx/dev/puppet/manifests/site.pp[often] puppet-master[12498]: Could not find class <class> for <node> on <node> [often] puppet-master[12498]: Could not find class <class> for <node> on <node> [often] puppet-master[12498]: Could not find class <class> for <node> on <node>> touch /opt/xxx/dev/puppet/puppet.conf[often] puppet-master[12498]: Could not find class <class> for <nodeY> on <node> [often] puppet-master[12498]: Could not find class <class> for <nodeY> on <node> [often] puppet-master[12498]: Could not find class <class> for <nodeY> on <node> [sometimes] puppet-master[12498]: Cannot find definition Class on node <node> [sometimes] puppet-master[12498]: Cannot find definition Class on node <node> [sometimes] puppet-master[12498]: Cannot find definition Class on node <node> [rare] puppet-master[12498]: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type <own custom type> at <class file> on node <node> [rare] puppet-master[12498]: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type <own custom type> at <class file> on node <node> [rare] puppet-master[12498]: Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type <own custom type> at <class file> on node <node>> touch /opt/xxx/dev/puppet/modules/dispatcher/manifests/init.pp[often] puppet-master[12498]: Could not find class <class not even part of module dispatcher> for <node not even affected by module dispatcher> on <node not even...> [often] puppet-master[12498]: Could not find class <class not even part of module dispatcher> for <node not even affected by module dispatcher> on <node not even...> [often] puppet-master[12498]: Could not find class <class not even part of module dispatcher> for <node not even affected by module dispatcher> on <node not even...> Affected classes and nodes seem to be randomly chosen. Environment this runs in:> ll /etc/puppetlrwxrwxrwx 1 root root 34 Mar 7 10:01 auth.conf -> /opt/xxx/dev/puppet/auth.conf lrwxrwxrwx 1 root root 38 Mar 7 10:01 autosign.conf -> /opt/xxx/dev/puppet/autosign.conf lrwxrwxrwx 1 root root 40 Mar 7 10:01 fileserver.conf -> /opt/xxx/dev/puppet/fileserver.conf lrwxrwxrwx 1 root root 36 Mar 7 10:01 puppet.conf -> /opt/xxx/dev/puppet/puppet.conf> ls -A /opt/xxx/dev/puppet/auth.conf autosign.conf fileserver.conf .git .gitignore manifests modules .project puppet.conf scripts> cat /etc/puppet/puppet.conf # on puppetmaster[main] logdir = /var/log/puppet rundir = /var/run/puppet ssldir = $vardir/ssl modulepath = /opt/xxx/dev/puppet/modules manifestdir = /opt/xxx/dev/puppet/manifests manifest = /opt/xxx/dev/puppet/manifests/site.pp [agent] classfile = $vardir/classes.txt localconfig = $vardir/localconfig server = <puppetmaster host> report = true splaylimit = 0 runinterval = 30 [master] certname=<puppetmaster host> reports = http,log reportdir = /var/lib/puppet/reports/upload reporturl = http://<puppetmaster host>:3000/reports> cat /opt/xxx/dev/puppet/manifests/site.ppimport ''nodes.pp'' $puppetserver = <puppetmaster host>> rpm -qa | egrep "puppet|ruby"rubygem-rake-0.8.7-2.1.el6.noarch ruby-mysql-2.8.2-1.el6.x86_64 libselinux-ruby-2.0.94-5.3.el6.x86_64 puppet-3.1.0-1.el6.noarch rubygem-fastthread-1.0.7-2.el6.x86_64 rubygem-mongrel-1.1.5-3.el6.x86_64 ruby-1.8.7.352-7.el6_2.x86_64 ruby-irb-1.8.7.352-7.el6_2.x86_64 ruby-augeas-0.4.1-1.el6.x86_64 ruby-shadow-1.4.1-13.el6.x86_64 puppetlabs-release-6-6.noarch rubygems-1.3.7-1.el6.noarch puppet-server-3.1.0-1.el6.noarch rubygem-gem_plugin-0.2.3-3.el6.noarch rubygem-daemons-1.0.10-2.el6.noarch puppet-dashboard-1.2.22-1.el6.noarch ruby-libs-1.8.7.352-7.el6_2.x86_64 ruby-rdoc-1.8.7.352-7.el6_2.x86_64 rubygem-json-1.4.6-1.el6.x86_64 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
David Schmitt
2013-Mar-08 08:36 UTC
Re: [Puppet Users] "Could not find class" hiccups *often* once after manifest/module changes
On 08.03.2013 09:33, christian.le.flamm@gmail.com wrote:> Hi, I’ve got no idea if this is a bug or some kind of > misuse/misconfiguration. Hope somebody can help. Thanks! > > If I change a file in a module or manifest directory this > /often/ results in errors, mostly of the following kinds > > * > > Could not find class for on node > > * > > Cannot find definition Class on node > > These errors occur when puppet modules/manifests are changed – but it > happens only once. If it happens affected puppet agents will use their > cached catalog once. *The next time these agents request a catalog it is > compiled flawlessly and everything will work like a charm* – all errors > are gone. Problem is we have agents with large catalogs. Their catalog > runs can take up to over a minute and so the time it takes for a change > to be applied will then be doubled to take up to 2 or even 3 minutes. > > Usually changes of modules/manifests are performed by “git pull”. But > it’s easy to reproduce these error messages by simple “touch” > operations. Here are some interesting log observations and their > [commonness]. Yes AFAICS they always appear in triplets. It rarely > happens that there are no problems at all after a modification.Hi, I''ve reported a similar issue recently here: https://projects.puppetlabs.com/issues/19638 I''ve tracked my problem to overshooting use of "import" calls in modules. You might want to check if you''re hitting the same bug. Best Regards, David -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
christian.le.flamm@gmail.com
2013-Mar-08 09:33 UTC
Re: [Puppet Users] "Could not find class" hiccups *often* once after manifest/module changes
Hi David, thanks for your answer! I only have the ''nodes.pp'' import: > egrep -R "import[ \t]" /opt/xxx/dev/puppet/ /opt/xxx/dev/puppet/manifests/site.pp:import ''nodes.pp'' Must I get rid of this? Best Regards, Christian> > Hi, > > I''ve reported a similar issue recently here: > > https://projects.puppetlabs.com/issues/19638 > > I''ve tracked my problem to overshooting use of "import" calls in modules. > > You might want to check if you''re hitting the same bug. > > > Best Regards, David > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
David Schmitt
2013-Mar-08 10:08 UTC
Re: [Puppet Users] "Could not find class" hiccups *often* once after manifest/module changes
On 08.03.2013 10:33, christian.le.flamm@gmail.com wrote:> Hi David, thanks for your answer! I only have the ''nodes.pp'' import: > > > egrep -R "import[ \t]" /opt/xxx/dev/puppet/ > /opt/xxx/dev/puppet/manifests/site.pp:import ''nodes.pp'' > > Must I get rid of this?You might want to run the puppetmaster from the shell with debugging enabled (--no-daemonize --masterport 8888 --verbose --debug) and post the resulting log when you run an agent against that (--masterport 8888 --test). In my case the puppet master actually loads the file which contains the class, but fails to correctly register the contained class until the next run. I only observed (and tested) that with parameterized classes. D.> Best Regards, Christian > > > Hi, > > I''ve reported a similar issue recently here: > > https://projects.puppetlabs.com/issues/19638 > <https://projects.puppetlabs.com/issues/19638> > > I''ve tracked my problem to overshooting use of "import" calls in > modules. > > You might want to check if you''re hitting the same bug. > > > Best Regards, David > > > -- > You received this message because you are subscribed to the Google > Groups "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to puppet-users+unsubscribe@googlegroups.com. > To post to this group, send email to puppet-users@googlegroups.com. > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
christian.le.flamm@gmail.com
2013-Mar-08 12:06 UTC
Re: [Puppet Users] "Could not find class" hiccups *often* once after manifest/module changes
Hey, here''s a detailed error example: Debug: importing ''/opt/xxx/dev/puppet/modules/xxxbase/manifests/packages/basictools.pp'' in environment production Debug: Automatically imported xxxbase::packages::basictools from xxxbase/packages/basictools into production Error: Could not find class xxxbase::packages::basictools for <node> on node <node> Error: Could not find class xxxbase::packages::basictools for <node> on node <node> Error: Could not find class xxxbase::packages::basictools for <node> on node <node> /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:168:in `evaluate_classes'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:151:in `each'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:151:in `evaluate_classes'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions/include.rb:11:in `real_function_include'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions.rb:146:in `send'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions.rb:146:in `function_include'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/function.rb:31:in `send'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/function.rb:31:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast.rb:62:in `safeevaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/astarray.rb:25:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/astarray.rb:20:in `each'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/astarray.rb:20:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast.rb:62:in `safeevaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/resource/type.rb:136:in `evaluate_code'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/resource.rb:81:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:166:in `evaluate_classes'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:151:in `each'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:151:in `evaluate_classes'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions/include.rb:11:in `real_function_include'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions.rb:146:in `send'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions.rb:146:in `function_include'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/function.rb:31:in `send'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/function.rb:31:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast.rb:62:in `safeevaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/astarray.rb:25:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/astarray.rb:20:in `each'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast/astarray.rb:20:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/ast.rb:62:in `safeevaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/resource/type.rb:136:in `evaluate_code'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/resource.rb:81:in `evaluate'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:220:in `evaluate_ast_node'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:98:in `compile'' /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:29:in `compile'' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/catalog/compiler.rb:76:in `compile'' /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:179:in `benchmark'' /usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'' /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:178:in `benchmark'' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/catalog/compiler.rb:74:in `compile'' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/catalog/compiler.rb:34:in `find'' /usr/lib/ruby/site_ruby/1.8/puppet/indirector/indirection.rb:191:in `find'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:108:in `do_find'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:71:in `send'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/handler.rb:71:in `process'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick/rest.rb:24:in `service'' /usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'' /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:33:in `listen'' /usr/lib/ruby/1.8/webrick/server.rb:173:in `call'' /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start'' /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'' /usr/lib/ruby/1.8/webrick/server.rb:95:in `start'' /usr/lib/ruby/1.8/webrick/server.rb:92:in `each'' /usr/lib/ruby/1.8/webrick/server.rb:92:in `start'' /usr/lib/ruby/1.8/webrick/server.rb:23:in `start'' /usr/lib/ruby/1.8/webrick/server.rb:82:in `start'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:30:in `listen'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:29:in `initialize'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:29:in `new'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:29:in `listen'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:26:in `synchronize'' /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:26:in `listen'' /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:92:in `listen'' /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:104:in `start'' /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:136:in `start'' /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:207:in `main'' /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:157:in `run_command'' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in `run'' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:456:in `plugin_hook'' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in `run'' /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:504:in `exit_on_fail'' /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in `run'' /usr/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:132:in `run'' /usr/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:86:in `execute'' /usr/bin/puppet:4 In /usr/lib/ruby/site_ruby/1.8/puppet/parser/compiler.rb:168 error "Puppet::Error, "Could not find class #{name} for #{node.name}"" is raised because a couple of lines above a check "if klass = scope.find_hostclass(name, :assume_fqname => fqname)" fails, see: def evaluate_classes(classes, scope, lazy_evaluate = true, fqname = false) #Puppet.info classes raise Puppet::DevError, "No source for scope passed to evaluate_classes" unless scope.source class_parameters = nil # if we are a param class, save the classes hash # and transform classes to be the keys if classes.class == Hash class_parameters = classes classes = classes.keys end classes.each do |name| # If we can find the class, then make a resource that will evaluate it. if klass = scope.find_hostclass(name, :assume_fqname => fqname) ### <<<-- This check fails # If parameters are passed, then attempt to create a duplicate resource # so the appropriate error is thrown. if class_parameters resource = klass.ensure_in_catalog(scope, class_parameters[name] || {}) else next if scope.class_scope(klass) resource = klass.ensure_in_catalog(scope) end # If they''ve disabled lazy evaluation (which the :include function does), # then evaluate our resource immediately. resource.evaluate unless lazy_evaluate else raise Puppet::Error, "Could not find class #{name} for #{node.name}" ### <<<--- Here error is raised end end end in /usr/lib/ruby/site_ruby/1.8/puppet/parser/scope.rb:134 def find_hostclass(name, options = {}) known_resource_types.find_hostclass(namespaces, name, options) end in /usr/lib/ruby/site_ruby/1.8/puppet/resource/type_collection.rb:114 def find_hostclass(namespaces, name, options = {}) find_or_load(namespaces, name, :hostclass, options) end in /usr/lib/ruby/site_ruby/1.8/puppet/resource/type_collection.rb:197 # Resolve namespaces and find the given object. Autoload it if # necessary. def find_or_load(namespaces, name, type, options = {}) searchspace = options[:assume_fqname] ? [name].flatten : resolve_namespaces(namespaces, name) searchspace.each do |fqname| result = send(type, fqname) unless result # do not try to autoload if we already tried and it wasn''t conclusive # as this is a time consuming operation. unless @notfound[fqname] result = loader.try_load_fqname(type, fqname) @notfound[fqname] = result.nil? end end return result if result end return nil end Am Freitag, 8. März 2013 11:08:57 UTC+1 schrieb David Schmitt:> On 08.03.2013 10:33, christian...@gmail.com <javascript:> wrote: > > Hi David, thanks for your answer! I only have the ''nodes.pp'' import: > > > > > egrep -R "import[ \t]" /opt/xxx/dev/puppet/ > > /opt/xxx/dev/puppet/manifests/site.pp:import ''nodes.pp'' > > > > Must I get rid of this? > > You might want to run the puppetmaster from the shell with debugging > enabled (--no-daemonize --masterport 8888 --verbose --debug) and post > the resulting log when you run an agent against that (--masterport 8888 > --test). In my case the puppet master actually loads the file which > contains the class, but fails to correctly register the contained class > until the next run. I only observed (and tested) that with parameterized > classes. > > > D. > > > Best Regards, Christian > > > > > > Hi, > > > > I''ve reported a similar issue recently here: > > > > https://projects.puppetlabs.com/issues/19638 > > <https://projects.puppetlabs.com/issues/19638> > > > > I''ve tracked my problem to overshooting use of "import" calls in > > modules. > > > > You might want to check if you''re hitting the same bug. > > > > > > Best Regards, David > > > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Puppet Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to puppet-users...@googlegroups.com <javascript:>. > > To post to this group, send email to puppet...@googlegroups.com<javascript:>. > > > Visit this group at http://groups.google.com/group/puppet-users?hl=en. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
christian.le.flamm@gmail.com
2013-Mar-18 16:09 UTC
Re: [Puppet Users] "Could not find class" hiccups *often* once after manifest/module changes
Switching to an Apache/Passenger/Rack solution (and dumping WEBrick) seems to solve this problem. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.