Daniel Pittman
2009-Mar-12 06:17 UTC
[Puppet Users] puppetmaster stops responding to some clients?
G''day.
We are running into a regular problem, which I can''t track down, where
puppetmaster will simply stop responding appropriately to clients.
I have exhausted my ideas for debugging this, so I turn to y''all for
advice on where next to look in terms of tracking down and killing the
root of these issues.
The errors, on the client side, are roughly:
Mar 12 16:15:27 amhocn04 puppetd[17153]: Could not call puppetmaster.getconfig:
#<NoMethodError: private method `split'' called for nil:NilClass>
Mar 12 16:15:27 amhocn04 puppetd[17153]: Could not retrieve catalog: private
method `split'' called for nil:NilClass
Mar 12 16:15:27 amhocn04 puppetd[17153]:
(//Node[default]/apt/Apt::Key[backports.org]/Exec[/usr/bin/apt-key add
/etc/apt/backports.org.key]) Dependency file[/etc/apt/backports.org.key] has 1
failures
Mar 12 16:15:27 amhocn04 puppetd[17153]:
(//Node[default]/apt/Apt::Key[backports.org]/Exec[/usr/bin/apt-key add
/etc/apt/backports.org.key]) Skipping because of failed dependencies
[... and so forth for the rest of the cached config ...]
On the server side, nothing. Specifically, no errors in the rails
related logs, no errors in syslog, nothing useful shows up when running
puppetmaster with debug enabled, etc.
So, hints? It seems to be that the puppetmaster either refuses
connections part way through the transaction, or that something gets
significantly out of sync.
Both ends are running puppet{,master} 0.24.5-3~sd-backport-1, which is
our backport of the 0.24.5 puppet release on Debian etch.
Regards,
Daniel
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---
Paul Lathrop
2009-Mar-12 19:58 UTC
[Puppet Users] Re: puppetmaster stops responding to some clients?
Daniel, I started seeing these symptoms when my puppetmaster was overloaded with more clients than Webrick could handle. Are you running Webrick or something else? --Paul On Wed, Mar 11, 2009 at 11:17 PM, Daniel Pittman <daniel@rimspace.net> wrote:> > G''day. > > We are running into a regular problem, which I can''t track down, where > puppetmaster will simply stop responding appropriately to clients. > > I have exhausted my ideas for debugging this, so I turn to y''all for > advice on where next to look in terms of tracking down and killing the > root of these issues. > > The errors, on the client side, are roughly: > > Mar 12 16:15:27 amhocn04 puppetd[17153]: Could not call puppetmaster.getconfig: #<NoMethodError: private method `split'' called for nil:NilClass> > Mar 12 16:15:27 amhocn04 puppetd[17153]: Could not retrieve catalog: private method `split'' called for nil:NilClass > Mar 12 16:15:27 amhocn04 puppetd[17153]: (//Node[default]/apt/Apt::Key[backports.org]/Exec[/usr/bin/apt-key add /etc/apt/backports.org.key]) Dependency file[/etc/apt/backports.org.key] has 1 failures > Mar 12 16:15:27 amhocn04 puppetd[17153]: (//Node[default]/apt/Apt::Key[backports.org]/Exec[/usr/bin/apt-key add /etc/apt/backports.org.key]) Skipping because of failed dependencies > [... and so forth for the rest of the cached config ...] > > On the server side, nothing. Specifically, no errors in the rails > related logs, no errors in syslog, nothing useful shows up when running > puppetmaster with debug enabled, etc. > > So, hints? It seems to be that the puppetmaster either refuses > connections part way through the transaction, or that something gets > significantly out of sync. > > Both ends are running puppet{,master} 0.24.5-3~sd-backport-1, which is > our backport of the 0.24.5 puppet release on Debian etch. > > Regards, > Daniel > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Daniel Pittman
2009-Mar-13 01:06 UTC
[Puppet Users] Re: puppetmaster stops responding to some clients?
Paul Lathrop <paul@tertiusfamily.net> writes:> I started seeing these symptoms when my puppetmaster was overloaded > with more clients than Webrick could handle. Are you running Webrick > or something else?Whatever the default server is, so presumably Webrick, yes. Regards, Daniel --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Paul Lathrop
2009-Mar-13 01:19 UTC
[Puppet Users] Re: puppetmaster stops responding to some clients?
Daniel, Yep, you''ve hit the scaling wall. You need to check out http://reductivelabs.com/trac/puppet/wiki/UsingMongrel and switch to Mongrel sooner rather than later :-) --Paul On Thu, Mar 12, 2009 at 6:06 PM, Daniel Pittman <daniel@rimspace.net> wrote:> > Paul Lathrop <paul@tertiusfamily.net> writes: > >> I started seeing these symptoms when my puppetmaster was overloaded >> with more clients than Webrick could handle. Are you running Webrick >> or something else? > > Whatever the default server is, so presumably Webrick, yes. > > Regards, > Daniel > > > >--~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---
Daniel Pittman
2009-Mar-13 06:51 UTC
[Puppet Users] Re: puppetmaster stops responding to some clients?
Paul Lathrop <paul@tertiusfamily.net> writes:> On Thu, Mar 12, 2009 at 6:06 PM, Daniel Pittman <daniel@rimspace.net> wrote: >> >> Paul Lathrop <paul@tertiusfamily.net> writes: >> >>> I started seeing these symptoms when my puppetmaster was overloaded >>> with more clients than Webrick could handle. Are you running Webrick >>> or something else? >> >> Whatever the default server is, so presumably Webrick, yes. > > Yep, you''ve hit the scaling wall. You need to check out > http://reductivelabs.com/trac/puppet/wiki/UsingMongrel and switch to > Mongrel sooner rather than later :-)Well, that was anything but fun. Still, after some hours of suffering Puppet is now actually working when run under Mongrel, so we will see if this improves the situation. Regards, Daniel --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en -~----------~----~----~----~------~----~------~--~---