After an unexpected and unreasonably long hiatus, I''ve finally returned to implementing Puppet in my environment. My puppetmaster: Puppet 0.25.4, Facter 1.5.7, Ruby 1.8.7, and FreeBSD 8.0. I''ve got five clients using the puppetmaster so far. After about 24 hours, I''ve got three puppetmaster processes running. This number continues to grow over time. If stored configs are enabled, catalog requests eventually start failing because they''ve maxed out my allowed DB connections. The children processes are in an Idle state. Sending a sigterm to the children has zero effect. Killing the parent sends it into its stop routine (which prevents new connections), but never actually terminates the process (presumably because it''s waiting for its idle children to terminate). Kill -9 does the trick, but I don''t like the idea of clobbering an in-progress transaction. Is this a known problem? -SH -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Feb 16, 2010 at 9:05 AM, S H <shdashbeta@gmail.com> wrote:> After an unexpected and unreasonably long hiatus, I''ve finally returned to > implementing Puppet in my environment. > > My puppetmaster: Puppet 0.25.4, Facter 1.5.7, Ruby 1.8.7, and FreeBSD 8.0.Is this ruby 1.8.7 patch level 249 SH ?> > I''ve got five clients using the puppetmaster so far. After about 24 hours, > I''ve got three puppetmaster processes running. This number continues to grow > over time. If stored configs are enabled, catalog requests eventually start > failing because they''ve maxed out my allowed DB connections. > > The children processes are in an Idle state. Sending a sigterm to the > children has zero effect. Killing the parent sends it into its stop routine > (which prevents new connections), but never actually terminates the process > (presumably because it''s waiting for its idle children to terminate). Kill > -9 does the trick, but I don''t like the idea of clobbering an in-progress > transaction. > > Is this a known problem? > > > -SH > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- nigel -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Feb 16, 2010 at 12:11 PM, Nigel Kersten <nigelk@google.com> wrote:> On Tue, Feb 16, 2010 at 9:05 AM, S H <shdashbeta@gmail.com> wrote: > > After an unexpected and unreasonably long hiatus, I''ve finally returned > to > > implementing Puppet in my environment. > > > > My puppetmaster: Puppet 0.25.4, Facter 1.5.7, Ruby 1.8.7, and FreeBSD > 8.0. > > Is this ruby 1.8.7 patch level 249 SH ? >Patch level 248. As far as I know, that''s the most recent available. -SH> > > I''ve got five clients using the puppetmaster so far. After about 24 > hours, > > I''ve got three puppetmaster processes running. This number continues to > grow > > over time. If stored configs are enabled, catalog requests eventually > start > > failing because they''ve maxed out my allowed DB connections. > > > > The children processes are in an Idle state. Sending a sigterm to the > > children has zero effect. Killing the parent sends it into its stop > routine > > (which prevents new connections), but never actually terminates the > process > > (presumably because it''s waiting for its idle children to terminate). > Kill > > -9 does the trick, but I don''t like the idea of clobbering an in-progress > > transaction. > > > > Is this a known problem? > > > > > > -SH > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > > For more options, visit this group at > > http://groups.google.com/group/puppet-users?hl=en. > > > > > > -- > nigel > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
S H wrote:> After an unexpected and unreasonably long hiatus, I''ve finally returned > to implementing Puppet in my environment. > > My puppetmaster: Puppet 0.25.4, Facter 1.5.7, Ruby 1.8.7, and FreeBSD 8.0. > > I''ve got five clients using the puppetmaster so far. After about 24 > hours, I''ve got three puppetmaster processes running. This number > continues to grow over time. If stored configs are enabled, catalog > requests eventually start failing because they''ve maxed out my allowed > DB connections. >Are you using the mysql gem? -scott -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Feb 16, 2010 at 1:13 PM, Scott Smith <scott@ohlol.net> wrote:> S H wrote: > >> After an unexpected and unreasonably long hiatus, I''ve finally returned to >> implementing Puppet in my environment. >> >> My puppetmaster: Puppet 0.25.4, Facter 1.5.7, Ruby 1.8.7, and FreeBSD 8.0. >> >> I''ve got five clients using the puppetmaster so far. After about 24 hours, >> I''ve got three puppetmaster processes running. This number continues to grow >> over time. If stored configs are enabled, catalog requests eventually start >> failing because they''ve maxed out my allowed DB connections. >> >> > Are you using the mysql gem? > > -scott >No, I''m using postgresql. This also happens with stored configs disabled, though I don''t know if it''s still loading up ruby-pg. -SH> > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Feb 16, 2010 at 1:21 PM, S H <shdashbeta@gmail.com> wrote:> > > On Tue, Feb 16, 2010 at 1:13 PM, Scott Smith <scott@ohlol.net> wrote: > >> S H wrote: >> >>> After an unexpected and unreasonably long hiatus, I''ve finally returned >>> to implementing Puppet in my environment. >>> >>> My puppetmaster: Puppet 0.25.4, Facter 1.5.7, Ruby 1.8.7, and FreeBSD >>> 8.0. >>> >>> I''ve got five clients using the puppetmaster so far. After about 24 >>> hours, I''ve got three puppetmaster processes running. This number continues >>> to grow over time. If stored configs are enabled, catalog requests >>> eventually start failing because they''ve maxed out my allowed DB >>> connections. >>> >>> >> Are you using the mysql gem? >> >> -scott >> > > No, I''m using postgresql. This also happens with stored configs disabled, > though I don''t know if it''s still loading up ruby-pg. > > -SH >Anything else make sense to check? After about a week I''ve got 19 processes running. -SH -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On 2/22/10 7:41 AM, S H wrote:> Anything else make sense to check? After about a week I''ve got 19 > processes running. >Are you using Passenger? If not, try it. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Mon, Feb 22, 2010 at 11:18 AM, Scott Smith <scott@ohlol.net> wrote:> On 2/22/10 7:41 AM, S H wrote: > >> Anything else make sense to check? After about a week I''ve got 19 >> processes running. >> >> > Are you using Passenger? If not, try it.I''ve just moved my Puppetmaster behind Passenger and will let you know if this fixes it. -SH> > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Marcello de Sousa
2010-Feb-26 16:56 UTC
[Puppet Users] Foreman environments vs. Puppet Environments
Does anybody know or have a Howto on how to use "Foreman environments" and their relationship and interaction with "puppet environments" ? If they are not related, is there a way to assign a machine to a "puppet environment" via Foreman''s interface ? Cheers, Marcello -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Ohad Levy
2010-Feb-26 17:55 UTC
Re: [Puppet Users] Foreman environments vs. Puppet Environments
Foreman can manage puppet environment assignments to hosts if you use external nodes mode (http://theforeman.org/wiki/foreman/External_Nodes). Foreman can scan your existing modules assigning the right classes to the relevant environments. there is another environment settings in foreman (e.g. if you start the server in the command line) - this is rails environment and is not related to puppet at all. hope this helps, Ohad On Fri, Feb 26, 2010 at 6:56 PM, Marcello de Sousa <lists@area151.com>wrote:> Does anybody know or have a Howto on how to use "Foreman environments" and > their relationship and interaction with "puppet environments" ? > > If they are not related, is there a way to assign a machine to a "puppet > environment" via Foreman''s interface ? > > Cheers, > Marcello > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Marcello de Sousa
2010-Feb-26 21:29 UTC
RE: [Puppet Users] Foreman environments vs. Puppet Environments
Hi Ohad,
On your forum I''ve posted a bit more extensive question
(http://theforeman.org/boards/2/topics/show/119) just in case this is a bit
offtopic here.
I''m already using Foreman for a while with external node mode.
But I''ve configured now puppet with environments like this:
=================(...)
[development]
modulepath = /etc/puppet/development/modules:/etc/puppet/modules
[testing]
modulepath = /etc/puppet/testing/modules:/etc/puppet/modules
=================But I don''t see these environments showing up on the
Foreman interface.
I''ve also tried to create an environment inside foreman and assign it
to a
host, but I don''t see it having any effect on the class assignment.
In other words, foreman''s environments are not working here and not
relating
to puppet''s, and I can''t find documentation about it in
foreman''s wiki or
anywhere.
It might be worth mentioning that I''m not using foreman to deploy new
machines yet. I''m managing existing machines and now they are all
assigned
to "production", and I would like now to assign some of them to
"development" and "testing" to be able to develop and deploy
new
modules/classes to them without affecting production.
Pls Help ? :)
Thanks!
Cheers,
Marcello
> -----Original Message-----
> From: puppet-users@googlegroups.com [mailto:puppet-
> users@googlegroups.com] On Behalf Of Ohad Levy
> Sent: vrijdag 26 februari 2010 18:55
> To: puppet-users@googlegroups.com
> Subject: Re: [Puppet Users] Foreman environments vs. Puppet
> Environments
>
> Foreman can manage puppet environment assignments to hosts if you use
> external nodes mode
> (http://theforeman.org/wiki/foreman/External_Nodes).
>
> Foreman can scan your existing modules assigning the right classes to
> the relevant environments.
>
> there is another environment settings in foreman (e.g. if you start the
> server in the command line) - this is rails environment and is not
> related to puppet at all.
>
> hope this helps,
> Ohad
>
>
> On Fri, Feb 26, 2010 at 6:56 PM, Marcello de Sousa
<lists@area151.com>
> wrote:
>
>
> Does anybody know or have a Howto on how to use "Foreman
> environments" and
> their relationship and interaction with "puppet environments" ?
>
> If they are not related, is there a way to assign a machine to a
> "puppet
> environment" via Foreman''s interface ?
>
> Cheers,
> Marcello
>
> --
> You received this message because you are subscribed to the
> Google Groups "Puppet Users" group.
> To post to this group, send email to puppet-
> users@googlegroups.com.
> To unsubscribe from this group, send email to puppet-
> users+unsubscribe@googlegroups.com <mailto:puppet-
> users%2Bunsubscribe@googlegroups.com> .
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
>
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to puppet-
> users+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
On Fri, Feb 26, 2010 at 11:41 AM, S H <shdashbeta@gmail.com> wrote:> > On Mon, Feb 22, 2010 at 11:18 AM, Scott Smith <scott@ohlol.net> wrote: > >> On 2/22/10 7:41 AM, S H wrote: >> >>> Anything else make sense to check? After about a week I''ve got 19 >>> processes running. >>> >>> >> Are you using Passenger? If not, try it. > > > > I''ve just moved my Puppetmaster behind Passenger and will let you know if > this fixes it. > > -SH >Passenger did the trick. Thanks for your help. -SH> > >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> >> >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Mar 9, 2010 at 11:52 AM, S H <shdashbeta@gmail.com> wrote:> > > On Fri, Feb 26, 2010 at 11:41 AM, S H <shdashbeta@gmail.com> wrote: > >> >> On Mon, Feb 22, 2010 at 11:18 AM, Scott Smith <scott@ohlol.net> wrote: >> >>> On 2/22/10 7:41 AM, S H wrote: >>> >>>> Anything else make sense to check? After about a week I''ve got 19 >>>> processes running. >>>> >>>> >>> Are you using Passenger? If not, try it. >> >> >> >> I''ve just moved my Puppetmaster behind Passenger and will let you know if >> this fixes it. >> >> -SH >> > > Passenger did the trick. Thanks for your help. > > -SH > >Update: Passenger helped, but did not eliminate the problem. The number of httpd processes gradually climbs (highest I''ve seen is 109 - I only have 18 clients talking to Puppet right now) until it reaches Apache''s max, then client runs start failing. As a bandaid I''m restarting Apache nightly. At least with Passenger I''m able to kill the child processes without any grief. Let me know if there''s nay other information I can provide. I''m not sure where else to look. -SH> >> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Puppet Users" group. >>> To post to this group, send email to puppet-users@googlegroups.com. >>> To unsubscribe from this group, send email to >>> puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/puppet-users?hl=en. >>> >>> >> >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Charles Johnson
2010-Apr-27 13:27 UTC
Re: [Puppet Users] Puppetmaster child processes hang
On Tue, Apr 27, 2010 at 8:07 AM, S H <shdashbeta@gmail.com> wrote:> > Update: Passenger helped, but did not eliminate the problem. The number of > httpd processes gradually climbs (highest I''ve seen is 109 - I only have 18 > clients talking to Puppet right now) until it reaches Apache''s max, then > client runs start failing. > > As a bandaid I''m restarting Apache nightly. At least with Passenger I''m > able to kill the child processes without any grief. > > Let me know if there''s nay other information I can provide. I''m not sure > where else to look. > > -SH > > What passenger directives are you using in your apache config files?Cheers-- Charles -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Charles Johnson
2010-Apr-27 13:30 UTC
Re: [Puppet Users] Puppetmaster child processes hang
As an aside, I am using puppet/passenger/apache managing 850 nodes with no issues like the one you have mentioned. Cheers-- Charles On Tue, Apr 27, 2010 at 8:27 AM, Charles Johnson <gm.johns276@gmail.com>wrote:> On Tue, Apr 27, 2010 at 8:07 AM, S H <shdashbeta@gmail.com> wrote: > >> >> Update: Passenger helped, but did not eliminate the problem. The number of >> httpd processes gradually climbs (highest I''ve seen is 109 - I only have 18 >> clients talking to Puppet right now) until it reaches Apache''s max, then >> client runs start failing. >> >> As a bandaid I''m restarting Apache nightly. At least with Passenger I''m >> able to kill the child processes without any grief. >> >> Let me know if there''s nay other information I can provide. I''m not sure >> where else to look. >> >> -SH >> >> What passenger directives are you using in your apache config files? > > Cheers-- > > Charles >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Apr 27, 2010 at 9:27 AM, Charles Johnson <gm.johns276@gmail.com>wrote:> > > What passenger directives are you using in your apache config files? > >> Cheers-- > > Charles >The relevant Apache configuration: ---- LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.9/ext/apache2/mod_passenger.so PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.9 PassengerRuby /usr/local/bin/ruby # you probably want to tune these settings PassengerHighPerformance on PassengerMaxPoolSize 12 PassengerPoolIdleTime 1500 # PassengerMaxRequests 1000 PassengerStatThrottleRate 120 RackAutoDetect Off RailsAutoDetect Off # Puppet Production Listen 8140 <VirtualHost *:8140> SSLEngine on SSLProtocol -ALL +SSLv3 +TLSv1 SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP SSLCertificateFile /puppet/ssl/certs/puppet.mydomain.local.pem SSLCertificateKeyFile /puppet/ssl/private_keys/puppet.mydomain.local.pem SSLCertificateChainFile /puppet/ssl/ca/ca_crt.pem SSLCACertificateFile /puppet/ssl/ca/ca_crt.pem # If Apache complains about invalid signatures on the CRL, you can try disabling # CRL checking by commenting the next line, but this is not recommended. SSLCARevocationFile /puppet/ssl/ca/ca_crl.pem SSLVerifyClient optional SSLVerifyDepth 1 SSLOptions +StdEnvVars DocumentRoot /puppet/production/rack/public/ RackBaseURI / <Directory /puppet/production/rack/> Options None AllowOverride None Order allow,deny allow from all </Directory> </VirtualHost> ---- /puppet/production/rack/config.ru: ---- $0 = "puppetmasterd" require ''puppet'' # ARGV << "--debug" ARGV << "--rack" ARGV << "--confdir" << "/puppet/production" require ''puppet/application/puppetmasterd'' run Puppet::Application[:puppetmasterd].run ---- On Tue, Apr 27, 2010 at 9:30 AM, Charles Johnson <gm.johns276@gmail.com>wrote:> > > As an aside, I am using puppet/passenger/apache managing 850 nodes with no > issues like the one you have mentioned. > >> Cheers-- > > Charles >Given the limited response to this thread, I''m assuming this either a problem with my configuration or architecture (FreeBSD 8, x86). What OS is your puppetmaster running on? -SH -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
I also just discovered that some of my clients are locking up and requiring a SIGKILL to end them. This could be the root cause of the puppetmaster issues, so I''m going to run the clients as cron jobs for a while and see if that changes anything. -SH -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Charles Johnson
2010-Apr-27 17:51 UTC
Re: [Puppet Users] Puppetmaster child processes hang
Mine do not look too dissimilar: PassengerUseGlobalQueue on PassengerMaxPoolSize 20 PassengerHighPerformance on PassengerPoolIdleTime 1800 PassengerStatThrottleRate 120 RackAutoDetect Off RailsAutoDetect Off 1, Try using the global queue 2. Try dropping you pool idle time, which should force turnover/restarts of puppet. (300 is the default, IIRC). 3. Are you using 0.25.X 4. On one of your nodes try puppetd --debug and see if there are telling details there. I am running CentOS 5.x (RedHat essentially) with ruby, passenger and and puppet installed by hand in /usr/local rather than using a package system. Ruby 1.8.7, passenger 2.2.11. Apache is the "packaged" version installed with yum. 2.2.3 I assume that you are using the ports collection to get your installs done? Cheers-- Charles On Tue, Apr 27, 2010 at 10:56 AM, S H <shdashbeta@gmail.com> wrote:> I also just discovered that some of my clients are locking up and requiring > a SIGKILL to end them. This could be the root cause of the puppetmaster > issues, so I''m going to run the clients as cron jobs for a while and see if > that changes anything. > > > -SH > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Charles Johnson
2010-Apr-27 17:52 UTC
Re: [Puppet Users] Puppetmaster child processes hang
BTW, I use the defaults that came with apache for server/worker params. ~Charles~ On Tue, Apr 27, 2010 at 12:51 PM, Charles Johnson <gm.johns276@gmail.com>wrote:> Mine do not look too dissimilar: > > PassengerUseGlobalQueue on > PassengerMaxPoolSize 20 > PassengerHighPerformance on > PassengerPoolIdleTime 1800 > PassengerStatThrottleRate 120 > RackAutoDetect Off > RailsAutoDetect Off > > 1, Try using the global queue > > 2. Try dropping you pool idle time, which should force turnover/restarts of > puppet. (300 is the default, IIRC). > > 3. Are you using 0.25.X > > 4. On one of your nodes try puppetd --debug and see if there are telling > details there. > > I am running CentOS 5.x (RedHat essentially) with ruby, passenger and and > puppet installed by hand in /usr/local rather than using a package system. > Ruby 1.8.7, passenger 2.2.11. Apache is the "packaged" version installed > with yum. 2.2.3 > > I assume that you are using the ports collection to get your installs done? > > Cheers-- > > Charles > > On Tue, Apr 27, 2010 at 10:56 AM, S H <shdashbeta@gmail.com> wrote: > >> I also just discovered that some of my clients are locking up and >> requiring a SIGKILL to end them. This could be the root cause of the >> puppetmaster issues, so I''m going to run the clients as cron jobs for a >> while and see if that changes anything. >> >> >> -SH >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Apr 27, 2010, at 10:51 AM, Charles Johnson wrote:> Mine do not look too dissimilar: > > PassengerUseGlobalQueue on > PassengerMaxPoolSize 20 > PassengerHighPerformance on > PassengerPoolIdleTime 1800 > PassengerStatThrottleRate 120 > RackAutoDetect Off > RailsAutoDetect Off > > 1, Try using the global queue > > 2. Try dropping you pool idle time, which should force turnover/restarts of puppet. (300 is the default, IIRC). > > 3. Are you using 0.25.X > > 4. On one of your nodes try puppetd --debug and see if there are telling details there. > > I am running CentOS 5.x (RedHat essentially) with ruby, passenger and and puppet installed by hand in /usr/local rather than using a package system. Ruby 1.8.7, passenger 2.2.11. Apache is the "packaged" version installed with yum. 2.2.3 > > I assume that you are using the ports collection to get your installs done? >I second all that. Also, did you try sending a SIGTERM to the clients and waiting a couple minutes? Sometimes puppetd can take a very long time to stop.> > On Tue, Apr 27, 2010 at 10:56 AM, S H <shdashbeta@gmail.com> wrote: > I also just discovered that some of my clients are locking up and requiring a SIGKILL to end them. This could be the root cause of the puppetmaster issues, so I''m going to run the clients as cron jobs for a while and see if that changes anything. > > > -SH > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
On Tue, Apr 27, 2010 at 1:57 PM, Patrick <kc7zzv@gmail.com> wrote:> > On Apr 27, 2010, at 10:51 AM, Charles Johnson wrote: > > Mine do not look too dissimilar: > > PassengerUseGlobalQueue on > PassengerMaxPoolSize 20 > PassengerHighPerformance on > PassengerPoolIdleTime 1800 > PassengerStatThrottleRate 120 > RackAutoDetect Off > RailsAutoDetect Off > > 1, Try using the global queue > > 2. Try dropping you pool idle time, which should force turnover/restarts of > puppet. (300 is the default, IIRC). > > I''ll try these suggestions after getting --debug output from a hung client.> 3. Are you using 0.25.X > > Yes. The puppetmaster is 0.25.4 and the client is 0.25.3.> > 4. On one of your nodes try puppetd --debug and see if there are telling > details there. > > Done. We''ll see what comes up.> I am running CentOS 5.x (RedHat essentially) with ruby, passenger and and > puppet installed by hand in /usr/local rather than using a package system. > Ruby 1.8.7, passenger 2.2.11. Apache is the "packaged" version installed > with yum. 2.2.3 > > I assume that you are using the ports collection to get your installs done > > With the exception of the puppetmaster''s Puppet install, yes.Puppetmaster - FreeBSD 8.0 i386 with Ruby 1.8.7.248,1, Passenger 2.2.9, Apache 2.2.14_5 all from packages and Puppet 0.25.4 from gems. Puppetclient - FreeBSD 8.0 x64 with Ruby 1.8.7.248,1 and Puppet 0.25.3 from packages.> I second all that. Also, did you try sending a SIGTERM to the clients and > waiting a couple minutes? Sometimes puppetd can take a very long time to > stop. >A cronned client hung last night and hadn''t stopped even three hours after I sent it a SIGTERM. kill -9 seems to be the only. -SH -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.