Mark Christian
2010-Mar-07 19:46 UTC
[Puppet Users] Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
I have updated the 1100 clients that prior to the update used to connect to the master just fine. Now after about 30 minutes of running, puppetmasterd starts to complain "puppetmasterd[6875]: Too many connections", and from that point forward will stop compiling catalogs. Puppetmaster is configured using the Mongrel Apache configuration as described here: http://reductivelabs.com/trac/puppet/wiki/UsingMongrel In addition I''ve disabled all modules except for the "puppet" module which helped push the client upgrade as well as configure the "runinterval" from the pre-upgrade 1 hour run to a 4 hour run interval. At this point there might be a couple dozen 24.8 clients still attempting to connect. I guess I''m wondering if I need to scrap the Mongrel config with 5 "Mongrel instances" that used to handle 1100+ clients all which had significant configurations, or if someone has an idea outside of implementing more puppetmasters. Thank you for any insight. Mark -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Ohad Levy
2010-Mar-08 01:47 UTC
Re: [Puppet Users] Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
Hi Mark, Are you using storeconfigs? I''ve seen the too many connections error in relationship to activerecord versions (and the lack of mysql gem existence). In general, I would expect the load on your mongrels to be lower than 0.24.x cheers, Ohad On Mon, Mar 8, 2010 at 3:46 AM, Mark Christian <supertrick@gmail.com> wrote:> I have updated the 1100 clients that prior to the update used to > connect to the master just fine. Now after about 30 minutes of > running, puppetmasterd starts to complain "puppetmasterd[6875]: Too > many connections", and from that point forward will stop compiling > catalogs. Puppetmaster is configured using the Mongrel Apache > configuration as described here: > http://reductivelabs.com/trac/puppet/wiki/UsingMongrel > > In addition I''ve disabled all modules except for the "puppet" module > which helped push the client upgrade as well as configure the > "runinterval" from the pre-upgrade 1 hour run to a 4 hour run > interval. At this point there might be a couple dozen 24.8 clients > still attempting to connect. > > I guess I''m wondering if I need to scrap the Mongrel config with 5 > "Mongrel instances" that used to handle 1100+ clients all which had > significant configurations, or if someone has an idea outside of > implementing more puppetmasters. > > Thank you for any insight. > > Mark > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Mark Christian
2010-Mar-08 02:07 UTC
[Puppet Users] Re: Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
Thanks for the suggestion. I''ve set storeconfigs = false, it''s been a couple hours and I''ve yet to see any "too many connections" errors. On Mar 7, 5:47 pm, Ohad Levy <ohadl...@gmail.com> wrote:> Hi Mark, > > Are you using storeconfigs? I''ve seen the too many connections error in > relationship to activerecord versions (and the lack of mysql gem existence). > > In general, I would expect the load on your mongrels to be lower than > 0.24.x > cheers, > Ohad > > On Mon, Mar 8, 2010 at 3:46 AM, Mark Christian <supertr...@gmail.com> wrote: > > I have updated the 1100 clients that prior to the update used to > > connect to the master just fine. Now after about 30 minutes of > > running, puppetmasterd starts to complain "puppetmasterd[6875]: Too > > many connections", and from that point forward will stop compiling > > catalogs. Puppetmaster is configured using the Mongrel Apache > > configuration as described here: > >http://reductivelabs.com/trac/puppet/wiki/UsingMongrel > > > In addition I''ve disabled all modules except for the "puppet" module > > which helped push the client upgrade as well as configure the > > "runinterval" from the pre-upgrade 1 hour run to a 4 hour run > > interval. At this point there might be a couple dozen 24.8 clients > > still attempting to connect. > > > I guess I''m wondering if I need to scrap the Mongrel config with 5 > > "Mongrel instances" that used to handle 1100+ clients all which had > > significant configurations, or if someone has an idea outside of > > implementing more puppetmasters. > > > Thank you for any insight. > > > Mark > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Matt
2010-Mar-08 09:55 UTC
Re: [Puppet Users] Re: Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
Out of interest Mark, how did you go about disabling all modules bar puppet? Matt On 8 March 2010 02:07, Mark Christian <supertrick@gmail.com> wrote:> Thanks for the suggestion. I''ve set storeconfigs = false, it''s been a > couple hours and I''ve yet to see any "too many connections" errors. > > On Mar 7, 5:47 pm, Ohad Levy <ohadl...@gmail.com> wrote: > > Hi Mark, > > > > Are you using storeconfigs? I''ve seen the too many connections error in > > relationship to activerecord versions (and the lack of mysql gem > existence). > > > > In general, I would expect the load on your mongrels to be lower than > > 0.24.x > > cheers, > > Ohad > > > > On Mon, Mar 8, 2010 at 3:46 AM, Mark Christian <supertr...@gmail.com> > wrote: > > > I have updated the 1100 clients that prior to the update used to > > > connect to the master just fine. Now after about 30 minutes of > > > running, puppetmasterd starts to complain "puppetmasterd[6875]: Too > > > many connections", and from that point forward will stop compiling > > > catalogs. Puppetmaster is configured using the Mongrel Apache > > > configuration as described here: > > >http://reductivelabs.com/trac/puppet/wiki/UsingMongrel > > > > > In addition I''ve disabled all modules except for the "puppet" module > > > which helped push the client upgrade as well as configure the > > > "runinterval" from the pre-upgrade 1 hour run to a 4 hour run > > > interval. At this point there might be a couple dozen 24.8 clients > > > still attempting to connect. > > > > > I guess I''m wondering if I need to scrap the Mongrel config with 5 > > > "Mongrel instances" that used to handle 1100+ clients all which had > > > significant configurations, or if someone has an idea outside of > > > implementing more puppetmasters. > > > > > Thank you for any insight. > > > > > Mark > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Puppet Users" group. > > > To post to this group, send email to puppet-users@googlegroups.com. > > > To unsubscribe from this group, send email to > > > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > <puppet-users%2Bunsubscribe@googlegroups.com<puppet-users%252Bunsubscribe@googlegroups.com> > > > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/puppet-users?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Mark Christian
2010-Mar-08 20:20 UTC
[Puppet Users] Re: Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
As far as disabling modules is concerned. The way I''ve organized my
manifests allows me to quickly comment out the various "include
someclass" entries per system role or hostgroup. The following works
for my environment and is not necessarily considered best practice.
For example, I might have a "generic role" that has numerous
"include
someclass" entries like so:
node generic {
include autofs
include puppet
include crontab
include ntp
...
}
a typical node might look like:
node ''typical-node'' inherits generic {
}
The above configuration allows me to quickly comment out modules for
my nodes.
Mark
On Mar 8, 1:55 am, Matt <mattmora...@gmail.com>
wrote:> Out of interest Mark, how did you go about disabling all modules bar
puppet?
>
> Matt
>
> On 8 March 2010 02:07, Mark Christian <supertr...@gmail.com> wrote:
>
> > Thanks for the suggestion. I''ve set storeconfigs = false,
it''s been a
> > couple hours and I''ve yet to see any "too many
connections" errors.
>
> > On Mar 7, 5:47 pm, Ohad Levy <ohadl...@gmail.com> wrote:
> > > Hi Mark,
>
> > > Are you using storeconfigs? I''ve seen the too many
connections error in
> > > relationship to activerecord versions (and the lack of mysql gem
> > existence).
>
> > > In general, I would expect the load on your mongrels to be lower
than
> > > 0.24.x
> > > cheers,
> > > Ohad
>
> > > On Mon, Mar 8, 2010 at 3:46 AM, Mark Christian
<supertr...@gmail.com>
> > wrote:
> > > > I have updated the 1100 clients that prior to the update
used to
> > > > connect to the master just fine. Now after about 30 minutes
of
> > > > running, puppetmasterd starts to complain
"puppetmasterd[6875]: Too
> > > > many connections", and from that point forward will
stop compiling
> > > > catalogs. Puppetmaster is configured using the Mongrel
Apache
> > > > configuration as described here:
> > > >http://reductivelabs.com/trac/puppet/wiki/UsingMongrel
>
> > > > In addition I''ve disabled all modules except for
the "puppet" module
> > > > which helped push the client upgrade as well as configure
the
> > > > "runinterval" from the pre-upgrade 1 hour run to a
4 hour run
> > > > interval. At this point there might be a couple dozen 24.8
clients
> > > > still attempting to connect.
>
> > > > I guess I''m wondering if I need to scrap the
Mongrel config with 5
> > > > "Mongrel instances" that used to handle 1100+
clients all which had
> > > > significant configurations, or if someone has an idea
outside of
> > > > implementing more puppetmasters.
>
> > > > Thank you for any insight.
>
> > > > Mark
>
> > > > --
> > > > You received this message because you are subscribed to the
Google
> > Groups
> > > > "Puppet Users" group.
> > > > To post to this group, send email to
puppet-users@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> >
<puppet-users%2Bunsubscribe@googlegroups.com<puppet-users%252Bunsubscribe@googlegroups.com>
>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/puppet-users?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google
Groups
> > "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/puppet-users?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Simon Mügge
2010-Mar-09 10:21 UTC
[Puppet Users] Re: Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
Hey Ohad!
Could you please elaborate on these "faulty" activerecord versions?
I ran into the same problem (just with fewer (180) clients as Im not
using mongrel yet), getting these errors on the clients: "Could not
retrieve catalog from remote server: Error 400 on SERVER: Too many
connections"
I am running:
0.25.4 master/client,
storeconfigs with local mysql5 (testing stage, it will move to a
seperate machine.. ;) ),
activrecord deb and
mysql gem:
activerecord:
puppet-01:~# aptitude search active | grep ^i
i libactiverecord-ruby - Ruby library that ties database
tables to
i A libactiverecord-ruby1.8 - Tie database tables to classes
(Ruby 1.8)
i A libactivesupport-ruby - utility classes and extensions
to the Ruby
i A libactivesupport-ruby1.8 - utility classes and extensions
(Ruby 1.8)
mysql:
puppet-01:~# gem list mysql
*** LOCAL GEMS ***
mysql (2.8.1)
puppet-01:~# aptitude search mysql | grep ^i
i A libdbd-mysql-perl - A Perl5 database interface to
the MySQL da
i libdbd-mysql-ruby1.8 - Ruby/DBI MySQL driver for Ruby
1.8
i libmysql-ruby1.8 - MySQL module for Ruby
1.8
i libmysqlclient15-dev - MySQL database development
files
i A libmysqlclient15off - MySQL database client
library
i A mysql-client-5.0 - MySQL database client
binaries
i A mysql-common - MySQL database common
files
i mysql-server-5.0 - MySQL database server binaries
lsof of puppetmaster looks like this:
puppet-01:/etc/puppet# lsof -p 1829 | grep -v sock
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
puppetmas 1829 puppet cwd DIR 104,5 4096 1753254 /home/
simu
puppetmas 1829 puppet rtd DIR 104,5 4096 2 /
puppetmas 1829 puppet txt REG 104,5 3608 503705 /usr/
bin/ruby1.8
puppetmas 1829 puppet mem REG 104,5 1995676 504547 /usr/
lib/libmysqlclient.so.15.0.0
puppetmas 1829 puppet mem REG 104,5 77244 542449 /usr/
lib/ruby/1.8/i486-linux/mysql.so
puppetmas 1829 puppet mem REG 104,5 67408 352284 /lib/
i686/cmov/libresolv-2.7.so
puppetmas 1829 puppet mem REG 104,5 17880 352282 /lib/
i686/cmov/libnss_dns-2.7.so
puppetmas 1829 puppet mem REG 104,5 42504 352285 /lib/
i686/cmov/libnss_files-2.7.so
puppetmas 1829 puppet mem REG 104,5 38444 352272 /lib/
i686/cmov/libnss_nis-2.7.so
puppetmas 1829 puppet mem REG 104,5 87800 352273 /lib/
i686/cmov/libnsl-2.7.so
puppetmas 1829 puppet mem REG 104,5 30436 352277 /lib/
i686/cmov/libnss_compat-2.7.so
puppetmas 1829 puppet mem REG 104,5 11904 543078 /usr/
lib/ruby/1.8/i486-linux/digest/sha1.so
puppetmas 1829 puppet mem REG 104,5 7512 542262 /usr/
lib/ruby/1.8/i486-linux/shadow.so
puppetmas 1829 puppet mem REG 104,5 6812 543082 /usr/
lib/ruby/1.8/i486-linux/digest/md5.so
puppetmas 1829 puppet mem REG 104,5 1375588 516954 /usr/
lib/i686/cmov/libcrypto.so.0.9.8
puppetmas 1829 puppet mem REG 104,5 285188 516953 /usr/
lib/i686/cmov/libssl.so.0.9.8
puppetmas 1829 puppet mem REG 104,5 10260 543075 /usr/
lib/ruby/1.8/i486-linux/digest.so
puppetmas 1829 puppet mem REG 104,5 265768 541618 /usr/
lib/ruby/1.8/i486-linux/openssl.so
puppetmas 1829 puppet mem REG 104,5 12044 543089 /usr/
lib/ruby/1.8/i486-linux/racc/cparse.so
puppetmas 1829 puppet mem REG 104,5 38588 543086 /usr/
lib/ruby/1.8/i486-linux/bigdecimal.so
puppetmas 1829 puppet mem REG 104,5 9484 504995 /usr/
lib/gconv/UTF-16.so
puppetmas 1829 puppet mem REG 104,5 25700 500846 /usr/
lib/gconv/gconv-modules.cache
puppetmas 1829 puppet mem REG 104,5 13384 543065 /usr/
lib/ruby/1.8/i486-linux/iconv.so
puppetmas 1829 puppet mem REG 104,5 15952 543074 /usr/
lib/ruby/1.8/i486-linux/strscan.so
puppetmas 1829 puppet mem REG 104,5 239804 543067 /usr/
lib/ruby/1.8/i486-linux/nkf.so
puppetmas 1829 puppet mem REG 104,5 7904 543083 /usr/
lib/ruby/1.8/i486-linux/etc.so
puppetmas 1829 puppet mem REG 104,5 109360 543088 /usr/
lib/ruby/1.8/i486-linux/syck.so
puppetmas 1829 puppet mem REG 104,5 10084 543070 /usr/
lib/ruby/1.8/i486-linux/syslog.so
puppetmas 1829 puppet mem REG 104,5 81012 501766 /usr/
lib/libz.so.1.2.3.3
puppetmas 1829 puppet mem REG 104,5 16944 543076 /usr/
lib/ruby/1.8/i486-linux/stringio.so
puppetmas 1829 puppet mem REG 104,5 32040 543084 /usr/
lib/ruby/1.8/i486-linux/zlib.so
puppetmas 1829 puppet mem REG 104,5 1413540 352280 /lib/
i686/cmov/libc-2.7.so
puppetmas 1829 puppet mem REG 104,5 149328 352260 /lib/
i686/cmov/libm-2.7.so
puppetmas 1829 puppet mem REG 104,5 38296 352265 /lib/
i686/cmov/libcrypt-2.7.so
puppetmas 1829 puppet mem REG 104,5 9680 352276 /lib/
i686/cmov/libdl-2.7.so
puppetmas 1829 puppet mem REG 104,5 116414 352271 /lib/
i686/cmov/libpthread-2.7.so
puppetmas 1829 puppet mem REG 104,5 851604 503703 /usr/
lib/libruby1.8.so.1.8.7
puppetmas 1829 puppet mem REG 104,5 3852 543071 /usr/
lib/ruby/1.8/i486-linux/fcntl.so
puppetmas 1829 puppet mem REG 104,5 12880 543087 /usr/
lib/ruby/1.8/i486-linux/thread.so
puppetmas 1829 puppet mem REG 104,5 113248 344415 /lib/
ld-2.7.so
puppetmas 1829 puppet 0u CHR 136,3 5 /dev/
pts/3
puppetmas 1829 puppet 1u CHR 136,3 5 /dev/
pts/3
puppetmas 1829 puppet 2u CHR 136,3 5 /dev/
pts/3
puppetmas 1829 puppet 3r FIFO 0,6 27024956 pipe
puppetmas 1829 puppet 4w FIFO 0,6 27024956 pipe
puppetmas 1829 puppet 5u REG 104,5 47688437 2016048 /var/
log/puppet/masterhttp.log
puppetmas 1829 puppet 6u IPv4 27024969 TCP *:8140
(LISTEN)
puppetmas 1829 puppet 8w REG 104,5 791 2015320 /var/
log/puppet/rails.log
and there are about 100 of these:
puppetmas 1829 puppet 107u unix 0xf78c7700 27034858 socket
And there is no mention of "You''re using the Ruby-based MySQL
library.. " in rails.log - just this:
"
Defined connections: {}
Active connections: {}
Active connection name:
"
Does anyone see what I am doing wrong? If so please tell me, I''d like
to keep storeconfigs.. :/
Any help is greatly appreciated!
Cheers,
Simon
On 8 Mrz., 02:47, Ohad Levy <ohadl...@gmail.com>
wrote:> Hi Mark,
>
> Are you using storeconfigs? I''ve seen the too many connections
error in
> relationship to activerecord versions (and the lack of mysql gem
existence).
>
> In general, I would expect the load on your mongrels to be lower than
> 0.24.x
> cheers,
> Ohad
>
> On Mon, Mar 8, 2010 at 3:46 AM, Mark Christian <supertr...@gmail.com>
wrote:
> > I have updated the 1100 clients that prior to the update used to
> > connect to the master just fine. Now after about 30 minutes of
> > running, puppetmasterd starts to complain "puppetmasterd[6875]:
Too
> > many connections", and from that point forward will stop
compiling
> > catalogs. Puppetmaster is configured using the Mongrel Apache
> > configuration as described here:
> >http://reductivelabs.com/trac/puppet/wiki/UsingMongrel
>
> > In addition I''ve disabled all modules except for the
"puppet" module
> > which helped push the client upgrade as well as configure the
> > "runinterval" from the pre-upgrade 1 hour run to a 4 hour
run
> > interval. At this point there might be a couple dozen 24.8 clients
> > still attempting to connect.
>
> > I guess I''m wondering if I need to scrap the Mongrel config
with 5
> > "Mongrel instances" that used to handle 1100+ clients all
which had
> > significant configurations, or if someone has an idea outside of
> > implementing more puppetmasters.
>
> > Thank you for any insight.
>
> > Mark
>
> > --
> > You received this message because you are subscribed to the Google
Groups
> > "Puppet Users" group.
> > To post to this group, send email to puppet-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> >
puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/puppet-users?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Ohad Levy
2010-Mar-10 05:43 UTC
Re: [Puppet Users] Re: Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
which version of activerecord is installed? you might want to give 2.3 a try and see if that solves your problems. Ohad On Tue, Mar 9, 2010 at 6:21 PM, Simon Mügge <simu@webde.de> wrote:> Hey Ohad! > > Could you please elaborate on these "faulty" activerecord versions? > > I ran into the same problem (just with fewer (180) clients as Im not > using mongrel yet), getting these errors on the clients: "Could not > retrieve catalog from remote server: Error 400 on SERVER: Too many > connections" > > I am running: > 0.25.4 master/client, > storeconfigs with local mysql5 (testing stage, it will move to a > seperate machine.. ;) ), > activrecord deb and > mysql gem: > > activerecord: > puppet-01:~# aptitude search active | grep ^i > i libactiverecord-ruby - Ruby library that ties database > tables to > i A libactiverecord-ruby1.8 - Tie database tables to classes > (Ruby 1.8) > i A libactivesupport-ruby - utility classes and extensions > to the Ruby > i A libactivesupport-ruby1.8 - utility classes and extensions > (Ruby 1.8) > > mysql: > puppet-01:~# gem list mysql > *** LOCAL GEMS *** > mysql (2.8.1) > puppet-01:~# aptitude search mysql | grep ^i > i A libdbd-mysql-perl - A Perl5 database interface to > the MySQL da > i libdbd-mysql-ruby1.8 - Ruby/DBI MySQL driver for Ruby > 1.8 > i libmysql-ruby1.8 - MySQL module for Ruby > 1.8 > i libmysqlclient15-dev - MySQL database development > files > i A libmysqlclient15off - MySQL database client > library > i A mysql-client-5.0 - MySQL database client > binaries > i A mysql-common - MySQL database common > files > i mysql-server-5.0 - MySQL database server binaries > > > > lsof of puppetmaster looks like this: > puppet-01:/etc/puppet# lsof -p 1829 | grep -v sock > COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME > puppetmas 1829 puppet cwd DIR 104,5 4096 1753254 /home/ > simu > puppetmas 1829 puppet rtd DIR 104,5 4096 2 / > puppetmas 1829 puppet txt REG 104,5 3608 503705 /usr/ > bin/ruby1.8 > puppetmas 1829 puppet mem REG 104,5 1995676 504547 /usr/ > lib/libmysqlclient.so.15.0.0 > puppetmas 1829 puppet mem REG 104,5 77244 542449 /usr/ > lib/ruby/1.8/i486-linux/mysql.so > puppetmas 1829 puppet mem REG 104,5 67408 352284 /lib/ > i686/cmov/libresolv-2.7.so > puppetmas 1829 puppet mem REG 104,5 17880 352282 /lib/ > i686/cmov/libnss_dns-2.7.so > puppetmas 1829 puppet mem REG 104,5 42504 352285 /lib/ > i686/cmov/libnss_files-2.7.so > puppetmas 1829 puppet mem REG 104,5 38444 352272 /lib/ > i686/cmov/libnss_nis-2.7.so > puppetmas 1829 puppet mem REG 104,5 87800 352273 /lib/ > i686/cmov/libnsl-2.7.so > puppetmas 1829 puppet mem REG 104,5 30436 352277 /lib/ > i686/cmov/libnss_compat-2.7.so > puppetmas 1829 puppet mem REG 104,5 11904 543078 /usr/ > lib/ruby/1.8/i486-linux/digest/sha1.so > puppetmas 1829 puppet mem REG 104,5 7512 542262 /usr/ > lib/ruby/1.8/i486-linux/shadow.so > puppetmas 1829 puppet mem REG 104,5 6812 543082 /usr/ > lib/ruby/1.8/i486-linux/digest/md5.so > puppetmas 1829 puppet mem REG 104,5 1375588 516954 /usr/ > lib/i686/cmov/libcrypto.so.0.9.8 > puppetmas 1829 puppet mem REG 104,5 285188 516953 /usr/ > lib/i686/cmov/libssl.so.0.9.8 > puppetmas 1829 puppet mem REG 104,5 10260 543075 /usr/ > lib/ruby/1.8/i486-linux/digest.so > puppetmas 1829 puppet mem REG 104,5 265768 541618 /usr/ > lib/ruby/1.8/i486-linux/openssl.so > puppetmas 1829 puppet mem REG 104,5 12044 543089 /usr/ > lib/ruby/1.8/i486-linux/racc/cparse.so > puppetmas 1829 puppet mem REG 104,5 38588 543086 /usr/ > lib/ruby/1.8/i486-linux/bigdecimal.so > puppetmas 1829 puppet mem REG 104,5 9484 504995 /usr/ > lib/gconv/UTF-16.so > puppetmas 1829 puppet mem REG 104,5 25700 500846 /usr/ > lib/gconv/gconv-modules.cache > puppetmas 1829 puppet mem REG 104,5 13384 543065 /usr/ > lib/ruby/1.8/i486-linux/iconv.so > puppetmas 1829 puppet mem REG 104,5 15952 543074 /usr/ > lib/ruby/1.8/i486-linux/strscan.so > puppetmas 1829 puppet mem REG 104,5 239804 543067 /usr/ > lib/ruby/1.8/i486-linux/nkf.so > puppetmas 1829 puppet mem REG 104,5 7904 543083 /usr/ > lib/ruby/1.8/i486-linux/etc.so > puppetmas 1829 puppet mem REG 104,5 109360 543088 /usr/ > lib/ruby/1.8/i486-linux/syck.so > puppetmas 1829 puppet mem REG 104,5 10084 543070 /usr/ > lib/ruby/1.8/i486-linux/syslog.so > puppetmas 1829 puppet mem REG 104,5 81012 501766 /usr/ > lib/libz.so.1.2.3.3 > puppetmas 1829 puppet mem REG 104,5 16944 543076 /usr/ > lib/ruby/1.8/i486-linux/stringio.so > puppetmas 1829 puppet mem REG 104,5 32040 543084 /usr/ > lib/ruby/1.8/i486-linux/zlib.so > puppetmas 1829 puppet mem REG 104,5 1413540 352280 /lib/ > i686/cmov/libc-2.7.so > puppetmas 1829 puppet mem REG 104,5 149328 352260 /lib/ > i686/cmov/libm-2.7.so > puppetmas 1829 puppet mem REG 104,5 38296 352265 /lib/ > i686/cmov/libcrypt-2.7.so > puppetmas 1829 puppet mem REG 104,5 9680 352276 /lib/ > i686/cmov/libdl-2.7.so > puppetmas 1829 puppet mem REG 104,5 116414 352271 /lib/ > i686/cmov/libpthread-2.7.so > puppetmas 1829 puppet mem REG 104,5 851604 503703 /usr/ > lib/libruby1.8.so.1.8.7 > puppetmas 1829 puppet mem REG 104,5 3852 543071 /usr/ > lib/ruby/1.8/i486-linux/fcntl.so > puppetmas 1829 puppet mem REG 104,5 12880 543087 /usr/ > lib/ruby/1.8/i486-linux/thread.so > puppetmas 1829 puppet mem REG 104,5 113248 344415 /lib/ > ld-2.7.so > puppetmas 1829 puppet 0u CHR 136,3 5 /dev/ > pts/3 > puppetmas 1829 puppet 1u CHR 136,3 5 /dev/ > pts/3 > puppetmas 1829 puppet 2u CHR 136,3 5 /dev/ > pts/3 > puppetmas 1829 puppet 3r FIFO 0,6 27024956 pipe > puppetmas 1829 puppet 4w FIFO 0,6 27024956 pipe > puppetmas 1829 puppet 5u REG 104,5 47688437 2016048 /var/ > log/puppet/masterhttp.log > puppetmas 1829 puppet 6u IPv4 27024969 TCP *:8140 > (LISTEN) > puppetmas 1829 puppet 8w REG 104,5 791 2015320 /var/ > log/puppet/rails.log > and there are about 100 of these: > puppetmas 1829 puppet 107u unix 0xf78c7700 27034858 socket > > And there is no mention of "You''re using the Ruby-based MySQL > library.. " in rails.log - just this: > " > Defined connections: {} > Active connections: {} > Active connection name: > " > > > > Does anyone see what I am doing wrong? If so please tell me, I''d like > to keep storeconfigs.. :/ > Any help is greatly appreciated! > > > Cheers, > Simon > > > > On 8 Mrz., 02:47, Ohad Levy <ohadl...@gmail.com> wrote: > > Hi Mark, > > > > Are you using storeconfigs? I''ve seen the too many connections error in > > relationship to activerecord versions (and the lack of mysql gem > existence). > > > > In general, I would expect the load on your mongrels to be lower than > > 0.24.x > > cheers, > > Ohad > > > > On Mon, Mar 8, 2010 at 3:46 AM, Mark Christian <supertr...@gmail.com> > wrote: > > > I have updated the 1100 clients that prior to the update used to > > > connect to the master just fine. Now after about 30 minutes of > > > running, puppetmasterd starts to complain "puppetmasterd[6875]: Too > > > many connections", and from that point forward will stop compiling > > > catalogs. Puppetmaster is configured using the Mongrel Apache > > > configuration as described here: > > >http://reductivelabs.com/trac/puppet/wiki/UsingMongrel > > > > > In addition I''ve disabled all modules except for the "puppet" module > > > which helped push the client upgrade as well as configure the > > > "runinterval" from the pre-upgrade 1 hour run to a 4 hour run > > > interval. At this point there might be a couple dozen 24.8 clients > > > still attempting to connect. > > > > > I guess I''m wondering if I need to scrap the Mongrel config with 5 > > > "Mongrel instances" that used to handle 1100+ clients all which had > > > significant configurations, or if someone has an idea outside of > > > implementing more puppetmasters. > > > > > Thank you for any insight. > > > > > Mark > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Puppet Users" group. > > > To post to this group, send email to puppet-users@googlegroups.com. > > > To unsubscribe from this group, send email to > > > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > <puppet-users%2Bunsubscribe@googlegroups.com<puppet-users%252Bunsubscribe@googlegroups.com> > > > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/puppet-users?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Simon Mügge
2010-Mar-10 09:34 UTC
[Puppet Users] Re: Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
Ohad, thanks for your reply! Last night ive had it and switched to gems "only" and had no problems at all with "too many connections" using storeconfigs The debian (lenny) packages where seriously outdated and now this is what my gem list looks like: ~# gem list *** LOCAL GEMS *** actionmailer (2.3.5) actionpack (2.3.5) activerecord (2.3.5) activeresource (2.3.5) activesupport (2.3.5) gem_plugin (0.2.3) rack (1.0.1) ##"gem install rack -v 1.0.1" ## - rails does not like latest avail rack gem rails (2.3.5) rake (0.8.7) ruby-mysql (2.9.2) test-spec (0.10.0) Aaand the debianpackages: libmysql-ruby1.8 libmysqlclient15-dev That did it, it works, Im happy - thanks a lot! :) Cheers, Simon On 10 Mrz., 06:43, Ohad Levy <ohadl...@gmail.com> wrote:> which version of activerecord is installed? you might want to give 2.3 a try > and see if that solves your problems. > > Ohad > > On Tue, Mar 9, 2010 at 6:21 PM, Simon Mügge <s...@webde.de> wrote: > > Hey Ohad! > > > Could you please elaborate on these "faulty" activerecord versions? > > > I ran into the same problem (just with fewer (180) clients as Im not > > using mongrel yet), getting these errors on the clients: "Could not > > retrieve catalog from remote server: Error 400 on SERVER: Too many > > connections" > > > I am running: > > 0.25.4 master/client, > > storeconfigs with local mysql5 (testing stage, it will move to a > > seperate machine.. ;) ), > > activrecord deb and > > mysql gem: > > > activerecord: > > puppet-01:~# aptitude search active | grep ^i > > i libactiverecord-ruby - Ruby library that ties database > > tables to > > i A libactiverecord-ruby1.8 - Tie database tables to classes > > (Ruby 1.8) > > i A libactivesupport-ruby - utility classes and extensions > > to the Ruby > > i A libactivesupport-ruby1.8 - utility classes and extensions > > (Ruby 1.8) > > > mysql: > > puppet-01:~# gem list mysql > > *** LOCAL GEMS *** > > mysql (2.8.1) > > puppet-01:~# aptitude search mysql | grep ^i > > i A libdbd-mysql-perl - A Perl5 database interface to > > the MySQL da > > i libdbd-mysql-ruby1.8 - Ruby/DBI MySQL driver for Ruby > > 1.8 > > i libmysql-ruby1.8 - MySQL module for Ruby > > 1.8 > > i libmysqlclient15-dev - MySQL database development > > files > > i A libmysqlclient15off - MySQL database client > > library > > i A mysql-client-5.0 - MySQL database client > > binaries > > i A mysql-common - MySQL database common > > files > > i mysql-server-5.0 - MySQL database server binaries > > > lsof of puppetmaster looks like this: > > puppet-01:/etc/puppet# lsof -p 1829 | grep -v sock > > COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME > > puppetmas 1829 puppet cwd DIR 104,5 4096 1753254 /home/ > > simu > > puppetmas 1829 puppet rtd DIR 104,5 4096 2 / > > puppetmas 1829 puppet txt REG 104,5 3608 503705 /usr/ > > bin/ruby1.8 > > puppetmas 1829 puppet mem REG 104,5 1995676 504547 /usr/ > > lib/libmysqlclient.so.15.0.0 > > puppetmas 1829 puppet mem REG 104,5 77244 542449 /usr/ > > lib/ruby/1.8/i486-linux/mysql.so > > puppetmas 1829 puppet mem REG 104,5 67408 352284 /lib/ > > i686/cmov/libresolv-2.7.so > > puppetmas 1829 puppet mem REG 104,5 17880 352282 /lib/ > > i686/cmov/libnss_dns-2.7.so > > puppetmas 1829 puppet mem REG 104,5 42504 352285 /lib/ > > i686/cmov/libnss_files-2.7.so > > puppetmas 1829 puppet mem REG 104,5 38444 352272 /lib/ > > i686/cmov/libnss_nis-2.7.so > > puppetmas 1829 puppet mem REG 104,5 87800 352273 /lib/ > > i686/cmov/libnsl-2.7.so > > puppetmas 1829 puppet mem REG 104,5 30436 352277 /lib/ > > i686/cmov/libnss_compat-2.7.so > > puppetmas 1829 puppet mem REG 104,5 11904 543078 /usr/ > > lib/ruby/1.8/i486-linux/digest/sha1.so > > puppetmas 1829 puppet mem REG 104,5 7512 542262 /usr/ > > lib/ruby/1.8/i486-linux/shadow.so > > puppetmas 1829 puppet mem REG 104,5 6812 543082 /usr/ > > lib/ruby/1.8/i486-linux/digest/md5.so > > puppetmas 1829 puppet mem REG 104,5 1375588 516954 /usr/ > > lib/i686/cmov/libcrypto.so.0.9.8 > > puppetmas 1829 puppet mem REG 104,5 285188 516953 /usr/ > > lib/i686/cmov/libssl.so.0.9.8 > > puppetmas 1829 puppet mem REG 104,5 10260 543075 /usr/ > > lib/ruby/1.8/i486-linux/digest.so > > puppetmas 1829 puppet mem REG 104,5 265768 541618 /usr/ > > lib/ruby/1.8/i486-linux/openssl.so > > puppetmas 1829 puppet mem REG 104,5 12044 543089 /usr/ > > lib/ruby/1.8/i486-linux/racc/cparse.so > > puppetmas 1829 puppet mem REG 104,5 38588 543086 /usr/ > > lib/ruby/1.8/i486-linux/bigdecimal.so > > puppetmas 1829 puppet mem REG 104,5 9484 504995 /usr/ > > lib/gconv/UTF-16.so > > puppetmas 1829 puppet mem REG 104,5 25700 500846 /usr/ > > lib/gconv/gconv-modules.cache > > puppetmas 1829 puppet mem REG 104,5 13384 543065 /usr/ > > lib/ruby/1.8/i486-linux/iconv.so > > puppetmas 1829 puppet mem REG 104,5 15952 543074 /usr/ > > lib/ruby/1.8/i486-linux/strscan.so > > puppetmas 1829 puppet mem REG 104,5 239804 543067 /usr/ > > lib/ruby/1.8/i486-linux/nkf.so > > puppetmas 1829 puppet mem REG 104,5 7904 543083 /usr/ > > lib/ruby/1.8/i486-linux/etc.so > > puppetmas 1829 puppet mem REG 104,5 109360 543088 /usr/ > > lib/ruby/1.8/i486-linux/syck.so > > puppetmas 1829 puppet mem REG 104,5 10084 543070 /usr/ > > lib/ruby/1.8/i486-linux/syslog.so > > puppetmas 1829 puppet mem REG 104,5 81012 501766 /usr/ > > lib/libz.so.1.2.3.3 > > puppetmas 1829 puppet mem REG 104,5 16944 543076 /usr/ > > lib/ruby/1.8/i486-linux/stringio.so > > puppetmas 1829 puppet mem REG 104,5 32040 543084 /usr/ > > lib/ruby/1.8/i486-linux/zlib.so > > puppetmas 1829 puppet mem REG 104,5 1413540 352280 /lib/ > > i686/cmov/libc-2.7.so > > puppetmas 1829 puppet mem REG 104,5 149328 352260 /lib/ > > i686/cmov/libm-2.7.so > > puppetmas 1829 puppet mem REG 104,5 38296 352265 /lib/ > > i686/cmov/libcrypt-2.7.so > > puppetmas 1829 puppet mem REG 104,5 9680 352276 /lib/ > > i686/cmov/libdl-2.7.so > > puppetmas 1829 puppet mem REG 104,5 116414 352271 /lib/ > > i686/cmov/libpthread-2.7.so > > puppetmas 1829 puppet mem REG 104,5 851604 503703 /usr/ > > lib/libruby1.8.so.1.8.7 > > puppetmas 1829 puppet mem REG 104,5 3852 543071 /usr/ > > lib/ruby/1.8/i486-linux/fcntl.so > > puppetmas 1829 puppet mem REG 104,5 12880 543087 /usr/ > > lib/ruby/1.8/i486-linux/thread.so > > puppetmas 1829 puppet mem REG 104,5 113248 344415 /lib/ > > ld-2.7.so > > puppetmas 1829 puppet 0u CHR 136,3 5 /dev/ > > pts/3 > > puppetmas 1829 puppet 1u CHR 136,3 5 /dev/ > > pts/3 > > puppetmas 1829 puppet 2u CHR 136,3 5 /dev/ > > pts/3 > > puppetmas 1829 puppet 3r FIFO 0,6 27024956 pipe > > puppetmas 1829 puppet 4w FIFO 0,6 27024956 pipe > > puppetmas 1829 puppet 5u REG 104,5 47688437 2016048 /var/ > > log/puppet/masterhttp.log > > puppetmas 1829 puppet 6u IPv4 27024969 TCP *:8140 > > (LISTEN) > > puppetmas 1829 puppet 8w REG 104,5 791 2015320 /var/ > > log/puppet/rails.log > > and there are about 100 of these: > > puppetmas 1829 puppet 107u unix 0xf78c7700 27034858 socket > > > And there is no mention of "You''re using the Ruby-based MySQL > > library.. " in rails.log - just this: > > " > > Defined connections: {} > > Active connections: {} > > Active connection name: > > " > > > Does anyone see what I am doing wrong? If so please tell me, I''d like > > to keep storeconfigs.. :/ > > Any help is greatly appreciated! > > > Cheers, > > Simon > > > On 8 Mrz., 02:47, Ohad Levy <ohadl...@gmail.com> wrote: > > > Hi Mark, > > > > Are you using storeconfigs? I''ve seen the too many connections error in > > > relationship to activerecord versions (and the lack of mysql gem > > existence). > > > > In general, I would expect the load on your mongrels to be lower than > > > 0.24.x > > > cheers, > > > Ohad > > > > On Mon, Mar 8, 2010 at 3:46 AM, Mark Christian <supertr...@gmail.com> > > wrote: > > > > I have updated the 1100 clients that prior to the update used to > > > > connect to the master just fine. Now after about 30 minutes of > > > > running, puppetmasterd starts to complain "puppetmasterd[6875]: Too > > > > many connections", and from that point forward will stop compiling > > > > catalogs. Puppetmaster is configured using the Mongrel Apache > > > > configuration as described here: > > > >http://reductivelabs.com/trac/puppet/wiki/UsingMongrel > > > > > In addition I''ve disabled all modules except for the "puppet" module > > > > which helped push the client upgrade as well as configure the > > > > "runinterval" from the pre-upgrade 1 hour run to a 4 hour run > > > > interval. At this point there might be a couple dozen 24.8 clients > > > > still attempting to connect. > > > > > I guess I''m wondering if I need to scrap the Mongrel config with 5 > > > > "Mongrel instances" that used to handle 1100+ clients all which had > > > > significant configurations, or if someone has an idea outside of > > > > implementing more puppetmasters. > > > > > Thank you for any insight. > > > > > Mark > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Puppet Users" group. > > > > To post to this group, send email to puppet-users@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > > <puppet-users%2Bunsubscribe@googlegroups.com<puppet-users%252Bunsubscribe@googlegroups.com> > > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/puppet-users?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscribe@googlegroups.com<puppet-users%2Bunsubscribe@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/puppet-users?hl=en.-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.