Colm O''Connor
2010-Sep-02 15:54 UTC
[Puppet Users] err: Too many open files - /var/lib/puppet/state/state.yaml.tmp
I keep on getting this message: err: Too many open files - /var/lib/puppet/state/state.yaml.tmp Does anybody know what might be causing it? I''m also getting "err: State got corrupted". Everything does seem to be kind of working, but it''s sort of worrying getting these new errors. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
James Turnbull
2010-Sep-02 17:21 UTC
Re: [Puppet Users] err: Too many open files - /var/lib/puppet/state/state.yaml.tmp
Colm O''Connor wrote:> I keep on getting this message: > > err: Too many open files - /var/lib/puppet/state/state.yaml.tmp > > Does anybody know what might be causing it? > > I''m also getting "err: State got corrupted". Everything does seem to > be kind of working, but it''s sort of worrying getting these new errors. >What version? What platform? Regards James Turnbull -- Puppet Labs - http://www.puppetlabs.com C: 503-734-8571 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Héctor Rivas Gándara
2010-Sep-03 11:29 UTC
Re: [Puppet Users] err: Too many open files - /var/lib/puppet/state/state.yaml.tmp
>> I keep on getting this message: >> err: Too many open files - /var/lib/puppet/state/state.yaml.tmp >> Does anybody know what might be causing it? >>I am having the same problem. I think that it is a socket leak in puppetmaster (it does not properly closes sockets). To workaround the problem you can increase the ulimit for open files (/etc/security/limits.conf) and restart puppetmaster periodically (¿each 6h?). I am using puppet 2.6.1rc3 on Debian Lenny with ruby 1.8.7. I use mongrel+apache and storeconfigs+mysql I made it fail executing puppetd in a loop in two nodes to test another bug. lsof output is: # lsof +p 26100|grep socket | wc 1017 8137 77326 # lsof +p 26100 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME ruby 26100 puppet cwd DIR 254,0 4096 2 / ruby 26100 puppet rtd DIR 254,0 4096 2 / ruby 26100 puppet txt REG 254,0 5288 115042 /usr/bin/ruby1.8 ruby 26100 puppet mem REG 254,0 88968 172152 /lib/libnsl-2.7.so ruby 26100 puppet mem REG 254,0 2138416 165012 /usr/lib/libmysqlclient.so.15.0.0 ruby 26100 puppet mem REG 254,0 78976 264552 /usr/lib/ruby/1.8/x86_64-linux/mysql.so ruby 26100 puppet mem REG 254,0 23560 266160 /usr/lib/ruby/1.8/x86_64-linux/http11.so ruby 26100 puppet mem REG 254,0 8208 264470 /usr/lib/ruby/1.8/x86_64-linux/digest/md5.so ruby 26100 puppet mem REG 254,0 10784 264492 /usr/lib/ruby/1.8/x86_64-linux/shadow.so ruby 26100 puppet mem REG 254,0 44448 264465 /usr/lib/ruby/1.8/x86_64-linux/bigdecimal.so ruby 26100 puppet mem REG 254,0 12720 264467 /usr/lib/ruby/1.8/x86_64-linux/digest.so ruby 26100 puppet mem REG 254,0 1667176 171245 /usr/lib/libcrypto.so.0.9.8 ruby 26100 puppet mem REG 254,0 330880 171246 /usr/lib/libssl.so.0.9.8 ruby 26100 puppet mem REG 254,0 298568 264491 /usr/lib/ruby/1.8/x86_64-linux/openssl.so ruby 26100 puppet mem REG 254,0 15184 279484 /usr/lib/ruby/1.8/x86_64-linux/racc/cparse.so ruby 26100 puppet mem REG 254,0 14312 169673 /usr/lib/gconv/UTF-16.so ruby 26100 puppet mem REG 254,0 16624 264477 /usr/lib/ruby/1.8/x86_64-linux/iconv.so ruby 26100 puppet mem REG 254,0 19256 264485 /usr/lib/ruby/1.8/x86_64-linux/strscan.so ruby 26100 puppet mem REG 254,0 271320 264480 /usr/lib/ruby/1.8/x86_64-linux/nkf.so ruby 26100 puppet mem REG 254,0 12512 264487 /usr/lib/ruby/1.8/x86_64-linux/syslog.so ruby 26100 puppet mem REG 254,0 10928 264475 /usr/lib/ruby/1.8/x86_64-linux/etc.so ruby 26100 puppet mem REG 254,0 5152 264476 /usr/lib/ruby/1.8/x86_64-linux/fcntl.so ruby 26100 puppet mem REG 254,0 45824 264483 /usr/lib/ruby/1.8/x86_64-linux/socket.so ruby 26100 puppet mem REG 254,0 93504 165898 /usr/lib/libz.so.1.2.3.3 ruby 26100 puppet mem REG 254,0 37856 264489 /usr/lib/ruby/1.8/x86_64-linux/zlib.so ruby 26100 puppet mem REG 254,0 15888 264488 /usr/lib/ruby/1.8/x86_64-linux/thread.so ruby 26100 puppet mem REG 254,0 122848 264486 /usr/lib/ruby/1.8/x86_64-linux/syck.so ruby 26100 puppet mem REG 254,0 20768 264484 /usr/lib/ruby/1.8/x86_64-linux/stringio.so ruby 26100 puppet mem REG 254,0 1375536 172156 /lib/libc-2.7.so ruby 26100 puppet mem REG 254,0 534736 172148 /lib/libm-2.7.so ruby 26100 puppet mem REG 254,0 39112 172150 /lib/libcrypt-2.7.so ruby 26100 puppet mem REG 254,0 14616 172154 /lib/libdl-2.7.so ruby 26100 puppet mem REG 254,0 130114 172171 /lib/libpthread-2.7.so ruby 26100 puppet mem REG 254,0 946760 169053 /usr/lib/libruby1.8.so.1.8.7 ruby 26100 puppet mem REG 254,0 119288 172167 /lib/ld-2.7.so ruby 26100 puppet DEL REG 254,2 37750356 /var/run/nscd/dbeKJQ5D ruby 26100 puppet mem REG 254,2 217016 12928501 /var/cache/nscd/services ruby 26100 puppet DEL REG 254,2 37750355 /var/run/nscd/dblolCC6 ruby 26100 puppet DEL REG 254,2 37750354 /var/run/nscd/db3DHabz ruby 26100 puppet mem REG 254,0 25700 164974 /usr/lib/gconv/gconv-modules.cache ruby 26100 puppet 0r CHR 1,3 565 /dev/null ruby 26100 puppet 1w CHR 1,3 565 /dev/null ruby 26100 puppet 2w CHR 1,3 565 /dev/null ruby 26100 puppet 3r FIFO 0,6 1514186 pipe ruby 26100 puppet 4w FIFO 0,6 1514186 pipe ruby 26100 puppet 5u unix 0xffff81002f412440 1514460 socket ruby 26100 puppet 6u IPv4 1514464 TCP localhost:18140 (LISTEN) ruby 26100 puppet 8w REG 254,6 1929 4436700 /srv/mnt/dsk/cgxappsvg1-dlv00data/cgx1/puppet/var/log/rails.log ruby 26100 puppet 9u unix 0xffff81008f9caf40 1518960 socket ruby 26100 puppet 10u unix 0xffff81004bc656c0 1531736 socket ruby 26100 puppet 11u unix 0xffff81003c53ac40 1525135 socket ruby 26100 puppet 12u unix 0xffff81004bc65c40 1527344 socket ruby 26100 puppet 13u unix 0xffff81002f4129c0 1763840 socket ruby 26100 puppet 14u unix 0xffff81002f413d00 1536103 socket ruby 26100 puppet 15u unix 0xffff81002f4134c0 1536500 socket ruby 26100 puppet 16u unix 0xffff81008f9ca440 1540994 socket ruby 26100 puppet 17u unix 0xffff81008f9ca9c0 1562781 socket ruby 26100 puppet 18u unix 0xffff8100aad0ec40 1569133 socket ruby 26100 puppet 19u unix 0xffff81003c53b740 1658921 socket ruby 26100 puppet 20u unix 0xffff81003c53a140 1691003 socket ruby 26100 puppet 21u unix 0xffff81003c53a400 1693685 socket ruby 26100 puppet 22u unix 0xffff81002f412f40 1731935 socket ... etc... -- Atentamente Héctor Rivas On Thu, Sep 2, 2010 at 7:21 PM, James Turnbull <james@puppetlabs.com> wrote:> Colm O''Connor wrote: >> I keep on getting this message: >> >> err: Too many open files - /var/lib/puppet/state/state.yaml.tmp >> >> Does anybody know what might be causing it? >> >> I''m also getting "err: State got corrupted". Everything does seem to >> be kind of working, but it''s sort of worrying getting these new errors. >> > > What version? What platform? > > Regards > > James Turnbull > > -- > Puppet Labs - http://www.puppetlabs.com > C: 503-734-8571 > > -- > You received this message because you are subscribed to the Google Groups "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Héctor Rivas Gándara
2010-Sep-03 11:53 UTC
Re: [Puppet Users] err: Too many open files - /var/lib/puppet/state/state.yaml.tmp
2010/9/3 Héctor Rivas Gándara <keymon@gmail.com>:> To workaround the problem you can increase the ulimit for open files > (/etc/security/limits.conf) and restart puppetmaster periodically > (¿each 6h?).About this solution, I think that I found another bug in puppetmasterd: http://projects.reductivelabs.com/issues/4707 If you change the ulimits probably you should start the daemon using "sudo -u puppet <initscript>". -- Atentamente Héctor Rivas -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Seemingly Similar Threads
- SSLVerifyClient required with Apache+Mongrel
- err: Could not request certificate: sslv3 alert handshake failure error
- Upgraded puppet-server from EPEL 24.8 to 25.1 - now seeing "puppetmasterd[xxxx]: Too many connections"
- Using a single class multiple times
- Gracefully recovering from state.yaml corruption.