Laurent Dumont
2014-Dec-31 07:10 UTC
[CentOS] CentOS 6.6 - net-snmp process - Too many open files
Hi gents, I have a CentOS 6.6 install with net-snmp running. I'm using SNMPv3 Auth and Encryption with Observium polling devices every five minutes. On one of my Centos server, the net-snmp process will die after 24-36 hours of running. It seems to be hitting my system maximum file open limit which is set to 1024.> [root at mail net-snmp-5.7]# ulimit -n > 1024That number will keep increasing till it trips the 1024 limit.> [root at mail ~]# lsof -p 26907 | wc -l > 1071> [root at mail net-snmp-5.7]# lsof -p 26907 > snmpd 26907 root 166r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 167r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 168r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 169r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 170r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 171r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 172r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 173r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 174r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 175r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 176r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 177r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 178r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp > snmpd 26907 root 179r REG 0,3 0 4026532160 > /proc/26907/net/sctp/snmp> #tail /var/log/snmpd.log > cannot open /proc/net/dev ... > /proc/partitions: Too many open files > /proc/stat: Too many open files > /proc/partitions: Too many open files > /proc/stat: Too many open files > could not create socket > /proc/partitions: Too many open files > /proc/stat: Too many open files > cannot open /proc/net/dev ... > /proc/partitions: Too many open files > /proc/stat: Too many open files > /proc/partitions: Too many open files > /proc/stat: Too many open filesIt seems that CentOS is frozen to net-snmp 5.5 and I'm don't really want to use anything that I can't easily manage with "yum". The weird thing is that I have another server with the same config/CentOS version/Net-Snmp version and I have no issues with the snmp process on that server. I could raise the open file limit but that just seems unnecessary since I don't think that snmp is supposed to work that way. Anything else I could try to troubleshoot the issue? Thanks!