A few weeks back I upgraded my mail server to -STABLE from a 4.2-STABLE
incarnation that had been running for years. Part of recompiling
everything on the box involved disassociating my use of the
sendmail port I was using and to use the base sendmail in -STABLE.
-STABLE builds with libmilter, so I simply recompiled one of my
milter clients with the milter headers, objs, etc that were produced
from buildworld. However, now I notice that the client (dccm
in this case -- www.rhyolite.com/dcc ) slowly uses all the descriptors
on the box and must be restarted.
I started logging it's use of descriptors over time via a shell
script and lsof, and found that it's leaking kqueue descriptors.
Has anyone:
1. noticed this problem with other libmilter linked clients that
you may be running.
2. have some ideas on how to track down this leakage? I've looked
at the dccm source and it seems to be using milter correctly
(as it was working fine with an older version of sendmail), but
maybe other excellent users have tracked down kqueue leakage
before.
3. was a new kqueue enabled libmilter introduced in the 8.x series of
upgrades of sendmail?
Here's a log excerpt, notice how the fd's for the sockets stay the
same, but the kqueue descriptors grow out of hand:
Wed Jul 16 15:24:09 PDT 2003
total fd:
78
kqueue fd:
56
ipv4 fd:
3
ps:
dcc 21265 0.0 2.0 4004 3084 ?? Ss 12:10PM 0:01.25
/var/dcc/libexec/dccm -l log -t CMN,5,10 -w whiteclnt
Wed Jul 16 15:41:28 PDT 2003
total fd:
78
kqueue fd:
56
ipv4 fd:
3
ps:
dcc 21265 0.0 2.0 4004 3084 ?? Ss 12:10PM 0:01.36
/var/dcc/libexec/dccm -l log -t CMN,5,10 -w whiteclnt
Wed Jul 16 16:17:36 PDT 2003
total fd:
109
kqueue fd:
87
ipv4 fd:
3
ps:
dcc 21265 0.0 1.9 4004 3072 ?? Ss 12:10PM 0:01.64
/var/dcc/libexec/dccm -l log -t CMN,5,10 -w whiteclnt
Wed Jul 16 18:36:17 PDT 2003
total fd:
137
kqueue fd:
115
ipv4 fd:
3
ps:
dcc 21265 0.0 2.0 4004 3080 ?? Ss 12:10PM 0:02.34
/var/dcc/libexec/dccm -l log -t CMN,5,10 -w whiteclnt
Wed Jul 16 20:36:41 PDT 2003
total fd:
165
kqueue fd:
143
ipv4 fd:
3
ps:
dcc 21265 0.0 2.0 4004 3080 ?? Ss 12:10PM 0:02.95
/var/dcc/libexec/dccm -l log -t CMN,5,10 -w whiteclnt
Thu Jul 17 01:17:06 PDT 2003
total fd:
193
kqueue fd:
171
ipv4 fd:
3
ps:
dcc 21265 0.0 2.0 4004 3084 ?? Ss 12:10PM 0:04.16
/var/dcc/libexec/dccm -l log -t CMN,5,10 -w whiteclnt
---
Mark
atkin901 at NOSPAM yahoo dot com
(!wired)?(coffee++):(wired);
In article <slrnbidfm6.11jf.atkin901@adkinson245.f5net.com>, othermark wrote:> 2. have some ideas on how to track down this leakage? I've looked > at the dccm source and it seems to be using milter correctly > (as it was working fine with an older version of sendmail), but > maybe other excellent users have tracked down kqueue leakage > before.I found this pr, which seems to be a likely candidate. http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/55007 dccm is threaded, but libmilter does not call kqueue directly, but based on other searches, I'm suspecting that various calls are implemented using kqueue, kevent (select(), poll(), etc..). I see some evidence of this in the sources, but I'm not familiar enough with the kernel code to say absolutely that these code paths are executed. --- Mark atkin901 at NOSPAM yahoo dot com (!wired)?(coffee++):(wired);
On Wed, 30-Jul-2003 at 15:46:24 +0000, othermark wrote:> In article <slrnbidfm6.11jf.atkin901@adkinson245.f5net.com>, othermark wrote: > > 2. have some ideas on how to track down this leakage? I've looked > > at the dccm source and it seems to be using milter correctly > > (as it was working fine with an older version of sendmail), but > > maybe other excellent users have tracked down kqueue leakage > > before. > > I found this pr, which seems to be a likely candidate. > > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/55007 > > dccm is threaded, but libmilter does not call kqueue directly, but > based on other searches, I'm suspecting that various calls are > implemented using kqueue, kevent (select(), poll(), etc..). I see > some evidence of this in the sources, but I'm not familiar enough > with the kernel code to say absolutely that these code paths are > executed.I have a self-written milter app. Today I upgraded to the recent -STABLE and now it eats KQUEUE filedscriptors like crazy. I tried to MFC the fix in lib/libc_r/uthread/uthread_kqueue.c but I always get complaints about multiple definition of `_kqueue': building shared library libc_r.so.4 kqueue.So: In function `_kqueue': kqueue.So(.text+0x14): multiple definition of `_kqueue' uthread_kqueue.So(.text+0x0): first defined here *** Error code 1 Seems it conflicts with /usr/obj/lib/libc_r/kqueue.S which is generated everytime from scratch during the build. I assume that I have to add kqueue.o to HIDDEN_SYSCALLS but I am not sure if this is correct and what it will break :-) -Andre