freebsd stable - Oct 2014 - [jcigar@ulb.ac.be: Listen queue overflow: 8 already in queue awaiting acceptance]

sorry for cross-posting, I'm forwarding this as it seems that part of
the problem is also related to:
https://lists.freebsd.org/pipermail/freebsd-net/2014-September/039664.html

I also wonder if something has been fixed in -STABLE in this area ..

(please keep me in CC as I'm not subscribed on freebsd-net@ an
freebsd-stable@)

-- 
Julien Cigar
Belgian Biodiversity Platform (http://www.biodiversity.be)
PGP fingerprint: EEF9 F697 4B68 D275 7B11  6A25 B2BB 3710 A204 23C0
No trees were killed in the creation of this message.
However, many electrons were terribly inconvenienced.
-------------- next part --------------
An embedded message was scrubbed...
From: Julien Cigar <jcigar at ulb.ac.be>
Subject: Listen queue overflow: 8 already in queue awaiting acceptance
Date: Thu, 2 Oct 2014 11:52:06 +0200
Size: 5788
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20141002/6fb8cfd0/attachment.mht>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20141002/6fb8cfd0/attachment.sig>

On Thu, Oct 2, 2014 at 12:42 PM, Julien Cigar <jcigar at ulb.ac.be> wrote:
> sonewconn: pcb 0xfffff8010e561310: Listen queue overflow: 8 already in
> queue awaiting acceptance
>
My immediate reaction is to find out which program is listening on that
socket, and that it doesn't have

    listen(s, 8);

in its code somewhere.

-- 
brandon s allbery kf8nh                               sine nomine associates
allbery.b at gmail.com                                  ballbery at
sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

On Thu, Oct 2, 2014 at 9:42 AM, Julien Cigar <jcigar at ulb.ac.be>
wrote:
> sorry for cross-posting, I'm forwarding this as it seems that part of
> the problem is also related to:
> https://lists.freebsd.org/pipermail/freebsd-net/2014-September/039664.html
Umm, this looks like a different problem than the subject of this
email.
>
> I also wonder if something has been fixed in -STABLE in this area ..
>
> (please keep me in CC as I'm not subscribed on freebsd-net@ an
> freebsd-stable@)
>
> --
> Julien Cigar
> Belgian Biodiversity Platform (http://www.biodiversity.be)
> PGP fingerprint: EEF9 F697 4B68 D275 7B11  6A25 B2BB 3710 A204 23C0
> No trees were killed in the creation of this message.
> However, many electrons were terribly inconvenienced.
>
>
> ---------- Forwarded message ----------
> From: Julien Cigar <jcigar at ulb.ac.be>
> To: freebsd-questions at freebsd.org
> Cc:
> Date: Thu, 2 Oct 2014 11:52:06 +0200
> Subject: Listen queue overflow: 8 already in queue awaiting acceptance
> Hello,
>
> I'm running 10-RELEASE on a HP Proliant DL160 Gen8 and I'm seeing
the
> following in my kernel logs:
> sonewconn: pcb 0xfffff8010e561310: Listen queue overflow: 8 already in
> queue awaiting acceptance
This usually means the application is not keeping up with the incoming
connections.
>
> I already raised kern.ipc.soacceptqueue to 1024 and  netstat -naA | grep
> "fffff8010e561310" returns nothing
This is the usual way of finding the culprit process. If this doesn't
return anything, it probably means that it is a short-lived process.

Here is an example of what you could do:

sonewconn: pcb 0xfffff8008f40cb10: Listen queue overflow: 1 already in queue
awaiting acceptance
>From kgdb,
(kgdb) p ((struct inpcb *)0xfffff8008f40cb10)->inp_inc
$3 = {inc_flags = 0 '\0', inc_len = 0 '\0', inc_fibnum = 0,
inc_ie = {ie_fport
= 0, ie_lport = 10295, ie_dependfaddr = {
      ie46_foreign = {ia46_pad32 = {0, 0, 0}, ia46_addr4 = {s_addr = 0}},
ie6_foreign = {__u6_addr = {
          __u6_addr8 = '\0' <repeats 15 times>, __u6_addr16 = {0,
0, 0, 0, 0,
0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}},
    ie_dependladdr = {ie46_local = {ia46_pad32 = {0, 0, 0}, ia46_addr4 {s_addr =
0}}, ie6_local = {__u6_addr = {
          __u6_addr8 = '\0' <repeats 15 times>, __u6_addr16 = {0,
0, 0, 0, 0,
0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}}}}

Here, ie_lport = 10295 which is in n/w byte order and converting it to host
byte order, 10295 -> 0x2837 and swapping them gives us 0x3728 which is 14120.

Now, use sockstat to find out what process is on that port:

$ sockstat -l | grep 14120

cheers,
Hiren