Lev Serebryakov
2017-Nov-01 15:35 UTC
Low default setting of UDBHASHSIZE leads to unresponsive system
Now 11-STABLE (and 12-CURRENT too) have this: sys/netinet/udp_usrreq.c:#define UDBHASHSIZE 128 Looks like such low value could lead to 100% consumption of CPU by interrupt threads (igb queues in my case) on heavy incoming UDP traffic (torrents with uTP in my case). My system (E3-1220v3 with I210 NICs) becomes completely unresponsive (nut complains about lost connection to UPS, ssh to system times out, etc) when system downloads torrent with many uTP (UDP) peers. Four igb0 queues consume 100% CPU each in this scenario. Total traffic could be very low like 500KiB/s (yes, 500KiB/s, not MiB/s!), I don't speak about 1Gbit/s or even 100Mbit/s here! Rebuilding kernel with UDBHASHSIZE=16384 seems to help. Why is this value so low and why I need to patch sources to change it? Many such settings are changeable via sysctl and/or tunables, but this one looks hardcoded. -- // Lev Serebryakov -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 963 bytes Desc: OpenPGP digital signature URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20171101/59900774/attachment.sig>
Eugene Grosbein
2017-Nov-02 07:19 UTC
Low default setting of UDBHASHSIZE leads to unresponsive system
On 01.11.2017 22:35, Lev Serebryakov wrote:> > Now 11-STABLE (and 12-CURRENT too) have this: > > sys/netinet/udp_usrreq.c:#define UDBHASHSIZE 128 > > Looks like such low value could lead to 100% consumption of CPU by > interrupt threads (igb queues in my case) on heavy incoming UDP traffic > (torrents with uTP in my case). > > My system (E3-1220v3 with I210 NICs) becomes completely unresponsive > (nut complains about lost connection to UPS, ssh to system times out, > etc) when system downloads torrent with many uTP (UDP) peers. Four igb0 > queues consume 100% CPU each in this scenario. > > Total traffic could be very low like 500KiB/s (yes, 500KiB/s, not > MiB/s!), I don't speak about 1Gbit/s or even 100Mbit/s here! > > Rebuilding kernel with UDBHASHSIZE=16384 seems to help. > > Why is this value so low and why I need to patch sources to change it? > Many such settings are changeable via sysctl and/or tunables, but this > one looks hardcoded.You should fill a PR. Attach these performance numbers you got. If possible, attach a patch introducing new loader tunnable, that should be easy. Keep me CC'd.