After updating my 5.3-STABLE/alpha from Dec 4 to Dec 22 on RELENG_5, network _receiving_ throughput on its re(4) interface has collapsed to a maximum of 0.5-1.0 MB/s. (Figures from scp. Yes, I realize this is not the most suitable test, but the box is not CPU starved and used to receive data at several times that rate.) I use the machine as X11 display and quickly noticed that remote X11 apps had become unusually slow. IPv4 and v6 are equally affected. There haven't been any changes to if_re.c in this period. Any other changes that could explain this? -- Christian "naddy" Weisgerber naddy@mips.inka.de
On Thu, 23 Dec 2004, Christian Weisgerber wrote:> After updating my 5.3-STABLE/alpha from Dec 4 to Dec 22 on RELENG_5, > network _receiving_ throughput on its re(4) interface has collapsed to a > maximum of 0.5-1.0 MB/s. (Figures from scp. Yes, I realize this is not > the most suitable test, but the box is not CPU starved and used to > receive data at several times that rate.) I use the machine as X11 > display and quickly noticed that remote X11 apps had become unusually > slow. IPv4 and v6 are equally affected.Could you use a tool like netperf to see whether the slowdown is specific to TCP, or affects UDP also? There have been some TCP tweaks and bugfixes, and this would help isolate that. Seeing the results of a netperf run with the UDP_RR and UDP_STREAM tests in the "before" and "after" scenarios would be helpful. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research> > There haven't been any changes to if_re.c in this period. Any other > changes that could explain this? > > -- > Christian "naddy" Weisgerber naddy@mips.inka.de > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >
Robert Watson:> Could you use a tool like netperf to see whether the slowdown is specific > to TCP, or affects UDP also? There have been some TCP tweaks and > bugfixes, and this would help isolate that. Seeing the results of a > netperf run with the UDP_RR and UDP_STREAM tests in the "before" and > "after" scenarios would be helpful.5.3-STABLE (RELENG_5) on alpha; GENERIC plus if_re.ko. The transmitting side is OpenBSD-current/amd64 (GENERIC, sk). The network is a no-name dumb gigabit switch. No jumbo frames. Results from five runs in a row. Kernel as of December 4: netperf -H 172.16.0.3 -t TCP_STREAM TCP STREAM TEST to 172.16.0.3 Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 65536 16384 16384 10.00 142.11 65536 16384 16384 10.01 142.58 65536 16384 16384 10.01 137.59 65536 16384 16384 10.01 135.01 65536 16384 16384 10.01 139.23 December 22: netperf -H 172.16.0.3 -t TCP_STREAM TCP STREAM TEST to 172.16.0.3 Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 65536 16384 16384 10.08 2.57 65536 16384 16384 10.66 4.50 65536 16384 16384 10.06 6.16 65536 16384 16384 12.10 6.73 65536 16384 16384 10.10 2.84 Both the switch lights and the feeling from, say, scrolling in a Firefox window from a remote host suggest that there are short periods where no packets are transmitted. December 4: netperf -H 172.16.0.3 -t UDP_STREAM UDP UNIDIRECTIONAL SEND TEST to 172.16.0.3 Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 9216 9216 10.00 22783 219677 167.94 42080 10.00 4 0.03 (The large socket size tests all fail, which I presume is irrelevant in this context. Figures from further runs:) 9216 9216 10.00 22285 225815 164.22 9216 9216 10.00 23852 224714 175.78 9216 9216 10.01 23635 225654 174.13 9216 9216 10.01 24139 224133 177.86 December 22: netperf -H 172.16.0.3 -t UDP_STREAM UDP UNIDIRECTIONAL SEND TEST to 172.16.0.3 Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 9216 9216 10.01 24410 221169 179.84 42080 10.01 20 0.15 I would have said that UDP is not affected, except that at this point the FreeBSD box locked up solid. -- Christian "naddy" Weisgerber naddy@mips.inka.de
On Thu, 23 Dec 2004, Christian Weisgerber wrote:> After updating my 5.3-STABLE/alpha from Dec 4 to Dec 22 on RELENG_5, > network _receiving_ throughput on its re(4) interface has collapsed > to a maximum of 0.5-1.0 MB/s. (Figures from scp. Yes, I realize > this is not the most suitable test, but the box is not CPU starved > and used to receive data at several times that rate.) I use the > machine as X11 display and quickly noticed that remote X11 apps had > become unusually slow. IPv4 and v6 are equally affected. > > There haven't been any changes to if_re.c in this period. Any other > changes that could explain this?Since it hasn't been asked .. have you checked the link speed and verified it isn't negotiating at 10Mbs/half instead of gig-e or whatever you're using? I found an rl card (yes I know rl, not re) that decided it really hated the bay switch it was attached to and would only negotiate 10mbit/half, even though the xl card in the same machine negotiated 100mbit/full. Silly stuff like that, and duplex mismatches, can cause problems like this that are seemingly random between reboots. -- Doug White | FreeBSD: The Power to Serve dwhite@gumbysoft.com | www.FreeBSD.org