Hi all, I spent some time trying to track down slow tcp performance on a small office switched 100 LAN. We just put in a number of whitebox PCs running FreeBSD 6.1-p2/PC-BSD 1.2 that all have onboard Via Rhine 10/100 ethernet controllers. Performace with scp was around 200KB/s, ftp wavered between 300-500KB/s. This did not appear to be a duplex mismatch - unmanaged switch showed them all at 100/Full, put some other hosts on the same ports/cabling and got near wire speed. I took the cabling out of the equation, the switch, no improvement. The only thing that got me decent performance was putting two hosts back to back with an xover cable. I eventually realized that the only hosts with any speed issues in the office were these boxes with the Via ethernet. Putting an equally cheap DLink (RealTek/rl) in one of them gave me much better performance. At another site, I was dealing with a new intranet server running FreeBSD 6.2-PRE (11/16) on a decent Asus board. This also has an onboard Via Rhine ethernet controller. While pulling some files over from the box it was replacing, I noticed that I was getting only a few hundred KB/s on this box. Before putting it into production, I grabbed a cheap Intel 10/100 card and put that in. Problem solved. So it seems to me like perhaps there's an issue with the vr driver. I noticed it does have some quirks mentioned in the manpage, and I don't see too many changes to the driver in the last year or so. Is there any information I can supply to help debug this? I've got a bunch of these machines around. I can get a tcpdump from both ends during an ftp transfer, and the boxes are mine to toy with after hours. I've posted a dmesg from both boxes (PC-BSD and 6.2-PRE): http://www.bway.net/~spork/6.1p2-dmesg.txt http://www.bway.net/~spork/6.2-dmesg.txt Thanks, Charles ___ Charles Sprickman NetEng/SysAdmin Bway.net - New York's Best Internet - www.bway.net spork@bway.net - 212.655.9344
Charles Sprickman wrote: [snipped]> Performace with scp was around 200KB/s, ftp > wavered between 300-500KB/s. This did not appear to be a duplex > mismatch - unmanaged switch showed them all at 100/Full, put some other > hosts on the same ports/cabling and got near wire speed.I just checked with the boxes I have here. One is an Athon XP on a Asus board with a VIA Rhine II on board; the other is a `old' Celeron 500 on a MSI board with a Intel ``Pro 100/S Desktop Adapter''. I transfered a 538MiB file (some old CURRENT snapshot) via sftp. My result: from box one to box two (vr to fxp) I get 2.3MB/s; from box two to box one (fxp to vr) I get 3.1MB/s. The first box is running 6.2-PRERELEASE from 11/18, the Intel box is running 7.0-CURRENT from 11/24. Output of $ dmesg | grep vr vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0x8000-0x80ff mem 0xd6000000-0xd60000ff at device 18.0 on pci0 miibus0: <MII bus> on vr0 and $ dmesg | grep fxp fxp0: <Intel 82550 Pro/100 Ethernet> port 0xc000-0xc03f mem 0xdd020000-0xdd020fff,0xdd000000-0xdd01ffff irq 11 at device 1.0 on pci1 miibus0: <MII bus> on fxp0 [...] fxp0: Microcode loaded, int_delay: 1000 usec bundle_max: 6 fxp0: Microcode loaded, int_delay: 1000 usec bundle_max: 6 If needed I can provide a full dmesg of both boxes. HTH, Philipp -- www.familie-ost.info/~pj
On Tue, Nov 28, 2006 at 10:50:17PM -0500, Charles Sprickman wrote: > Hi all, > > I spent some time trying to track down slow tcp performance on a small > office switched 100 LAN. We just put in a number of whitebox PCs running > FreeBSD 6.1-p2/PC-BSD 1.2 that all have onboard Via Rhine 10/100 ethernet > controllers. Performace with scp was around 200KB/s, ftp wavered between > 300-500KB/s. This did not appear to be a duplex mismatch - unmanaged > switch showed them all at 100/Full, put some other hosts on the same > ports/cabling and got near wire speed. I took the cabling out of the > equation, the switch, no improvement. The only thing that got me decent > performance was putting two hosts back to back with an xover cable. > > I eventually realized that the only hosts with any speed issues in the > office were these boxes with the Via ethernet. Putting an equally cheap > DLink (RealTek/rl) in one of them gave me much better performance. > > At another site, I was dealing with a new intranet server running FreeBSD > 6.2-PRE (11/16) on a decent Asus board. This also has an onboard Via > Rhine ethernet controller. While pulling some files over from the box it > was replacing, I noticed that I was getting only a few hundred KB/s on > this box. Before putting it into production, I grabbed a cheap Intel > 10/100 card and put that in. Problem solved. > > So it seems to me like perhaps there's an issue with the vr driver. I > noticed it does have some quirks mentioned in the manpage, and I don't see > too many changes to the driver in the last year or so. > > Is there any information I can supply to help debug this? I've got a > bunch of these machines around. I can get a tcpdump from both ends during > an ftp transfer, and the boxes are mine to toy with after hours. > > I've posted a dmesg from both boxes (PC-BSD and 6.2-PRE): > > http://www.bway.net/~spork/6.1p2-dmesg.txt > http://www.bway.net/~spork/6.2-dmesg.txt > VIA Rhine has severe hardware limitations and you can't expect good performance from the NIC. On Tx side the NIC need 4 bytes aligned mbuf so the driver defragments the mbuf chains with m_defrag(9). On Rx side it strips off CRC with m_devget(9) as the NIC has no way to remove the CRC from the received frame. These m_defrag(9)/ m_devget(9) results in bcopy operation and they nullyfies the advantage of DMA which in trun waste significant CPU cycles. If you are in bad need of getting full 100Mbps speed you could buy a cheap PCI gigabit NIC that is supported by re(4)/sk(4)/stge(4) etc. -- Regards, Pyun YongHyeon
Charles Sprickman wrote:> Hi all, > > I spent some time trying to track down slow tcp performance on a small > office switched 100 LAN. We just put in a number of whitebox PCs > running FreeBSD 6.1-p2/PC-BSD 1.2 that all have onboard Via Rhine > 10/100 ethernet controllers. Performace with scp was around 200KB/s, > ftp wavered between 300-500KB/s. This did not appear to be a duplex > mismatch - unmanaged switch showed them all at 100/Full, put some > other hosts on the same ports/cabling and got near wire speed. I took > the cabling out of the equation, the switch, no improvement. The only > thing that got me decent performance was putting two hosts back to > back with an xover cable. > > I eventually realized that the only hosts with any speed issues in the > office were these boxes with the Via ethernet. Putting an equally > cheap DLink (RealTek/rl) in one of them gave me much better performance.[...] There might be something in this. I have a 2-3 yr old box with an Asus A7V8X mainboard. This has been running 4.10-RELEASE for about 2.5 years and yesterday I decided to bite the bullet and update to RELENG_6. I also decided to upgrade the internal storage as well so this entailed a backup/restore over the wire. Backing up the 4.10 box was within an acceptable margin of wire speed (~8MB/s on 100M ethernet) given that a router was in the middle. That's roughly how the box has always performed. After installing 6.1_RELEASE and updating to RELENG_6 I started the restore. Maxed out at about 380kB/s. Now I tested the backup speed again and it has also dropepd to 380k. This would definitely lead me to assume that something has gone awry in the driver over the past years. Unfortunately I have not been frequently updating this machine (my imap server) so I have no idea when the driver became broken. I realize that this is not much help in tracking the problem - but it seems to concur with the problem noted here earlier. FreeBSD 6.2-PRERELEASE #0: Thu Nov 30 16:29:04 SGT pciconf: vr0@pci0:18:0: class=0x020000 card=0x80a11043 chip=0x30651106 rev=0x74 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT6102 Rhine II PCI Fast Ethernet Controller' class = network subclass = ethernet dmesg: vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0xb000-0xb0ff mem 0xf2800000-0xf28000ff at device 18.0 on pci0 miibus0: <MII bus> on vr0 rlphy0: <RTL8201L 10/100 media interface> on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: Ethernet address: 00:0c:6e:3d:b9:0e $ ifconfig vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet6 fe80::20c:6eff:fe3d:b90e%vr0 prefixlen 64 scopeid 0x1 inet 203.117.131.34 netmask 0xffffffc0 broadcast 203.117.131.63 inet 203.117.131.35 netmask 0xffffffff broadcast 203.117.131.35 ether 00:0c:6e:3d:b9:0e media: Ethernet autoselect (100baseTX <full-duplex>) status: active -Jamie -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3233 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20061201/51612bab/smime.bin
On 12/4/06, Charles Sprickman <spork@bway.net> wrote:> On Mon, 4 Dec 2006, Jorn Argelo wrote: > > > Jamie Clark wrote: > >> Steven Hartland wrote: > >>> Charles Sprickman wrote: > >>>>> Backing up the 4.10 box was within an acceptable margin of wire > >>>>> speed (~8MB/s on 100M ethernet) given that a router was in the > >>>>> middle. That's roughly how the box has always performed. > >>>>> > >>>>> After installing 6.1_RELEASE and updating to RELENG_6 I started the > >>>>> restore. Maxed out at about 380kB/s. Now I tested the backup speed > >>>>> again and it has also dropepd to 380k. > >>> > >>> That speed is indicative of a FD / HD mismatch between the switch > >>> and the NIC if its hard coded try setting auto if its auto try > >>> hardcoding. N.B. Ensure both ends are set in the same way i.e. hard/auto > >>> or problems start. > >> Both are auto and show 100 FD. The switch port stats show zeros on all the > >> error counters. > >> > >> Good thinking though. I wouldn't yet rule out an external influence as I > >> have not performed any in-depth diagnosis of this - but I can't think of > >> anything obvious aside from the OS upgrade. > >> > >> -Jamie > >> _______________________________________________ > >> freebsd-stable@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > Well my two embedded vr NICs work just fine. I get 8 - 9 MByte / second with > > FTP, using vsftpd, and 3-4 Mbyte / sec with SCP. Though this could also be > > current CPU issues, as the box is quite busy and the Via C3 isn't all that > > fast. I am running 6.2-PRERELEASE. Also, did you try device polling? Maybe > > that helps, although I currently don't have it in my kernel configuration. > > > > Here's an ifconfig output, let me know if you need anything else. > > Can you snip the relevant section of "pciconf -vl" to see the chip > version? It seems like this only happens with some of the Via Rhine II > chips. > > Thanks, > > Charles > > > [user@host] ~> ifconfig > > > > vr0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > > inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 > > ether 00:40:63:df:e5:ee > > media: Ethernet autoselect (100baseTX <full-duplex>) > > status: active > > vr1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > > inet 192.168.1.5 netmask 0xffffff00 broadcast 192.168.1.255 > > ether 00:40:63:df:e5:4e > > media: Ethernet autoselect (100baseTX <full-duplex>) > > status: active > > > > Cheers, > > > > Jornjust to give more info, i have a EPoX old KT600 Motherboard and had difficult times running linux here. some 2.6.x kernel's driver had my NIC resseting when in quite stress ... when then the card would stop working for some time. when i installed FreeBSD in it (think 6.1 till now, could bo 6.0R also) it stopped. i cant say it is the best NIC ever, but it is usable and no longer hangs when someone does a big samba copy throught it. vr0@pci0:18:0: class=0x020000 card=0x300c1695 chip=0x30651106 rev=0x78 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT6102 Rhine II PCI Fast Ethernet Controller' class = network subclass = ethernet also had this one (standalone pci card): vr0@pci0:8:0: class=0x020000 card=0x01051106 chip=0x31061106 rev=0x86 hdr=0x00 vendor = 'VIA Technologies Inc' device = 'VT6105M/LOM Rhine III PCI Fast Ethernet Controller' class = network subclass = ethernet and the same thing happened matheus -- We will call you cygnus, The God of balance you shall be