Arjan Filius
2008-Jul-10 12:15 UTC
[Xen-users] TX tcp checksum errors with Xen GPLPV 0.9.9 Drivers (xen 3.2.1 and windows Server x86 2003 R2)
Hello, My first post on Xen-Users, so .. i''ve discovered a strange problem. Setup: -A Windows server 2003R2 (x86) with GPL PV driver 0.9.9 ipferf 1.7.0 (from 2003) -dom0 a opensuse 11.0 xen: # rpm -q -a |grep -i xen kqemu-kmp-xen-1.3.0pre11_2.6.25.5_1.1-7.1 kiwi-desc-xenboot-2.38-67.1 xen-3.2.1_16881_04-4.2 xen-tools-3.2.1_16881_04-4.2 xen-libs-3.2.1_16881_04-4.2 kernel-xen-2.6.25.9-0.2 iperf 2.0.4 -1 interface as trunk on dom0, with some vlans''s and xen bridges -1 test machine in same subnet as the domU windows 2003 server. iperf 2.0.4 test: 1) on testmachachine: iperf -c <domU-IP> -p 2000 on domU iperf.exe -s -p 2000 gives great performance, about 850Mbit/s 2) The other way around: on domU iperf -c <test-machine-IP> -p 2000 on test machine; iperf -s -p 2000 gives dramatically low results like 50KBit/s (no typo!) 3) same tests as above but instead of a remote test machine using to dom0 gives values beyound our physical limit (1.5GBit/s) in both ways. (That may lead to a conclusion the something is wrong with the physical network, but it''s not, see point 4 ) 4) iperf tests in both directions from dom0 to the remote machine are just excellent. So nothing seems wrong with the physical network. After some research i found exceptional many transmit tcp checksum errors. for that i: -disabled all NIC offloading on the test machine (ethtool -K ethx ... off) -disables all NIC offloading on dom0 (ethtool -K ethx .... off) -and finally disabled offloading in gplpv driver settings (gui) -just to be sure also created a registry setting to disable NIC offloading And still, after all this , while sniffing (tcpdump) on the virtual interface at the dom0 (vifx.0) the first syn packet seems to have a correct checksum, but the next packet (ACK 3th packet in tcp 3-way handshake) has an incorrect checksum. All folowing packets that are transmitted have all tcp checksum errors So exept for the first outgoing SYN no other correct TCP checksum packets seems to leave the system. An other test met simple ftp results in terrible slow uploads (from domU to test machine) I did''t investigate this ftp further, however it intreges me why from my findings _any_ data gets transferred just created a tcpdump on the vif of the ftp session, and found by the eye no packet without checksum error. the relative acknowledge numer is in all transmitted packets ''0''. While using the regular realtek NIC driver in xen i got a TX and RX performance from about 85Mbit/s witch is acceptable for a 100MBit/s NIC emulation, but not for the perpose i want to use (Gbit) I also tried to sniff in the domU itself, but the M$ monitor tool seems just not to see the NIC. didn''t try winpcap/wireshark yet. Any thoughts, experiences solutions? Many thanks in advance please while reply-ing, reply directly to my email adres too please Regards, -- Arjan Filius mailto:iafilius@xs4all.nl _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Arjan Filius
2008-Jul-10 13:34 UTC
Re: [Xen-users] TX tcp checksum errors with Xen GPLPV 0.9.9 Drivers (xen 3.2.1 and windows Server x86 2003 R2)
Hello, i''m sorry, but i just noticed more uptodate drivers, and that give much more performance, and i think therefor the tx checksummin gissue is been fixed. Now with 0.9.11.pre5 driver: domU CPU about 50% Test Server:~# ./iperf -c Windows2003-DomU -p 2000 -t 60 ------------------------------------------------------------ Client connecting to 10.11.2.7, TCP port 2000 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local Test-Server port 50993 connected with Windows-2003-dom0 port 2000 [ ID] Interval Transfer Bandwidth [ 3] 0.0-60.0 sec 6.18 GBytes 885 Mbits/sec the other way around: domU CPU about 45% tes server~# ./iperf -s -p 2000 -t 60 ------------------------------------------------------------ Server listening on TCP port 2000 TCP window size: 85.3 KByte (default) ------------------------------------------------------------ [ 4] local Test-Server port 2000 connected with Windows-2003-Dom0 port 1033 [ ID] Interval Transfer Bandwidth [ 4] 0.0-60.0 sec 2.19 GBytes 313 Mbits/sec So transmitting packets (with all offloading disabled) isn''t just perfect yet, but got 1/3 wirespeed for 1GB Nic''s/network. Regards, On Thu, 10 Jul 2008, Arjan Filius wrote:> Hello, > > My first post on Xen-Users, so .. > > i''ve discovered a strange problem. > > Setup: > -A Windows server 2003R2 (x86) with GPL PV driver 0.9.9 > ipferf 1.7.0 (from 2003) > -dom0 a opensuse 11.0 xen: > # rpm -q -a |grep -i xen > kqemu-kmp-xen-1.3.0pre11_2.6.25.5_1.1-7.1 > kiwi-desc-xenboot-2.38-67.1 > xen-3.2.1_16881_04-4.2 > xen-tools-3.2.1_16881_04-4.2 > xen-libs-3.2.1_16881_04-4.2 > kernel-xen-2.6.25.9-0.2 > iperf 2.0.4 > -1 interface as trunk on dom0, with some vlans''s and xen bridges > > -1 test machine in same subnet as the domU windows 2003 server. > iperf 2.0.4 > > test: > 1) > on testmachachine: iperf -c <domU-IP> -p 2000 > on domU iperf.exe -s -p 2000 > gives great performance, about 850Mbit/s > > 2) > The other way around: > on domU iperf -c <test-machine-IP> -p 2000 > on test machine; iperf -s -p 2000 > gives dramatically low results like 50KBit/s (no typo!) > > 3) > same tests as above but instead of a remote test machine using to dom0 gives > values beyound our physical limit (1.5GBit/s) in both ways. > > (That may lead to a conclusion the something is wrong with the physical > network, but it''s not, see point 4 ) > > 4) iperf tests in both directions from dom0 to the remote machine are just > excellent. So nothing seems wrong with the physical network. > > > After some research i found exceptional many transmit tcp checksum errors. > for that i: > -disabled all NIC offloading on the test machine (ethtool -K ethx ... off) > -disables all NIC offloading on dom0 (ethtool -K ethx .... off) > -and finally disabled offloading in gplpv driver settings (gui) > -just to be sure also created a registry setting to disable NIC offloading > > And still, after all this , while sniffing (tcpdump) on the virtual interface > at the dom0 (vifx.0) the first syn packet seems to have a correct checksum, > but the next packet (ACK 3th packet in tcp 3-way handshake) has an incorrect > checksum. > All folowing packets that are transmitted have all tcp checksum errors > So exept for the first outgoing SYN no other correct TCP checksum packets > seems to leave the system. > > An other test met simple ftp results in terrible slow uploads (from domU to > test machine) > I did''t investigate this ftp further, however it intreges me why from my > findings _any_ data gets transferred > just created a tcpdump on the vif of the ftp session, and found by the eye no > packet without checksum error. the relative acknowledge numer is in all > transmitted packets ''0''. > > While using the regular realtek NIC driver in xen i got a TX and RX > performance from about 85Mbit/s witch is acceptable for a 100MBit/s NIC > emulation, but not for the perpose i want to use (Gbit) > > > I also tried to sniff in the domU itself, but the M$ monitor tool seems just > not to see the NIC. didn''t try winpcap/wireshark yet. > > Any thoughts, experiences solutions? > > Many thanks in advance > > please while reply-ing, reply directly to my email adres too please > > Regards, > >-- Arjan Filius mailto:iafilius@xs4all.nl _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users