Hey gang, I have run into the dredded tcp and udp checksum error in xen, but with a twist. Regardless of whether or not I use a dom0 kernel or a domU kernel I get failed checksum packets whenever I try to route across a virtual interface. I have a pure virtual setup (ala I just have xend bring up a new interface with an ip on it that I route traffic to the domU on) and just like in the FAQ, ping works but no other traffic does because of checksums. I have tried the ethtool thing on all interfaces, tried changing the code in netback.c to force a checksum and have had no luck. Any else having trouble? -- Jason The place where you made your stand never mattered, only that you were there... and still on your feet _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hallo,> Regardless of whether or not I use a dom0 kernel or a domU kernel I > get failed checksum packets whenever I try to route across a virtual > interface. I have a pure virtual setupAh, I just dived into that recently... Is there a bugzilla id for that already? I had the same problem as you, Dom0 can''t communicate with DomU due to the checksum errors in a routed setup, everything works when bridged. But I don''t want to bridge :)> thing on all interfaces, tried changing the code in netback.c to > force a checksum and have had no luck. Any else having trouble?Which netback.c patch did you use? I currently have a working setup using the "interface.c" patch which removes the NETIF_F_NO_CSUM from the dev->features in Dom0 (attached). Using this "sum ok" packets go from Dom0 to DomU (also leaving Dom0 intact), DomU to Dom0 packets are broken leaving DomU but Dom0 ignores that. Using ethtool -K eth0 tx off in DomU I can bring DomU to generate valid sums also, but that''s cosmetic and not needed for a working setup. I''d rather also patch this in the kernel instead of running ethtool in every DomU but I haven''t searched for that yet. Should be easy, though :) As far as I understood the discussion, this is not an easy issue and possibly will be dealt with in in clean way after the 3.0.2 release? For now I have other problems, the above setup seems to work fine for me (also for DomU-DomU communication, which of course is routed over Dom0. Will have to test tunnels/vpn in DomU). (:ul8er, r@y _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
My bad, I meant to say interface.c not netback. I didn''t have any luck the first time I ran a kernel with that patch, but Im going to try again because Im not 100% sure I ran the correct kernel. -- Jason The place where you made your stand never mattered, only that you were there... and still on your feet On Fri, 31 Mar 2006, Florian Kirstein wrote:> Hallo, > >> Regardless of whether or not I use a dom0 kernel or a domU kernel I >> get failed checksum packets whenever I try to route across a virtual >> interface. I have a pure virtual setup > Ah, I just dived into that recently... Is there a bugzilla id > for that already? I had the same problem as you, Dom0 can''t communicate > with DomU due to the checksum errors in a routed setup, everything > works when bridged. But I don''t want to bridge :) > >> thing on all interfaces, tried changing the code in netback.c to >> force a checksum and have had no luck. Any else having trouble? > Which netback.c patch did you use? I currently have a working setup > using the "interface.c" patch which removes the NETIF_F_NO_CSUM from > the dev->features in Dom0 (attached). Using this "sum ok" packets go > from Dom0 to DomU (also leaving Dom0 intact), DomU to Dom0 packets > are broken leaving DomU but Dom0 ignores that. Using > ethtool -K eth0 tx off > in DomU I can bring DomU to generate valid sums also, but that''s cosmetic > and not needed for a working setup. I''d rather also patch this in the > kernel instead of running ethtool in every DomU but I haven''t searched > for that yet. Should be easy, though :) > > As far as I understood the discussion, this is not an easy issue and > possibly will be dealt with in in clean way after the 3.0.2 release? > For now I have other problems, the above setup seems to work fine > for me (also for DomU-DomU communication, which of course is routed > over Dom0. Will have to test tunnels/vpn in DomU). > > (:ul8er, r@y >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
w00t! That fixed it. I read your email on the logic for not doing the tcp checksums and, while I am by no means in possesion of the brains the developers have, I am left wondering why that decision was made. Would anyone care to comment? -- Jason The place where you made your stand never mattered, only that you were there... and still on your feet On Fri, 31 Mar 2006, Florian Kirstein wrote:> Hi, > >> My bad, I meant to say interface.c not netback. I didn''t have any luck the first time I ran a > > It also took me two tries, had a second problem with iptables preventing > me from detecting the working setup correctly %-) But as said, for me > patching interface.c in Dom0 solved the problem. You should see > the intact packets with tcpdump -vvv -n -i eth0 in DomU then. > > (:ul8er, r@y >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jason wrote:> w00t! That fixed it. I read your email on the logic for not doing the > tcp checksums and, while I > am by no means in possesion of the brains the developers have, I am left > wondering why that > decision was made. Would anyone care to comment? >I think I missed a post somewhere along this thread... Which decision are you referring to? Avoiding the TCP checksum between domains? The rationale for not doing the checksum is that it is a significant savings in performance. Even for traffic that goes out to a public net and must contain a checksum, deferring it to dom0 when the OS can offload it to the physical NIC (most NICs these days are capable of computing the checksums in hardware) is a significant saving. The native Linux stack offloads the calculation of the checksum to the NIC, and not doing so in the virtual environment increases the gap when comparing Xen to bare metal Linux. There are a lot of other issues here to resolve, of course, and fixing some of the current issues is being worked. It''s likely going to be an issue for discussion when we''re up for mainline inclusion. thanks, Nivedita _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
If I understand correctly, the offloading to a physical NIC only works when a physical NIC is present and being used for that traffic. In a purely virtual route setup, that traffic should be subject to this checksum issue. Is this just a recent development or is routing a fairly uncommon configuration in comparison to bridging with xen? -- Jason The place where you made your stand never mattered, only that you were there... and still on your feet On Sat, 1 Apr 2006, Nivedita Singhvi wrote:> Jason wrote: >> w00t! That fixed it. I read your email on the logic for not doing the tcp >> checksums and, while I >> am by no means in possesion of the brains the developers have, I am left >> wondering why that >> decision was made. Would anyone care to comment? >> > I think I missed a post somewhere along this thread... > > Which decision are you referring to? Avoiding the > TCP checksum between domains? The rationale for not > doing the checksum is that it is a significant savings > in performance. Even for traffic that goes out to > a public net and must contain a checksum, deferring > it to dom0 when the OS can offload it to the physical > NIC (most NICs these days are capable of computing the > checksums in hardware) is a significant saving. > > The native Linux stack offloads the calculation of > the checksum to the NIC, and not doing so in the virtual > environment increases the gap when comparing Xen to > bare metal Linux. > > There are a lot of other issues here to resolve, of > course, and fixing some of the current issues is being > worked. It''s likely going to be an issue for discussion > when we''re up for mainline inclusion. > > thanks, > Nivedita > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel