Hi, I''m running Ubuntu 8.04 XEN with latest kernel as a dom0. I''m running Windows 2003 SP1 as domU, where I also have installed the 0.9.11 PV Drivers. Everything works fine except for the outgoing IP traffic from the domU, which seems to be limited to a maximum of around 0.5 mbit. The incoming IP traffic works fine. No limitations. If I start the dumU without the PV drivers activated, the outgoing IP traffic is working like normal. What am I doing wrong here? /Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi, I''m running Ubuntu 8.04 XEN with latest kernel as a dom0. I''m running Windows 2003 SP1 as domU, where I also have installed the 0.9.11 PV Drivers. Everything works fine except for the outgoing IP traffic from the domU, which seems to be limited to a maximum of around 0.5 mbit. The incoming IP traffic works fine. No limitations. If I start the dumU without the PV drivers activated, the outgoing IP traffic is working like normal. What am I doing wrong here? /Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha
2009-Feb-28 19:52 UTC
Re: [Xen-users] Slow outgoing IP traffic with PV drivers
On Sat, Feb 28, 2009 at 11:22 PM, Peter Privat <peter.privat@breilind.com> wrote:> Everything works fine except for the outgoing IP traffic from the domU, > which seems to be limited to a maximum of around 0.5 mbit. The incoming IP > traffic works fine. No limitations. If I start the dumU without the PV > drivers activated, the outgoing IP traffic is working like normal. What am I > doing wrong here?The usual recommendations would be "try turning off all checksums, starting from domU, and then dom0". Others have also say "Ubuntu''s version of Xen sucked, use Debian instead or compile Xen from source". What I can say is that I''m using several RHEL5.3 dum0, with both RH''s bundled Xen and 3.3.1 from Gitco, both can run Windows domU with PV drivers just fine, having domU <-> dom0 network performance > 1 Gbps, and domU <-> outside world close to 100Mbps (mainly because I''m stuck with 100Mbps Cisco switches). Regards, Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
James Harper
2009-Mar-01 04:56 UTC
RE: [Xen-users] Slow outgoing IP traffic with PV drivers
> Hi, > > I''m running Ubuntu 8.04 XEN with latest kernel as a dom0. I''m running > Windows 2003 SP1 as domU, where I also have installed the 0.9.11 PV > Drivers. Everything works fine except for the outgoing IP traffic fromthe> domU, which seems to be limited to a maximum of around 0.5 mbit. The > incoming IP traffic works fine. No limitations. If I start the dumU > without the PV drivers activated, the outgoing IP traffic is workinglike> normal. What am I doing wrong here? >Where is your IP traffic going? (to Dom0, to another DomU, or to a physical network adapter) If a physical network adapter, what sort? I''d start by turning off the available offloads in the advanced properties of the gplpv xennet adapter. James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Peter Privat
2009-Mar-01 13:29 UTC
RE: [Xen-users] Slow outgoing IP traffic with PV drivers
On Sun, 2009-03-01 at 15:56 +1100, James Harper wrote:> > Hi, > > > > I''m running Ubuntu 8.04 XEN with latest kernel as a dom0. I''m running > > Windows 2003 SP1 as domU, where I also have installed the 0.9.11 PV > > Drivers. Everything works fine except for the outgoing IP traffic from > the > > domU, which seems to be limited to a maximum of around 0.5 mbit. The > > incoming IP traffic works fine. No limitations. If I start the dumU > > without the PV drivers activated, the outgoing IP traffic is working > like > > normal. What am I doing wrong here? > > > > Where is your IP traffic going? (to Dom0, to another DomU, or to a > physical network adapter) > > If a physical network adapter, what sort? >Hi James, I''m using Xen on two different servers. On one server, the IP traffic is going through the onboard 100 mbit ethernet port IP adaptor (nVidia Corporation MCP61 Ethernet). vif = [ ''type=ioemu, bridge=eth0'' ] The other server is using both the onboard 1 Gbit ( Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller) port and an PCI slot 1 Gbit (Realtek Semiconductor Co., Ltd. RTL-8169 Gigabit Ethernet) port. Either: vif = [ ''type=ioemu, bridge=eth0'' ] or vif = [ ''type=ioemu, bridge=eth1'' ]> I''d start by turning off the available offloads in the advanced > properties of the gplpv xennet adapter. >James, I''ve received following from Øyvind at this list, which really helped me out: "Try turning off Large Segment Offloading in the driver. Most likely your bridge is pushing packets out on the physical network with a larger size than the recipient can handle. You could test this by sniffing the recipient end the bridge. If you see large packets out from the bridge that doesn''t get to the recipient, then you have found your problem. Large Segment Offloading is enabled by default, and set to a high value (larger then MTU). Since it seems that the PV driver does not respect the MTU, and does not segment the TCP-data (for better domu-domu speeds?) this causes the bridge (this might be due to a bug in my setup which is gentoo with vanilla xen configuration) to transmit frames that are larger than the supported MTU of the recipients. I haven''t yet tried enabling support for jumbo frames on my clients, but still." James... Are there any more "tunings" possible? Regards, /Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
James Harper
2009-Mar-02 01:58 UTC
RE: [Xen-users] Slow outgoing IP traffic with PV drivers
> James, I''ve received following from Øyvind at this list, which really > helped me out: > > "Try turning off Large Segment Offloading in the driver. > Most likely your bridge is pushing packets out on the physical network > with a larger size than the recipient can handle. > You could test this by sniffing the recipient end the bridge. If you see > large packets out from the bridge that doesn''t get to the recipient, then > you have found your problem. > > Large Segment Offloading is enabled by default, and set to a high value > (larger then MTU). Since it seems that the PV driver does not respect the > MTU, and does not segment the TCP-data (for better domu-domu speeds?) this > causes the bridge (this might be due to a bug in my setup which is gentoo > with vanilla xen configuration) to transmit frames that are larger than > the supported MTU of the recipients. I haven''t yet tried enabling support > for jumbo frames on my clients, but still." > > James... Are there any more "tunings" possible? >Not really. All of those offloads will make things go faster if they work, but much much slower if they don''t. The idea of LSO is to give the physical network card 64k packets and let it break them into MTU sized chunks. If your linux kernel network card driver has bugs in that code path (many do) then it will either completely not work at all, or will seem to work for a few days/weeks/months and then start misbehaving. James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Peter Privat
2009-Mar-04 09:55 UTC
RE: [Xen-users] Slow outgoing IP traffic with PV drivers
On Mon, 2009-03-02 at 12:58 +1100, James Harper wrote:> > Large Segment Offloading is enabled by default, and set to a high value > > (larger then MTU). Since it seems that the PV driver does not respect the > > MTU, and does not segment the TCP-data (for better domu-domu speeds?) this > > causes the bridge (this might be due to a bug in my setup which is gentoo > > with vanilla xen configuration) to transmit frames that are larger than > > the supported MTU of the recipients. I haven''t yet tried enabling support > > for jumbo frames on my clients, but still." > > > > James... Are there any more "tunings" possible? > > > > Not really. All of those offloads will make things go faster if they work, but much much slower if they don''t. The idea of LSO is to give the physical network card 64k packets and let it break them into MTU sized chunks. If your linux kernel network card driver has bugs in that code path (many do) then it will either completely not work at all, or will seem to work for a few days/weeks/months and then start misbehaving.If I got it right then... Large Segment Offloading activated, if it works correctly with the dom0, will improve network traffic speed over having it disabled. Right? /Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users