Hi all, I have two Dell R720xd servers dom0 running with Centos 5.8 (with xen 3.4.4) installed with Intel 10GE (ixgbe driver) interface. I test TCP transfer rate between dom0, speed is 7.x Gbps (with Netperf tool). However, I try similar test between two domU, speed drop to 1.x Gbps. domU is running with Centos5.8 (64bit) para-virtualized (yum updated to latest packages). Test 1: dom0 -> dom0 , speed 7.x Gbps Test 2: domU -> dom0 , speed 7.xGbps Test 3: dom0-> domU , speed 1.x Gbps Test 4: domU -> domU ,speed 1.x Gbps Above tests are carried between two physical machines. From above 4 tests, seems bottleneck on domU RX speed. I suspect related to xen network driver issue on domU. Anyone know any optimization can be done on domU? Thanks a lot. Felix _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hi, Thanks your info. I have read this doc and tried some of the optimization on it, but not much significant improvement. About “SR-IOV” , I am not sure how to enable it as this doc is for Xen 4 while I am using xen3.4.4 (there is no “xe” command”). I also searched about “SR-IOV on Xen3”, seems not much info can be found, anyway tried? Thanks. Felix From: xen-users-bounces@lists.xen.org [mailto:xen-users-bounces@lists.xen.org] On Behalf Of Sebastien Riccio Sent: Wednesday, June 06, 2012 6:10 PM To: Felix Chu Cc: xen-users@lists.xensource.com Subject: Re: [Xen-users] About 10GE performane on domU (with xen3.4.4) Hi, Have you already tried thoses suggestions: http://wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide Cheers, Sébastien On 06.06.2012 10:35, Felix Chu wrote: Hi all, I have two Dell R720xd servers dom0 running with Centos 5.8 (with xen 3.4.4) installed with Intel 10GE (ixgbe driver) interface. I test TCP transfer rate between dom0, speed is 7.x Gbps (with Netperf tool). However, I try similar test between two domU, speed drop to 1.x Gbps. domU is running with Centos5.8 (64bit) para-virtualized (yum updated to latest packages). Test 1: dom0 -> dom0 , speed 7.x Gbps Test 2: domU -> dom0 , speed 7.xGbps Test 3: dom0-> domU , speed 1.x Gbps Test 4: domU -> domU ,speed 1.x Gbps Above tests are carried between two physical machines. From above 4 tests, seems bottleneck on domU RX speed. I suspect related to xen network driver issue on domU. Anyone know any optimization can be done on domU? Thanks a lot. Felix _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users _____ 在此邮件中未发现病毒。 检查工具：AVG - www.avg.com 版本：2012.0.2177 / 病毒数据库：2425/5051 - 发布日期：06/06/12 _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
I think for Xen 3.4 you should get in touch with SolarFlare sales. They make Xen ready NICs for a long time and maybe they can help you through it if you get one of the NICs. The cool side is those nics are also the fastest nics on the market. The starting point for using SR-IOV is the Xen option iommu=1 There is also one more if you have an AMD CPU. The bottleneck you're seeing is partially in the Linux bridge. I mean. It's a bridge. Like in those "hubs" we ditched in 1998. What else is there to say. Using OpenVSwitch is the second best solution over getting a better NIC. It will never be the same though and you just replace the hardware price with a few dozen work-hours to add openvswitch to Centos5 :)) Florian 2012/6/11 Felix Chu <felixchu@powerallnetworks.com>:> Hi, > > Thanks your info. I have read this doc and tried some of the optimization on > it, but not much significant improvement. About "SR-IOV" , I am not sure how > to enable it as this doc is for Xen 4 while I am using xen3.4.4 (there is no > "xe" command"). I also searched about "SR-IOV on Xen3", seems not much info > can be found, anyway tried? Thanks. > > > > Felix > > > > From: xen-users-bounces@lists.xen.org > [mailto:xen-users-bounces@lists.xen.org] On Behalf Of Sebastien Riccio > Sent: Wednesday, June 06, 2012 6:10 PM > To: Felix Chu > Cc: xen-users@lists.xensource.com > Subject: Re: [Xen-users] About 10GE performane on domU (with xen3.4.4) > > > > Hi, > > Have you already tried thoses suggestions: > > http://wiki.xen.org/wiki/Network_Throughput_and_Performance_Guide > > Cheers, > Sébastien > > On 06.06.2012 10:35, Felix Chu wrote: > > Hi all, > > > > I have two Dell R720xd servers dom0 running with Centos 5.8 (with xen 3.4.4) > installed with Intel 10GE (ixgbe driver) interface. I test TCP transfer rate > between dom0, speed is 7.x Gbps (with Netperf tool). However, I try similar > test between two domU, speed drop to 1.x Gbps. > > > > domU is running with Centos5.8 (64bit) para-virtualized (yum updated to > latest packages). > > > > Test 1: dom0 -> dom0 , speed 7.x Gbps > > Test 2: domU -> dom0 , speed 7.xGbps > > Test 3: dom0-> domU , speed 1.x Gbps > > Test 4: domU -> domU ,speed 1.x Gbps > > > > Above tests are carried between two physical machines. From above 4 tests, > seems bottleneck on domU RX speed. > > > > I suspect related to xen network driver issue on domU. Anyone know any > optimization can be done on domU? > > > > Thanks a lot. > > Felix > > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xen.org > > http://lists.xen.org/xen-users > > > > ________________________________ > > 在此邮件中未发现病毒。 > 检查工具：AVG - www.avg.com > 版本：2012.0.2177 / 病毒数据库：2425/5051 - 发布日期：06/06/12 > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users-- the purpose of libvirt is to provide an abstraction layer hiding all xen features added since 2006 until they were finally understood and copied by the kvm devs. _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users