Hi all,
I have a rhel 5.1 xen server with two rhel 3 ES hvm guests installed. Both
rhel3 guests use an internal xen bridge (xenbr1) which it isn''t binded
to any
physical interface host. On this bridge throughput it is very very poor, only
2.5 Mbs. How can I increase this throughput???
Many thanks.
--
CL Martinez
carlopmart {at} gmail {d0t} com
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
>From experience, RHEL3 is real bad at performance in general (NFS)At the lab I work in, my first recommendation was to move to RHEL4U2. There was almost a 2x performance increase across the network. As for tuning, I did the following (it only works for 2.6+ kernels though) on both my Host and guests (all rhel5.1) The following NFS mount options: hard,intr,vers=3,tcp,wsize=32768,wsize=32768,timeo=600 Here are the changes to the /etc/sysctl.conf file on the Guests (for the host, the last line, sunrpc, is not available, so remove it) net.core.netdev_max_backlog = 3000 net.core.rmem_default = 256960 net.core.rmem_max = 16777216 net.core.wmem_default = 256960 net.core.wmem_max = 16777216 net.core.rmem_default = 65536 net.core.wmem_default = 65536 net.core.rmem_max = 8388608 net.core.wmem_max = 8388608 net.ipv4.tcp_rmem = 4096 87380 4194304 net.ipv4.tcp_wmem = 4096 16384 4194304 net.ipv4.tcp_mem = 4096 4096 4096 sunrpc.tcp_slot_table_entries = 128 Also, add "/sbin/sysctl -p" as the first entry in /etc/init.d/netfs to make sure that the setrtings get read before any NFS mounts take place. For the record, I get 95-102MB/sec each with a simple DD On Jan 6, 2008 5:36 AM, carlopmart <carlopmart@gmail.com> wrote:> Hi all, > > I have a rhel 5.1 xen server with two rhel 3 ES hvm guests installed. Both > rhel3 guests use an internal xen bridge (xenbr1) which it isn''t binded to any > physical interface host. On this bridge throughput it is very very poor, only > 2.5 Mbs. How can I increase this throughput??? > > Many thanks. > -- > CL Martinez > carlopmart {at} gmail {d0t} com > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- --tmac RedHat Certified Engineer #804006984323821 (RHEL4) RedHat Certified Engineer #805007643429572 (RHEL5) Principal Consultant, RABA Technologies 240-373-3926 (office) 301-688-4705 (Lab) 214-279-3926 (eFAX) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Mark Williamson
2008-Jan-07 00:50 UTC
Re: [Xen-users] Increasing throughput on xen bridges
> I have a rhel 5.1 xen server with two rhel 3 ES hvm guests installed. > Both rhel3 guests use an internal xen bridge (xenbr1) which it isn''t binded > to any physical interface host. On this bridge throughput it is very very > poor, only 2.5 Mbs. How can I increase this throughput???The bridge driver does add overhead but I suspect that your poor performance is more likely due to the HVM emulation of the network card than the bridge driver. If there''s a Linux 2.6 kernel available for RHEL 3, you might be able to compile the PV drivers from the "unmodified_drivers" directory in the Xen tree. These would improve performance. There were some patches posted to the list from Virtual Iron that enabled PV drivers to be used under a Linux 2.4-based kernel, which would enable you to compile against a 2.4 kernel in your HVM machine. The above suggestions would mean that you keep using a RHEL3 kernel in the HVM virtual machine but just add PV drivers. Finally, http://xen.org/download/dl_304guest_rhel3.html contains a download link for a Xen-patched 2.4.21 kernel for RHEL 3.8. This is not an official RedHat supported kernel, since it''s been patched by XenSource. Therefore it could perhaps impact your support agreement... But if you converted to a PV domain using it you might improve performance, particularly for network. This suggestion differs in that you would actually be converting from using an HVM domain to using a PV domain with an entirely different kernel. Cheers. Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Mark Williamson wrote:>> I have a rhel 5.1 xen server with two rhel 3 ES hvm guests installed. >> Both rhel3 guests use an internal xen bridge (xenbr1) which it isn''t binded >> to any physical interface host. On this bridge throughput it is very very >> poor, only 2.5 Mbs. How can I increase this throughput??? > > The bridge driver does add overhead but I suspect that your poor performance > is more likely due to the HVM emulation of the network card than the bridge > driver. > > If there''s a Linux 2.6 kernel available for RHEL 3, you might be able to > compile the PV drivers from the "unmodified_drivers" directory in the Xen > tree. These would improve performance. > > There were some patches posted to the list from Virtual Iron that enabled PV > drivers to be used under a Linux 2.4-based kernel, which would enable you to > compile against a 2.4 kernel in your HVM machine. > > The above suggestions would mean that you keep using a RHEL3 kernel in the HVM > virtual machine but just add PV drivers. > > Finally, http://xen.org/download/dl_304guest_rhel3.html contains a download > link for a Xen-patched 2.4.21 kernel for RHEL 3.8. This is not an official > RedHat supported kernel, since it''s been patched by XenSource. Therefore it > could perhaps impact your support agreement... But if you converted to a PV > domain using it you might improve performance, particularly for network. > > This suggestion differs in that you would actually be converting from using an > HVM domain to using a PV domain with an entirely different kernel. > > Cheers. > Mark >Thanks Mark, but i can''t find Virtual Iron''s patches that you say ... Where they are?? -- CL Martinez carlopmart {at} gmail {d0t} com _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Mark Williamson
2008-Jan-08 00:57 UTC
Re: [Xen-users] Increasing throughput on xen bridges
> Thanks Mark, but i can''t find Virtual Iron''s patches that you say ... Where > they are??Discussion here: http://lists.xensource.com/archives/html/xen-devel/2007-12/msg00719.html Patches here: http://lists.xensource.com/archives/html/xen-devel/2007-12/msg00720.html http://lists.xensource.com/archives/html/xen-devel/2007-12/msg00721.html The first patch applies to the Xen source tree, the second applies to the XenLinux tree. The result is that by using the two trees it''s possible to build the unmodified_drivers directories for a 2.4 kernel. In fact it looks like Virtual Iron wrote these to support RHEL3.8. It''s likely to be a bit of a challenge getting this all patched and built right. You may have to download the hg repositories of xen-unstable, although you would only be using the PV drivers, which probably are quite stable. You''d need to patch the repositories then build modules for your target 2.4 kernel (building on the target host probably easiest), then get things configured. Finally, you''d need to configure the guest to use these. I''m not entirely sure if you can boot off the PV drivers but assuming you can, you''d need to build them into the initrd (and possibly exclude the "real" drivers for the hardware so they don''t take over). Cheers, Mark -- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users