Hi there,
I have setup Debian etch host with 4 etch and 4 lenny guests on one of my
servers.
Dell PowerEdge 2970:2 x AMD Dual Core Opteron processors, 2GHz, 2
MB L2 Cache.8 GB RAM (4 x 2 GB)4 x 146 GB Hard DisksDual Port
Gigabit Ethernet NIC10 Gbps PCI EthernetI am performing different
experiments for network throughput analysis of virtual bridge. For this purpose
I am using a network benchmark. Most of the results are comparable to the other
plate forms like fedora8 except one. That is While sending packets from Guest1
to Gueat2 on the same physical machine the throughput varies from 10 to 40 Mbps.
But on 5th, 6th turn it increases all of a sudden with no change in
specifications. It reaches almost 1800Mbps.
What could be the reason for it? Here is one more thing that I observed about
it, the throughput is ok (around 1000 to 3000 Mbps) if the UDP is used instead
of TCP. And even if we talk about Guest to Guest communication on different
physical machines.
Why is that so... What is the cause of this variation in network throughput?
Regards,
Fasiha Ashraf
Looking for local information? Find it on Yahoo! Local
http://in.local.yahoo.com/
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
I have tested Xen guest <-> guest communication on a single physical host using the latest Xen 3.5 unstable. I have a similar hardware configuration to what you have. I used the netperf to measure TCP and UDP throughput . I could achieve about 2.9Gbit/s between the two guests, and the results were stable. The guests were connected via a single bridge, and both were running PV guest kernels. What sort of kernel are you running, and what tool are you using to measure the throughput? Achieving only 10-40Mbit/s seems very low. How do you allocate your memory to the guests/Dom0? Have you also measure CPU utilization? What does it show? Are you also using iptables/ebtables or some sort of packet filtering in Dom0? What Xen version are you using? From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Fasiha Ashraf Sent: 30 June 2009 08:34 To: xen-users@lists.xensource.com Subject: [Xen-users] Xen-bridge throughput Hi there, I have setup Debian etch host with 4 etch and 4 lenny guests on one of my servers. Dell PowerEdge 2970: * 2 x AMD Dual Core Opteron processors, 2GHz, 2 MB L2 Cache. * 8 GB RAM (4 x 2 GB) * 4 x 146 GB Hard Disks * Dual Port Gigabit Ethernet NIC * 10 Gbps PCI Ethernet I am performing different experiments for network throughput analysis of virtual bridge. For this purpose I am using a network benchmark. Most of the results are comparable to the other plate forms like fedora8 except one. That is While sending packets from Guest1 to Gueat2 on the same physical machine the throughput varies from 10 to 40 Mbps. But on 5th, 6th turn it increases all of a sudden with no change in specifications. It reaches almost 1800Mbps. What could be the reason for it? Here is one more thing that I observed about it, the throughput is ok (around 1000 to 3000 Mbps) if the UDP is used instead of TCP. And even if we talk about Guest to Guest communication on different physical machines. Why is that so... What is the cause of this variation in network throughput? Regards, Fasiha Ashraf ________________________________ See the Web''s breaking stories, chosen by people like you. Check out Yahoo! Buzz<http://in.rd.yahoo.com/tagline_buzz_1/*http:/in.buzz.yahoo.com/>. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Wed, Jul 1, 2009 at 4:43 PM, Fischer, Anna<anna.fischer@hp.com> wrote:> Achieving only 10-40Mbit/s seems very low. > How do you allocate your memory to the guests/Dom0? Have you also measure > CPU utilization? What does it show?Assigning a dedicated CPU or core to dom0 should help. The default scheduler should be smart enough to allocate CPUs dynamically, but if you occasionally experience low network throughput while CPU usage (from xm top) on one CPU is high, you might want to give dedicated CPU for dom0 and prevent all other domUs from using it. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
debian:/home/xen# xm vcpu-list Name ID VCPUs CPU State Time(s) CPU Affinity D4G1 1 0 1 -b- 4.6 any cpu D4G4 4 0 3 -b- 4.5 any cpu D5G2 6 0 0 -b- 807.3 any cpu Domain-0 0 0 2 r-- 146.5 any cpu Domain-0 0 1 - --p 3.7 any cpu Domain-0 0 2 - --p 2.7 any cpu Domain-0 0 3 - --p 2.5 any cpu debian:/home/xen# cat /proc/interrupts CPU0 8: 1 Phys-irq rtc 9: 27 Phys-irq acpi 10: 0 Phys-irq libata 11: 113 Phys-irq ohci_hcd:usb1, ohci_hcd:usb2, ehci_hcd:usb3 14: 85260 Phys-irq ide0 18: 673428 Phys-irq megasas, eth1, peth0 256: 709397 Dynamic-irq timer0 257: 1501 Dynamic-irq resched0 258: 32 Dynamic-irq callfunc0 259: 1 Dynamic-irq vif1.0 260: 4076 Dynamic-irq blkif-backend 261: 1 Dynamic-irq blkif-backend 268: 9370 Dynamic-irq xenbus 269: 0 Dynamic-irq console 270: 1 Dynamic-irq vif4.0 271: 92 Dynamic-irq blkif-backend 272: 1 Dynamic-irq blkif-backend 276: 25750 Dynamic-irq blkif-backend 277: 1 Dynamic-irq blkif-backend 278: 11258 Dynamic-irq vif6.0 NMI: 0 LOC: 0 ERR: 0 MIS: 0 Regards, Fasiha Ashraf --- On Sat, 4/7/09, Fajar A. Nugraha <fajar@fajar.net> wrote: From: Fajar A. Nugraha <fajar@fajar.net> Subject: Re: [Xen-users] Xen-bridge throughput To: "Fasiha Ashraf" <feehapk@yahoo.co.in> Date: Saturday, 4 July, 2009, 4:11 AM On Fri, Jul 3, 2009 at 2:22 PM, Fasiha Ashraf<feehapk@yahoo.co.in> wrote:> Hi Fajar, > I tried what you suggest me to. After pinning the throughput still varies > from 10 to 300Mbps. > Things are getting out of my mind, where the problem could be? > PFA, some basic bridge specs in dom0can you post the output of xm vcpu-list cat /proc/interrupts -- Fajar Looking for local information? Find it on Yahoo! Local http://in.local.yahoo.com/ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Sat, Jul 4, 2009 at 2:04 PM, Fasiha Ashraf<feehapk@yahoo.co.in> wrote:> debian:/home/xen# xm vcpu-list > Name ID VCPUs CPU State Time(s) CPU > Affinity > D4G1 1 0 1 -b- > 4.6 any cpu > D4G4 4 0 3 -b- 4.5 > any cpu > D5G2 6 0 0 -b- > 807.3 any cpu > Domain-0 0 0 2 r-- > 146..5 any cpu > Domain-0 0 1 - --p 3.7 > any cpu > Domain-0 0 2 - --p 2.7 > any cpu > Domain-0 0 3 - --p 2.5 > any cpuHmmm ... At this point I have to say I don't know. Perhaps it's kernel-related. I'm mainly using RHEL5.3 for production with RH's kernel 2.6.18, and getting stable throughput domU <-> domU (tested with iperf) about 2 Gbps. -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
veth1... veth3 are just 3 unused frontend network interfaces for Domain0.
They are created automatically when we setup xen-bridge.
Regards,
Fasiha Ashraf
--- On Mon, 6/7/09, Fischer, Anna <anna.fischer@hp.com> wrote:
From: Fischer, Anna <anna.fischer@hp.com>
Subject: RE: [Xen-users] Xen-bridge throughput
To: "Fasiha Ashraf" <feehapk@yahoo.co.in>
Date: Monday, 6 July, 2009, 1:51 PM
What are the vethX device in your setup? You are using VLANs?
This might have some additional overhead, so you might want to try and disable
those.
From: Fasiha Ashraf [mailto:feehapk@yahoo.co.in]
Sent: 03 July 2009 08:18
To: Fischer, Anna
Subject: RE: [Xen-users] Xen-bridge throughput
Yeah... I am measuring CPU utilization with different
ways. xm top, top and even i observed it using vmstat
1
to monitor it every sec while running netperf in guests. CPU parameter Idle
just vary from 100% to 97%. That shows in communication only a little
fraction of cpu is being utilized that is 1 to 3%.
PFA, the dom0 specs in detail.
Regards,
Fasiha Ashraf
--- On Thu, 2/7/09, Fischer, Anna <anna.fischer@hp.com>
wrote:
From: Fischer, Anna <anna.fischer@hp.com>
Subject: RE: [Xen-users] Xen-bridge throughput
To: "Fasiha Ashraf" <feehapk@yahoo.co.in>
Date: Thursday, 2 July, 2009, 5:39 PM
Such
a low CPU utilization is not normal. When I run netperf, the CPU is almost
maxed out. Do you measure it using xm top?
Xen
3.0.3 is very old. You should consider upgrading to the latest Xen which is
Xen 3.4!
Can
you show what “brctl show” and “ifconfig” and “ifconfig –a” looks like on
your system? It looks like you are not using a bridge but routing for your
guests?
From: Fasiha
Ashraf [mailto:feehapk@yahoo.co.in]
Sent: 02 July 2009 12:36
To: Fischer, Anna
Subject: RE: [Xen-users] Xen-bridge throughput
Hi,
Thanks alot for your concern. The guests were connected through a single
bridge and bother are running PV in my case as well. I am Using
Linux-2.6.18-6-xen-686 kernel. And the
tool to measure throughput is same as yours(Netperf).
I have told you that 8GB RAM (4 x 2 GB)
is available in Dom0. While creating guest VMs I used this command to
create a PV guest.
xen-create-image
--hostname=D4G1 --size=7Gb --swap=1Gb \--ip=10.11.21.101
--netmask=255.255.254.0 --gateway=10.11.21.3 \--dir=/home/xen
\--debootstrap --dist=etch --mirror=http://ftp.us.debian.org/debian/
--passwd
Below is some general
information about created guest
General Information
--------------------
Hostname : D4G1
Distribution : etch
Fileystem Type : ext3
Size Information
----------------
Image size : 7Gb
Swap size : 1Gb
Image type : sparse
Memory size : 500Mb
Kernel path :
/boot/vmlinuz-2.6.18-6-xen-686
Initrd path :
/boot/initrd.img-2.6.18-6-xen-686
See! I allocated memory once(statically) and VM uses only allocated amount
512RAM.. About CPU utilization, yes I
have measured it while running netperf and observed it very low. Only 1% to
3% CPU was utilized during
communication. Xen version is xen-hypervisor-3.0.3-1-i386.
I have flushed all type of packet filtering in Dom0 using this
command iptables -F. There is some
strange thing that I observed about it. When I check the TX and RX packets
by xm top its about 5.6GB. But the
netperf throughput results are 15.26Mbps.
I am running it for 300sec (5min), to get the stable results.
I am not getting whats wrong with it...
Regards,
Fasiha Ashraf
--- On Wed, 1/7/09, Fischer, Anna <anna.fischer@hp.com>
wrote:
From: Fischer, Anna <anna.fischer@hp.com>
Subject: RE: [Xen-users] Xen-bridge throughput
To: "Fasiha Ashraf" <feehapk@yahoo..co.in>
Cc: "xen-users@lists.xensource.com"
<xen-users@lists.xensource..com>
Date: Wednesday, 1 July, 2009, 2:43 PM
I
have tested Xen guest <-> guest communication on a single physical
host using the latest Xen 3.5 unstable.. I have a similar hardware
configuration to what you have. I used the netperf to measure TCP and UDP
throughput . I could achieve about 2.9Gbit/s between the two guests, and
the results were stable. The guests were connected via a single bridge, and
both were running PV guest kernels. What sort of kernel are you running,
and what tool are you using to measure the throughput? Achieving only
10-40Mbit/s seems very low. How do you allocate your memory to the
guests/Dom0? Have you also measure CPU utilization? What does it show? Are
you also using iptables/ebtables or some sort of packet filtering in Dom0?
What Xen version are you using?
From:
xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Fasiha
Ashraf
Sent: 30 June 2009 08:34
To: xen-users@lists.xensource.com
Subject: [Xen-users] Xen-bridge throughput
Hi
there,
I have setup Debian etch host with 4 etch and 4 lenny guests on one of my
servers.
Dell PowerEdge 2970:
2 x AMD Dual Core Opteron processors,
2GHz, 2 MB L2 Cache.
8 GB RAM (4 x 2 GB)
4 x 146 GB Hard Disks
Dual Port Gigabit Ethernet NIC
10 Gbps PCI Ethernet
I
am performing different experiments for network throughput analysis of
virtual bridge. For this purpose I am using a network benchmark. Most of
the results are comparable to the other plate forms like fedora8 except
one. That is While sending packets from Guest1 to Gueat2 on the same
physical machine the throughput varies from 10 to 40 Mbps. But on 5th,
6th turn it increases all of a sudden with no change in specifications..
It reaches almost 1800Mbps.
What could be the reason for it? Here is one more thing that I observed
about it, the throughput is ok (around 1000 to 3000 Mbps) if the UDP is
used instead of TCP. And even if we talk about Guest to Guest
communication on different physical machines.
Why is that so... What is the cause of this variation in network
throughput?
Regards,
Fasiha Ashraf
See the Web''s
breaking stories, chosen by people like you. Check out Yahoo! Buzz.
Love Cricket?
Check out live scores, photos, video highlights and more. Click here.
See
the Web''s breaking stories, chosen by people like you. Check out Yahoo!
Buzz.
Looking for local information? Find it on Yahoo! Local
http://in.local.yahoo.com/
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users