RE: [Xen-devel] vif interface dropping 80% packets

Santos, Jose Renato G
Tue, 06 Feb 2007 18:12:39 -0000

Most likely your dom0 CPU is saturated.. It is unrealistic to expect
full throughput of a 10 Gig NIC. I would be surprised if even linux
could keep up with a receive rate of 10 Gb/s. Did you try the same
experiment on native linux? In  my experiments, on a 4-way 2.8 Ghz Xeon,
dom0 consumes ~75% of one cpu for processing receive packets while the
guest consume ~55% of another CPU. It is  expected that your dual CPU
system saturates at a rate slighlty above 1 Ghz. (You are also consuming
extra cycles to drop packets)
You could confirm that the CPU is saturating by running xentop while
running your experiments.
You could avoid dropping packets and get a more realistic experiment by
running a TCP experiment as Ian suggested

Regards

Renato


________________________________

	From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Reese
Faucette
	Sent: Monday, February 05, 2007 4:31 PM
	To: xen-devel@lists.xensource.com
	Subject: [Xen-devel] vif interface dropping 80% packets


	I''m trying to characterize the performance of 10G ethernet to a
user domain.
	The machine has 2 configured ethernet cards, eth0 is a 1G card,
and eth2 is a 10G card.  (eth1 is another 1G card and eth3 is another
10G, but these are not configured)
	I have network bridge set up on eth2 with defaults, nothing
special, just:
	   /etc/xen/scripts/network-bridge start vifnum=2
	I have a user domain that attaches eth1 to this vif, and when I
try netperf tests, I get tons of packet loss in the vif interface.

	vif3.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
	          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
	          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
	          RX packets:165602 errors:0 dropped:0 overruns:0
frame:0
	          TX packets:12982847 errors:0 dropped:48987465
overruns:0 carrier:0
	          collisions:0 txqueuelen:0
	          RX bytes:8679444 (8.2 MiB)  TX bytes:20888548648 (19.4
GiB)

	As can be seen above, vif3.1 drops about 80% of the packets it
should transmit.  Is this expected due to something in the dom0 -> domU
interface just not being able to keep up, or is there something I can do
about it?  It almost looks like the dom0->domU link is throttled at
about 1Gb/s.  I say this because the 10G link is running about 50-60%
capacity when the drop rate is 80%.

	Everything has MTU 1500, btw - I noticed that Xen is pretty
unhappy with an MTU of 9000.

	thanks,
	-reese