Kaushik Bhandankar
2007-Oct-04 15:00 UTC
[Xen-devel] Dom0 crash at 40Mbps Iperf traffic with only 80% CPU utilization ??
Hello I have a 3 node experiment as detailed below to estimate the bridged networking performance in para-virtualized Xen 3.0 on Emulab (https://www.emulab.net/) Here is how the 3-node topology looks (This topology is specified by means of an NS-2 file) topology: _______________________________ | | Node0:eth0 ------|Node1:eth0 Node1:eth1|------ Node2:eth0 |_______________________________| Node0 and Node2 run some standard 2.6.* kernel whereas Node1 runs para-virtualized Xen 3.0 using LVM-created root & swap partitions for DomU (2.6.12-xenU) ******************************************************************* Traffic flow: _____________________________________ | Node1 | | ______________________________ | | | DomU | | | | | | | | eth0 ---------> eth1 | | | |_____|_______________|________| | | | | | | vif1.0 vif1.1 | | | | | | | | | | xenbr0 xenbr1 | | | Dom0 | | | | | | Node0:eth0--|----> eth0 eth1 ---------|-----> Node2:eth0 |____________________________________| Node0:eth0, Node1:DomU:eth0 are in 10.1.1/24 subnet whereas Node1:DomU:eth1 and Node2:eth0 are in 10.1.2/24 subnet (Since Node1:eth0 and Node1:eth1 are attached to bridges, they do not have IP addresses on them and are not ping''able) At Node0:eth0, I set the next hop of all traffic as Node1:DomU:eth0 and at Node2, I set the next hop of all traffic as Node1:DomU:eth1. Now, I try to run an Iperf server at Node2 for UDP traffic (const 64-byte packets size): iperf -s -i 2 -l 64 -u and an Iperf client at Node0 for UDP traffic to node2:eth0 (const 64-bit packet size, vary the Bandwidth from 10Mbps till possibly 80Mbps) iperf -c <IP address of Node2:eth0> -u -l 64 -t 90 -i 2 -w 256k -b 10M Where 10M is the bandwidth of iperf traffic between Node0 and Node2 (in Mbps). I also monitor the CPU usage at Node1 Dom0 using "vmstat 1" at Dom0. When B/W is 36.6M (Mbps), the CPU utilization at Dom0 is about 70-80% but when the B/W is increased to 39.4M (Mbps), the Dom0 crashes in typically 5-15 seconds (M SSH session to Dom0 ''hangs'') but before it crashes, the CPU utiliation is till 75-85%. At this point, I have to again boot Dom0 and the DomU on top of it. Does anybody know why this happens ?? and how can I debug this situation (if at all) ? I am new to the Xen source code so please be as detailed as possible _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel