i installed xen3.0.3 on debian sarge with 2 vms (also sarge), but i
was getting very bad inter-vm network performance (like 15-20Mbps max)
i converted the network setup from bridged to routed, and applied the
following kernel parameter tweaks on dom0 and domUs too:
echo 256960 > /proc/sys/net/core/rmem_default
echo 256960 > /proc/sys/net/core/wmem_default
echo 2569600 > /proc/sys/net/core/rmem_max
echo 2569600 > /proc/sys/net/core/wmem_max
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 1 > /proc/sys/net/ipv4/tcp_sack
echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
echo "40960 873800 1747600" > /proc/sys/net/ipv4/tcp_rmem
echo "40960 163840 1310720" > /proc/sys/net/ipv4/tcp_wmem
netperf-tcp output:
65: 12288 bytes 419 times --> 606.15 Mbps in 154.67 usec
66: 12291 bytes 431 times --> 624.73 Mbps in 150.10 usec
performance is acceptable until here, xm top shows full cpu
utilization, but after this point it drops badly. cpu utilization also
drops to <10%
67: 16381 bytes 222 times --> 28.96 Mbps in 4315.32 usec
68: 16384 bytes 11 times --> 583.25 Mbps in 214.32 usec
69: 16387 bytes 233 times --> 22.51 Mbps in 5553.60 usec
70: 24573 bytes 9 times --> 5.64 Mbps in 33220.50 usec
71: 24576 bytes 3 times --> 670.06 Mbps in 279.83 usec
72: 24579 bytes 238 times --> 23.08 Mbps in 8123.88 usec
73: 32765 bytes 4 times --> 714.72 Mbps in 349.76 usec
74: 32768 bytes 142 times --> 22.70 Mbps in 11011.61 usec
75: 32771 bytes 4 times --> 47.53 Mbps in 5259.99 usec
76: 49149 bytes 9 times --> 814.98 Mbps in 460.11 usec
77: 49152 bytes 144 times --> 145.25 Mbps in 2581.81 usec
78: 49155 bytes 25 times --> 887.54 Mbps in 422.54 usec
79: 65533 bytes 78 times --> 30.32 Mbps in 16489.71 usec
80: 65536 bytes 3 times --> 18.78 Mbps in 26617.32 usec
81: 65539 bytes 3 times --> 10.72 Mbps in 46664.85 usec
82: 98301 bytes 3 times --> 7.50 Mbps in 99982.16 usec
83: 98304 bytes 3 times --> 9.38 Mbps in 79990.15 usec
dom0 vif interfaces show some dropped packets, domUs do not.
vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:192.168.100.128 Bcast:0.0.0.0 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1457121 errors:0 dropped:0 overruns:0 frame:0
TX packets:1458240 errors:0 dropped:970 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2051710970 (1.9 GiB) TX bytes:2069975753 (1.9 GiB)
vif2.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:192.168.99.128 Bcast:0.0.0.0 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1459850 errors:0 dropped:0 overruns:0 frame:0
TX packets:1457278 errors:0 dropped:924 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2052283308 (1.9 GiB) TX bytes:2069732297 (1.9 GiB)
dom0 has 512, domUs have 256M ram allocated, there is now swapping
involved, host is dual-xeon 3200, 4G ram, high-mem enabled
--
Tibor Veres
tibor.veres@gmail.com
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users