All,
With some help I''m able to run Xen 4.0.1 rc3 and bring up
the domUs. Earlier in a local network (Cisco switched), I''ve measured
114 MB/s network throughput (Rx side) using a single port of a Gbit card. Now
with a domU-domU test, I''m only getting around 30 MB/s. (2.6.32.15
kernel, 4.0.1-rc3 Xen). I''m yet to run Xenoprof. Do you guys suggest
any obvious thing I might check like,
1. CPU / SMP scalability config. options in dom0 or any other kernel config
option in that matter.
2. Network driver (igb)
lsmod | grep igb
igb 82854 0
dca 5816 1 igb
Does that mean igb is not being used.
3. lsmod ...
xen_netback 105755 0 [permanent]
Probably I had compiled them statically. Is that bad or could be the
reason?
4. Earlier bare-machine test
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp ->
172.16.30.206
ttcp-t: socket
ttcp-t: nodelay
ttcp-t: connect
ttcp-t: 16777216 bytes in 0.14 real seconds = 113394.28 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.07, calls/sec = 14174.29
ttcp-t: 0.0user 0.0sys 0:00real 7% 0i+0d 370maxrss 0+2pf 317+0csw
Rx:
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp
ttcp-r: socket
ttcp-r: accept from 172.16.30.96
ttcp-r: 16777216 bytes in 0.15 real seconds = 112929.24 KB/sec +++ := 110 MB/s =
880 Mb/s
ttcp-r: 6157 I/O calls, msec/call = 0.02, calls/sec = 42438.07
ttcp-r: 0.0user 0.1sys 0:00real 85% 0i+0d 382maxrss 0+2pf 5577+1csw
Now,
Tx:
root@EC2-TEST-2:/LinuxTTCP# ./ttcp -t 172.16.30.207
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp ->
172.16.30.207
ttcp-t: socket
ttcp-t: nodelay
ttcp-t: connect
ttcp-t: 16777216 bytes in 0.71 real seconds = 23092.58 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.35, calls/sec = 2886.57
ttcp-t: 0.0user 0.7sys 0:00real 98% 0i+0d 416maxrss 0+2pf 3+3csw
Rx:
root@EC1-TEST-1:/LinuxTTCP# ./ttcp -r
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp
ttcp-r: socket
ttcp-r: accept from 172.16.30.96
ttcp-r: 16777216 bytes in 0.71 real seconds = 23055.21 KB/sec +++
<<<<<<<<<<<<<< BAD
ttcp-r: 2890 I/O calls, msec/call = 0.25, calls/sec = 4066.75
ttcp-r: 0.0user 0.3sys 0:00real 52% 0i+0d 440maxrss 0+2pf 2283+1csw
As you can see [ 42438.07 ] VS [ 4066.75 ] calls/sec probably indicates hyp/dom0
kernel is the culprit. Any early pointers is greatly appreciated.
-Thanks
Deb
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
I''m not at all interested in domU as I''ve figured out
dom0-dom0 has similar values of throughput. So netfront stuff is not the
culprit.
-Thanks
Deb
________________________________________
From: Debasish Bose
Sent: Wednesday, June 30, 2010 7:56 PM
To: xen-users@lists.xensource.com
Subject: Xen Network Performance
All,
With some help I''m able to run Xen 4.0.1 rc3 and bring up
the domUs. Earlier in a local network (Cisco switched), I''ve measured
114 MB/s network throughput (Rx side) using a single port of a Gbit card. Now
with a domU-domU test, I''m only getting around 30 MB/s. (2.6.32.15
kernel, 4.0.1-rc3 Xen). I''m yet to run Xenoprof. Do you guys suggest
any obvious thing I might check like,
1. CPU / SMP scalability config. options in dom0 or any other kernel config
option in that matter.
2. Network driver (igb)
lsmod | grep igb
igb 82854 0
dca 5816 1 igb
Does that mean igb is not being used.
3. lsmod ...
xen_netback 105755 0 [permanent]
Probably I had compiled them statically. Is that bad or could be the
reason?
4. Earlier bare-machine test
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp ->
172.16.30.206
ttcp-t: socket
ttcp-t: nodelay
ttcp-t: connect
ttcp-t: 16777216 bytes in 0.14 real seconds = 113394.28 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.07, calls/sec = 14174.29
ttcp-t: 0.0user 0.0sys 0:00real 7% 0i+0d 370maxrss 0+2pf 317+0csw
Rx:
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp
ttcp-r: socket
ttcp-r: accept from 172.16.30.96
ttcp-r: 16777216 bytes in 0.15 real seconds = 112929.24 KB/sec +++ := 110 MB/s =
880 Mb/s
ttcp-r: 6157 I/O calls, msec/call = 0.02, calls/sec = 42438.07
ttcp-r: 0.0user 0.1sys 0:00real 85% 0i+0d 382maxrss 0+2pf 5577+1csw
Now,
Tx:
root@EC2-TEST-2:/LinuxTTCP# ./ttcp -t 172.16.30.207
ttcp-t: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp ->
172.16.30.207
ttcp-t: socket
ttcp-t: nodelay
ttcp-t: connect
ttcp-t: 16777216 bytes in 0.71 real seconds = 23092.58 KB/sec +++
ttcp-t: 2048 I/O calls, msec/call = 0.35, calls/sec = 2886.57
ttcp-t: 0.0user 0.7sys 0:00real 98% 0i+0d 416maxrss 0+2pf 3+3csw
Rx:
root@EC1-TEST-1:/LinuxTTCP# ./ttcp -r
ttcp-r: buflen=8192, nbuf=2048, align=16384/0, port=5001 tcp
ttcp-r: socket
ttcp-r: accept from 172.16.30.96
ttcp-r: 16777216 bytes in 0.71 real seconds = 23055.21 KB/sec +++
<<<<<<<<<<<<<< BAD
ttcp-r: 2890 I/O calls, msec/call = 0.25, calls/sec = 4066.75
ttcp-r: 0.0user 0.3sys 0:00real 52% 0i+0d 440maxrss 0+2pf 2283+1csw
As you can see [ 42438.07 ] VS [ 4066.75 ] calls/sec probably indicates hyp/dom0
kernel is the culprit. Any early pointers is greatly appreciated.
-Thanks
Deb
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users