Emre ERENOGLU
2008-Mar-24 01:35 UTC
[Xen-users] HVM Linux DomU with PV Drivers - Performance
Hi Guys, In order to compare James GPL PV drivers for Windows apple-to-apple to a counterpart, I decided to compile and install the "unmodified_drivers" from xen-unstable tree to my HVM DomU Linux guest. My Dom0 is Ubuntu 7.10 , xen-3.2 with 2.6.18 Dom0 kernel. DomU is Pardus 2007.3 with it''s own 2.6.18-86 kernel (stock kernel) The compiled and loaded drivers are xen-balloon.ko , xen-platform-pci.ko , xen-vbd.ko and xen-vnif.ko from the Xen 3.2 unmodified_drivers folder. After compiling, I''ve included them in the initramfs and using a special init script, I booted the system off a PV root disk. Boot partition is still ioemu. voyager tmp # mount /dev/xvda1 on / type ext3 (rw,errors=remount-ro) /dev/hda1 on /boot type ext3 (rw) /dev/xvda2 on /home type ext3 (rw,errors=remount-ro) Here''s the output of lsmod voyager tmp # lsmod Module Size Used by nls_utf8 2240 1 cifs 212064 1 serio_raw 7012 0 joydev 9472 0 usbhid 45824 0 lp 11364 0 tsdev 7680 0 i2c_piix4 8332 0 i2c_core 19968 1 i2c_piix4 parport_pc 36804 1 parport 33896 2 lp,parport_pc xen_vnif 30768 0 xen_balloon 15496 1 xen_vnif xen_vbd 16496 2 uhci_hcd 21900 0 usbcore 116964 3 usbhid,uhci_hcd xen_platform_pci 68632 3 xen_vnif,xen_balloon,xen_vbd,[permanent] As seen, no physical NIC drivers are loaded. Then it comes to the test. I use iperf for this purpose. Note that I was remotely connected to the DomU through XDMCP, so this may have some little impact even though there were no important screen updates. *Packet flow from Dom0 to HVM-DomU-w/PV-Drivers : 2.05 Gbits/sec* root@xenroot:~# iperf -c 192.168.1.7 -p 5002 -t 20 ------------------------------------------------------------ Client connecting to 192.168.1.7, TCP port 5002 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.2 port 40777 connected with 192.168.1.7 port 5002 [ 3] 0.0-20.0 sec 4.76 GBytes 2.05 Gbits/sec *Packet flow from HVM-DomU-w/PV-Drivers to Dom0: 3,3 Gbits/sec* voyager tmp # ./iperf -c 192.168.1.2 ------------------------------------------------------------ Client connecting to 192.168.1.2, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.7 port 34685 connected with 192.168.1.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 3.84 GBytes 3.30 Gbits/sec *Packet flow from HVM-DomU-w/PV-Drivers to Dom0, when there''s also a packet flow from Dom0 to DomU: 3,12 Gbits/sec* voyager tmp # ./iperf -c 192.168.1.2 -t 20 ------------------------------------------------------------ Client connecting to 192.168.1.2, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.7 port 60230 connected with 192.168.1.2 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-20.0 sec 7.26 GBytes 3.12 Gbits/sec *Packet flow from Dom0 to HVM-DomU-w/PV-Drivers, when there''s also a packet flow from DomU to Dom0: 62,9 Mbits/sec* root@xenroot:~# iperf -c 192.168.1.7 -p 5002 -t 30 ------------------------------------------------------------ Client connecting to 192.168.1.7, TCP port 5002 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.2 port 54553 connected with 192.168.1.7 port 5002 [ 3] 0.0-30.0 sec 225 MBytes 62.9 Mbits/sec * So in summary: * Dom0 --> DomU = 2.05 Gbits/sec DomU --> Dom0: 3,3 Gbits/sec DomU --> Dom0, simulatenous with Dom0 --> DomU: 3,12 Gbits/sec Dom0 --> DomU, simultaneous with DomU --> Dom0: 62,9 Gbits/sec *Conclusion:* - PV Drivers in HVM Linux setup are really fast - Dom0 to DomU traffic is severely impacted when there''s heavy traffic from DomU to Dom0. (don''t know why). Hope this helps for tuning James'' GPL PV Drivers for Windows. Best regards, Emre Erenoglu _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Emre ERENOGLU
2008-Mar-24 02:15 UTC
[Xen-users] Re: HVM Linux DomU with PV Drivers - Performance
Hi again, this time with disk benchmark.
I forgot to mention that this system is a Core2Duo E6300 1.83 Ghz running at
2.8 Ghz overclocked.
For the disk benchmark, I used Bonnie 1.4 (not bonnie++).
Machine: Pardus HVM DomU with PV drivers for root disk
Test Disk: /dev/xvda1 --> a LVM volume in Dom0 on RAID stripe set (2 disks)
emre@voyager bonnie $ ./Bonnie -d /tmp -s 1500 -m Pardus
Bonnie 1.4: File ''/tmp/Bonnie.4911'', size: 1572864000,
volumes: 1
Writing with putc()... done: 74270 kB/s 87.8 %CPU
Rewriting... done: 43062 kB/s 4.9 %CPU
Writing intelligently... done: 81596 kB/s 10.0 %CPU
Reading with getc()... done: 51825 kB/s 68.4 %CPU
Reading intelligently... done: 95262 kB/s 5.3 %CPU
Seeker 1...Seeker 2...Seeker 3...start ''em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
Pardus 1*1500 74270 87.8 81596 10.0 43062 4.9 51825 68.4 95262 5.3 221.2
0.1
Machine: Ubuntu - Dom0
Test Disk: Same DomU disk, /dev/mapper/os-pardus but mounted on /mnt in
Dom0 after DomU is shut down.
emre@xenroot:/tmp/bonnie$ ./Bonnie -d /mnt/tmp/ -s 1500 -m Ubuntu
Bonnie 1.4: File ''/mnt/tmp//Bonnie.26707'', size: 1572864000,
volumes: 1
Writing with putc()... done: 65001 kB/s 93.1 %CPU
Rewriting... done: 48770 kB/s 3.7 %CPU
Writing intelligently... done: 158724 kB/s 27.1 %CPU
Reading with getc()... done: 41054 kB/s 70.4 %CPU
Reading intelligently... done: 107393 kB/s 3.4 %CPU
Seeker 1...Seeker 2...Seeker 3...start ''em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
Ubuntu 1*1500 65001 93.1158724 27.1 48770 3.7 41054 70.4107393 3.4 290.5
0.1
Machine: Ubuntu - Dom0
Test Disk: /dev/mapper/os-data mounted on /d with a lot of free disk space
emre@xenroot:/tmp/bonnie$ ./Bonnie -d /d -s 1500 -m Ubuntu
Bonnie 1.4: File ''/d/Bonnie.26712'', size: 1572864000, volumes:
1
Writing with putc()... done: 64011 kB/s 90.9 %CPU
Rewriting... done: 50202 kB/s 3.4 %CPU
Writing intelligently... done: 183438 kB/s 28.2 %CPU
Reading with getc()... done: 40139 kB/s 69.5 %CPU
Reading intelligently... done: 111579 kB/s 3.9 %CPU
Seeker 1...Seeker 2...Seeker 3...start ''em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
Ubuntu 1*1500 64011 90.9183438 28.2 50202 3.4 40139 69.5111579 3.9 314.7
0.0
Machine: Ubuntu - Dom0
Test Disk: Native partition, no RAID, /dev/sdb2 mounted on /mnt (lower
results due to non-RAID)
emre@xenroot:/tmp/bonnie$ sudo ./Bonnie -d /mnt -s 1500 -m Ubuntu
Bonnie 1.4: File ''/mnt/Bonnie.26738'', size: 1572864000,
volumes: 1
Writing with putc()... done: 59680 kB/s 83.8 %CPU
Rewriting... done: 22911 kB/s 1.3 %CPU
Writing intelligently... done: 108945 kB/s 16.0 %CPU
Reading with getc()... done: 31999 kB/s 52.6 %CPU
Reading intelligently... done: 51561 kB/s 0.9 %CPU
Seeker 1...Seeker 2...Seeker 3...start ''em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
Ubuntu 1*1500 59680 83.8108945 16.0 22911 1.3 31999 52.6 51561 0.9 283.6
0.1
*Conclusion:
*The HVM Linux DomU running with unmodified_drivers (PV drivers for HVM
guest, xen-vbd) seems to perform as good as Dom0 itself for disk I/O.
Best regards,
Emre
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
jim burns
2008-Mar-24 02:39 UTC
Re: [Xen-users] Re: HVM Linux DomU with PV Drivers - Performance
On Sunday 23 March 2008 10:15:08 pm Emre ERENOGLU wrote:> Conclusion: > The HVM Linux DomU running with unmodified_drivers (PV drivers for HVM > guest, xen-vbd) seems to perform as good as Dom0 itself for disk I/O.And similarly for net i/o. Well, now no one can complain that we don''t have real numbers for other PV drivers! Thanx, Emre. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users