Filip Sergeys
2005-Apr-07 11:58 UTC
[Xen-users] IO performance testing between dom0 and guests
Hi,
I''m using debian testing as dom0 with xen 2.0.4 compiled from source.
The machine is named intra2.
The guest OS is SuSE 9.1, named staging, and runs a dataserver (maxdb).
The disk layout is as follows:
disk1 160G
disk2 80G
partition /dev/sda3 5G running dom0
partition /dev/sda5 5G running guest; mounted as:
''phy:sda5,sda5,w'';
containing the root installation
partition /dev/md0 software raid0 chunk size 64K build from /dev/sda8
and /dev/sdb1; mounted as ''phy:md0,sda7,w''; containing the
database
My impression is that the database performance is less good compaired to
running native on the system, so I started running iostat -xd 5 to see
what is happening on the disks. Clearly, during queries the iowait is
between 99 and 100% for an unusual long time.
I tried to test read performance with with hdparm -t /dev/sda5 in dom0
...
/dev/sda5 DOM0
Timing buffered disk reads: 66 MB in 3.04 seconds = 21.71 MB/sec
Timing buffered disk reads: 50 MB in 3.06 seconds = 16.34 MB/sec
Timing buffered disk reads: 108 MB in 3.02 seconds = 35.76 MB/sec
Timing buffered disk reads: 112 MB in 3.06 seconds = 36.60 MB/sec
Timing buffered disk reads: 166 MB in 3.01 seconds = 55.15 MB/sec
Timing buffered disk reads: 170 MB in 3.00 seconds = 56.67 MB/sec
Timing buffered disk reads: 172 MB in 3.03 seconds = 56.77 MB/sec
Timing buffered disk reads: 170 MB in 3.02 seconds = 56.29 MB/sec
... but ran into a strange phenomenon when the same command was run in
the guest OS ...
/dev/sda5 DOM1
Timing buffered disk reads: 56 MB in 3.05 seconds = 18.36 MB/sec
Timing buffered disk reads: 94 MB in 3.02 seconds = 31.13 MB/sec
Timing buffered disk reads: 138 MB in 3.04 seconds = 45.39 MB/sec
Timing buffered disk reads: 172 MB in 3.00 seconds = 57.33 MB/sec
Timing buffered disk reads: 208 MB in 3.05 seconds = 68.20 MB/sec
Timing buffered disk reads: 246 MB in 3.07 seconds = 80.13 MB/sec
Timing buffered disk reads: 286 MB in 3.13 seconds = 91.37 MB/sec
Timing buffered disk reads: 322 MB in 3.15 seconds = 102.22 MB/sec
Timing buffered disk reads: 358 MB in 3.12 seconds = 114.74 MB/sec
... it''s getting faster every time. Why is that?
Not getting answers from hdparm, it tried bonnie++ -d /tmp -r 200 -s 400
-n 0 -b -u root on both dom0 and guest. This is the result for dom0
(intra2) root disk
Version 1.03 ------Sequential Output------
--Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite-
-Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec
%CP /sec %CP
intra2 400M 26427 98 51944 12 22950 1 17090 58
51638 3 195.4 0
This is the result for guest(staging)
Version 1.01d ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
staging 400M 132810 96 48009 11 53177 5 120590 99 +++++ +++
623.1 0
DOM0,400M,26427 ,98,51944,12,22950 ,1,17090 ,58 ,51638,3 ,195.4,0
Guest ,400M,132810,96,48009 ,11,53177,5 ,120590,99,+++++,+++,623.1,0
For the raid0 device, these are the results:
Version 1.03 ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
intra2 400M 26442 98 71103 16 31414 5 24701 94 69430 6
316.1 1
Version 1.01d ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
staging 400M 133632 96 69970 17 70488 8 120936 99 +++++ +++
619.5 0
DOM0 , 400M,26442 ,98,71103,16,31414,5,24701 ,94,69430,6 ,316.1,1
staging,400M,133632,96,69970,17,70488,8,120936,99,+++++,+++,619.5,0
Based on what I saw with hdparm, can I trust the results from bonnie++?
If somebody knows how to tackle may IO testing problem, please let me
know
Thanx in advance
Cheers,
Filip.
--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
* System Engineer, Verzekeringen NV *
* www.verzekeringen.be *
* Oostkaai 23 B-2170 Merksem *
* 03/6416673 - 0477/340942 *
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
* System Engineer, Verzekeringen NV *
* www.verzekeringen.be *
* Oostkaai 23 B-2170 Merksem *
* 03/6416673 - 0477/340942 *
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
M.A. Williamson
2005-Apr-07 12:43 UTC
Re: [Xen-users] IO performance testing between dom0 and guests
You should find big improvements in the testing tree - there have been several recent bug fixes relating to block IO. Cheers, Mark On Apr 7 2005, Filip Sergeys wrote:> (Message body was not text: suppressed)_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Filip Sergeys
2005-Apr-07 17:22 UTC
Re: [Xen-users] IO performance testing between dom0 and guests
-----Forwarded Message-----
From: Filip Sergeys <filip.sergeys@verzekeringen.be>
To: mark.williamson@cl.cam.ac.uk
Subject: Re: [Xen-users] IO performance testing between dom0 and guests
Date: 07 Apr 2005 19:20:31 +0200
I downloaded, untarred, build and installed the xen-2.0-testing kernel.
However I still have a small question (actually a few small questions):
This time I used the SuSE original configfile to build the unpriviledged
kernel.
This kernel loads almost everything from modules, so it needs an initrd.
Here are my questions?
Where do build the initrd for the unpriviledged domain?
- in the dom0 under debian
- under the guest os (SuSE), by mounting the guest OS partition,
chroot to it, copy the dom0 /lib/modules/2.6.11-xenU directory from dom0
to guest and then run the mkinitrd?
Where should I store that initrd?
- in /boot from dom0
- in /boot from guest
How do I make clear in /etc/xen/staging (config file for my guest called
staging) that an initrd is needed?
What parameters would you use in your mkinitrd? For what is have read,
bootsplash should be avoided with -s off.
Thanks in advance,
Greetz,
Filip
On Thu, 2005-04-07 at 14:43, M.A. Williamson wrote:
You should find big improvements in the testing tree - there have been
several recent bug fixes relating to block IO.
Cheers,
Mark
On Apr 7 2005, Filip Sergeys wrote:
> (Message body was not text: suppressed)
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
* System Engineer, Verzekeringen NV *
* www.verzekeringen.be *
* Oostkaai 23 B-2170 Merksem *
* 03/6416673 - 0477/340942 *
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
* System Engineer, Verzekeringen NV *
* www.verzekeringen.be *
* Oostkaai 23 B-2170 Merksem *
* 03/6416673 - 0477/340942 *
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Filip Sergeys
2005-Apr-11 07:32 UTC
Re: [Xen-users] IO performance testing between dom0 and guests
I couldn''t get xen-U kernel for SuSE with modules support to work
properly yet, so I build an "all in" kernel for testing.
THe bug fixes in xen-2.0-testing for block IO are TREMENDIOUSLY GREAT:
xen-2.0.4 bonnie++ -d /var/maxdb -r 10 -s 400 -n 0 -u root
--------------------------------------------------------------------
Version 1.01d ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
staging 400M 132810 96 48009 11 53177 5 120590 99 +++++ +++
623.1 0
xen-2.0-testing bonnie++ -d /var/maxdb -r 10 -s 400 -n 0 -u root
--------------------------------------------------------------------------
Version 1.01d ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
staging 400M 134499 98 184713 42 70006 3 115162 99 +++++
+++ 1621 0
staging,400M,134499,98,184713,42,70006,3,115162,99,+++++,+++,1620.6,0,,,,,,,,,,,,,
See the block sequential output: 48009 before, now 184713 !!! wow great
work. (I repeated the tests a few times, and the results are more or
less the same every time)
Also the queries are running way faster, need to do benchmarks on them
too
Next step, get to compile a xen-U kernel for SuSE with modules support
Thanx so far
Greetz
Filip
On Fri, 2005-04-08 at 19:59, Mark Williamson wrote:
Did you get a reply to this?
You want to put an "initrd=" line in your config file.
Chroot seems like the most straightforward solution but you should probably
tell mkinitrd explicitly the name of the SuSE Xen kernel anyhow.
You''ll need
to copy the initrd out into the dom0 filesystem and unmount the guest
filesystem before starting the domain (Xend has to be able to find it,
according to the path specified in the config).
Bootsplash etc won''t matter for a domU, so you can probably just
ignore it.
/me goes to watch Doctor Who now :-)
HTH,
Mark
On Thursday 07 April 2005 18:20, you wrote:
> I downloaded, untarred, build and installed the xen-2.0-testing kernel.
> However I still have a small question (actually a few small questions):
> This time I used the SuSE original configfile to build the
unpriviledged
> kernel.
> This kernel loads almost everything from modules, so it needs an
initrd.
> Here are my questions?
> Where do build the initrd for the unpriviledged domain?
> - in the dom0 under debian
> - under the guest os (SuSE), by mounting the guest OS partition,
> chroot to it, copy the dom0 /lib/modules/2.6.11-xenU directory from
dom0
> to guest and then run the mkinitrd?
>
> Where should I store that initrd?
> - in /boot from dom0
> - in /boot from guest
>
> How do I make clear in /etc/xen/staging (config file for my guest
called
> staging) that an initrd is needed?
>
> What parameters would you use in your mkinitrd? For what is have read,
> bootsplash should be avoided with -s off.
>
> Thanks in advance,
>
> Greetz,
>
> Filip
>
>
> On Thu, 2005-04-07 at 14:43, M.A. Williamson wrote:
>
> You should find big improvements in the testing tree - there have
been
> several recent bug fixes relating to block IO.
>
> Cheers,
> Mark
>
> On Apr 7 2005, Filip Sergeys wrote:
> > (Message body was not text: suppressed)
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
>
> http://lists.xensource.com/xen-users
--
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
* System Engineer, Verzekeringen NV *
* www.verzekeringen.be *
* Oostkaai 23 B-2170 Merksem *
* 03/6416673 - 0477/340942 *
*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users