Hi all, I've been working on a set of easy-to-run benchmarks for hypervisor optimization called virtbench, and the latest commit adds Xen support. http://ozlabs.org/~rusty/virtbench http://ozlabs.org/~rusty/virtbench/?archive/tip.tar.bz2>From the README:# Build the code make # Set the guest kernel location and maybe more vi ./SETTINGS # Run the benchmark (local mode) ./virtbench local # Run the benchmark (kvm mode) sudo ./virtbench kvm # Run the benchmark (Xen mode) sudo ./virtbench xen # Run the benchmark (lguest mode) sudo ./virtbench lguest Feedback welcome! Rusty.
Jan Michael
2007-May-15 07:44 UTC
Re: [Xen-devel] [ANNOUNCE] virtbench now has xen support
Hi Rusty, thanks for your work. I tried your benchmark on my XEN installation. <xm info> release : 2.6.18-1.2835.slc4xen version : #1 SMP Wed Nov 29 21:05:58 CET 2006 machine : i686 nr_cpus : 2 nr_nodes : 1 sockets_per_node : 2 cores_per_socket : 1 threads_per_core : 1 cpu_mhz : 2800 hw_caps : bfebfbff:20000000:00000000:00000180:0000641d total_memory : 2047 free_memory : 1024 xen_major : 3 xen_minor : 0 xen_extra : .3-rc5-1.2835.s xen_caps : xen-3.0-x86_32p xen_pagesize : 4096 platform_params : virt_start=0xf5800000 xen_changeset : unavailable cc_compiler : gcc version 3.4.6 20060404 (Red Hat 3.4.6-3) cc_compile_by : root cc_compile_date : Wed Nov 29 20:56:00 CET 2006 xend_config_format : 2 </xm info> and ended up in the following error. <make error> [root@lxdev19 virtbench-bd8e67e50775]# make cc -g -Wall -Wmissing-prototypes -DNUM_MACHINES=4 -o virtbench server.c results.c stdrusty.c talloc.c micro/context-switch.c micro/ cow.c micro/exec.c micro/fork.c micro/host-net-bandwidth.c micro/int- syscall.c micro/libc-syscall.c micro/memburn.c micro/pio.c micro/pte- update.c micro/read-bandwidth.c micro/read-latency.c micro/vmcall.c inter/bandwidth.c inter/pingpong.c inter/sendfile.c inter/udp- bandwidth.c micro/pte-update.c: In function `do_pte_update'': micro/pte-update.c:45: error: too many arguments to function `mremap'' make: *** [virtbench] Error 1 </make error> So first thing I would like to know if my XEN installation is compatible with your benchmark suite and if not which are the requirements? Thank you for your help and support, Jan Michael On 15.05.2007, at 04:38, Rusty Russell wrote:> Hi all, > > I''ve been working on a set of easy-to-run benchmarks for hypervisor > optimization called virtbench, and the latest commit adds Xen support. > > http://ozlabs.org/~rusty/virtbench > http://ozlabs.org/~rusty/virtbench/?archive/tip.tar.bz2 > >> From the README: > > # Build the code > make > # Set the guest kernel location and maybe more > vi ./SETTINGS > # Run the benchmark (local mode) > ./virtbench local > # Run the benchmark (kvm mode) > sudo ./virtbench kvm > # Run the benchmark (Xen mode) > sudo ./virtbench xen > # Run the benchmark (lguest mode) > sudo ./virtbench lguest > > Feedback welcome! > Rusty. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Anthony Liguori
2007-May-15 19:50 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Jan Michael wrote:> Hi Rusty, > > thanks for your work. I tried your benchmark on my XEN installation. > > <xm info> > release : 2.6.18-1.2835.slc4xen > version : #1 SMP Wed Nov 29 21:05:58 CET 2006 > machine : i686 > nr_cpus : 2 > nr_nodes : 1 > sockets_per_node : 2 > cores_per_socket : 1 > threads_per_core : 1 > cpu_mhz : 2800 > hw_caps : bfebfbff:20000000:00000000:00000180:0000641d > total_memory : 2047 > free_memory : 1024 > xen_major : 3 > xen_minor : 0 > xen_extra : .3-rc5-1.2835.s > xen_caps : xen-3.0-x86_32p > xen_pagesize : 4096 > platform_params : virt_start=0xf5800000 > xen_changeset : unavailable > cc_compiler : gcc version 3.4.6 20060404 (Red Hat 3.4.6-3) > cc_compile_by : root > cc_compile_date : Wed Nov 29 20:56:00 CET 2006 > xend_config_format : 2 > </xm info> > > and ended up in the following error. > > <make error> > [root@lxdev19 virtbench-bd8e67e50775]# make > cc -g -Wall -Wmissing-prototypes -DNUM_MACHINES=4 -o virtbench server.c > results.c stdrusty.c talloc.c micro/context-switch.c micro/cow.c > micro/exec.c micro/fork.c micro/host-net-bandwidth.c micro/int-syscall.c > micro/libc-syscall.c micro/memburn.c micro/pio.c micro/pte-update.c > micro/read-bandwidth.c micro/read-latency.c micro/vmcall.c > inter/bandwidth.c inter/pingpong.c inter/sendfile.c inter/udp-bandwidth.c > micro/pte-update.c: In function `do_pte_update'': > micro/pte-update.c:45: error: too many arguments to function `mremap'' > make: *** [virtbench] Error 1 > </make error>I''ve just pushed an update that should disable this test for older versions of glibc. Would appreciate if you let me know whether this fixed your build. Regards, Anthony Liguori> > So first thing I would like to know if my XEN installation is compatible > with your benchmark suite and if not which are the requirements? > > Thank you for your help and support, > > Jan Michael > > > On 15.05.2007, at 04:38, Rusty Russell wrote: >> Hi all, >> >> I''ve been working on a set of easy-to-run benchmarks for hypervisor >> optimization called virtbench, and the latest commit adds Xen support. >> >> http://ozlabs.org/~rusty/virtbench >> http://ozlabs.org/~rusty/virtbench/?archive/tip.tar.bz2 >> >>> From the README: >> >> # Build the code >> make >> # Set the guest kernel location and maybe more >> vi ./SETTINGS >> # Run the benchmark (local mode) >> ./virtbench local >> # Run the benchmark (kvm mode) >> sudo ./virtbench kvm >> # Run the benchmark (Xen mode) >> sudo ./virtbench xen >> # Run the benchmark (lguest mode) >> sudo ./virtbench lguest >> >> Feedback welcome! >> Rusty. >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Michael
2007-May-18 10:14 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Hi Anthony, On 15.05.2007, at 21:50, Anthony Liguori wrote:> I''ve just pushed an update that should disable this test for older > versions of glibc. Would appreciate if you let me know whether > this fixed your build.That did the trick. Virtbench was sucessfully build on my system. I was also able to run virtbench in local mode but when I tried to run virtbench for xen I got the following error: <error> [root@lxdev19 virtbench-44dc217a9422]# KERNEL="/etc/xen/vmlinuz" ./ virtbench xen --progress virtbench: Start command failed for xen </error> I already tried to investigate this error viewing the virtbench logfile which I specified in SETTINGS file. But the logfile was not created :-(. What can I do to get your benchmark work for my XEN installation? Thanks, Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Rusty Russell
2007-May-18 11:51 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
On Fri, 2007-05-18 at 12:14 +0200, Jan Michael wrote:> Hi Anthony, > > On 15.05.2007, at 21:50, Anthony Liguori wrote: > > I''ve just pushed an update that should disable this test for older > > versions of glibc. Would appreciate if you let me know whether > > this fixed your build. > > That did the trick. Virtbench was sucessfully build on my system. I > was also able to run virtbench in local mode but when I tried to run > virtbench for xen I got the following error: > > <error> > [root@lxdev19 virtbench-44dc217a9422]# KERNEL="/etc/xen/vmlinuz" ./ > virtbench xen --progress > virtbench: Start command failed for xen > </error> > > I already tried to investigate this error viewing the virtbench > logfile which I specified in SETTINGS file. But the logfile was not > created :-(.This is a failure of the "xen/start" command. I had this happen when "/etc/init.d/xend restart" failed. You can probably comment that line out if you''re already running Xen (perhaps xen''s init scripts are flaky). Thanks for the report! Rusty. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Michael
2007-May-18 17:56 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Hi Rusty, On 18.05.2007, at 13:51, Rusty Russell wrote:> On Fri, 2007-05-18 at 12:14 +0200, Jan Michael wrote: >> Hi Anthony, >> >> On 15.05.2007, at 21:50, Anthony Liguori wrote: >>> I''ve just pushed an update that should disable this test for older >>> versions of glibc. Would appreciate if you let me know whether >>> this fixed your build. >> >> That did the trick. Virtbench was sucessfully build on my system. I >> was also able to run virtbench in local mode but when I tried to run >> virtbench for xen I got the following error: >> >> <error> >> [root@lxdev19 virtbench-44dc217a9422]# KERNEL="/etc/xen/vmlinuz" ./ >> virtbench xen --progress >> virtbench: Start command failed for xen >> </error> >> >> I already tried to investigate this error viewing the virtbench >> logfile which I specified in SETTINGS file. But the logfile was not >> created :-(. > > This is a failure of the "xen/start" command. I had this happen when > "/etc/init.d/xend restart" failed. You can probably comment that line > out if you''re already running Xen (perhaps xen''s init scripts are > flaky).Now I''m one step further. The domUs will be created, but they are destroyed after seconds. I started one of them manually with console so that I could see the error message: <boot error> md: autorun ... md: ... autorun DONE. Kernel panic - not syncing: VFS: Unable to mount root fs on unknown- block(1,0) </boot error> That must have been something to do with the kernel I''m using. I have to investigate (google) this. But if you can also help me in this case - you''re welcome! Cheers, Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Rusty Russell
2007-May-21 06:16 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
On Fri, 2007-05-18 at 19:56 +0200, Jan Michael wrote:> Hi Rusty,Hi JaN!> Now I''m one step further. The domUs will be created, but they are > destroyed after seconds. I started one of them manually with console > so that I could see the error message: > > <boot error> > md: autorun ... > md: ... autorun DONE. > Kernel panic - not syncing: VFS: Unable to mount root fs on unknown- > block(1,0) > </boot error> > > That must have been something to do with the kernel I''m using. I have > to investigate (google) this. But if you can also help me in this > case - you''re welcome!Hmm, perhaps your kernel doesn''t have INITRD support? The default Xen kernel works (at least, for me on i386 it does)... Thanks, Rusty. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Michael
2007-May-21 08:13 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Salut Rusty, On 21.05.2007, at 08:16, Rusty Russell wrote:> On Fri, 2007-05-18 at 19:56 +0200, Jan Michael wrote: >> Hi Rusty, > > Hi JaN! > >> Now I''m one step further. The domUs will be created, but they are >> destroyed after seconds. I started one of them manually with console >> so that I could see the error message: >> >> <boot error> >> md: autorun ... >> md: ... autorun DONE. >> Kernel panic - not syncing: VFS: Unable to mount root fs on unknown- >> block(1,0) >> </boot error> >> >> That must have been something to do with the kernel I''m using. I have >> to investigate (google) this. But if you can also help me in this >> case - you''re welcome! > > Hmm, perhaps your kernel doesn''t have INITRD support? The default Xen > kernel works (at least, for me on i386 it does)...Hm. No. Unfortunately the kernel I''m using has INITRD support, because we need it also for self-installation of our domUs in productive environment. I attached the whole startup log. The RAM disk driver is loaded in line 50. Do you have any other suggestions? Thanks, Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2007-May-21 13:13 UTC
Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Jan Michael wrote:> Salut Rusty, > > On 21.05.2007, at 08:16, Rusty Russell wrote: >> On Fri, 2007-05-18 at 19:56 +0200, Jan Michael wrote: >>> Hi Rusty, >> >> Hi JaN! >> >>> Now I''m one step further. The domUs will be created, but they are >>> destroyed after seconds. I started one of them manually with console >>> so that I could see the error message: >>> >>> <boot error> >>> md: autorun ... >>> md: ... autorun DONE. >>> Kernel panic - not syncing: VFS: Unable to mount root fs on unknown- >>> block(1,0) >>> </boot error> >>> >>> That must have been something to do with the kernel I''m using. I have >>> to investigate (google) this. But if you can also help me in this >>> case - you''re welcome! >> >> Hmm, perhaps your kernel doesn''t have INITRD support? The default Xen >> kernel works (at least, for me on i386 it does)... > > Hm. No. Unfortunately the kernel I''m using has INITRD support, because > we need it also for self-installation of our domUs in productive > environment. I attached the whole startup log. The RAM disk driver is > loaded in line 50. > Do you have any other suggestions?66 XENBUS: Device with no driver: device/vbd/51713 67 XENBUS: Device with no driver: device/vif/0 looks worrysome. Do you have netfront and blkfront compiled into (your kernel, either built-in or as modules in your initrd)? Though it seems you have a more basic problem with mounting the initrd. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Rusty Russell
2007-May-21 23:45 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
On Mon, 2007-05-21 at 10:13 +0200, Jan Michael wrote:> Salut Rusty, > > On 21.05.2007, at 08:16, Rusty Russell wrote: > > Hmm, perhaps your kernel doesn''t have INITRD support? The default Xen > > kernel works (at least, for me on i386 it does)... > > Hm. No. Unfortunately the kernel I''m using has INITRD support,Hi Jan! Hmm, it has found the initrd, and ramdisk seems to be there. Any chance you can send your kernel config? Obviously we want it to "just work" with whatever configs it can... Thanks for chasing this! Rusty. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Michael
2007-May-22 07:33 UTC
[Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Hi Rusty, On 22.05.2007, at 01:45, Rusty Russell wrote:> On Mon, 2007-05-21 at 10:13 +0200, Jan Michael wrote: >> Salut Rusty, >> >> On 21.05.2007, at 08:16, Rusty Russell wrote: >>> Hmm, perhaps your kernel doesn''t have INITRD support? The >>> default Xen >>> kernel works (at least, for me on i386 it does)... >> >> Hm. No. Unfortunately the kernel I''m using has INITRD support, > > Hi Jan! > > Hmm, it has found the initrd, and ramdisk seems to be there. Any > chance you can send your kernel config? Obviously we want it to "just > work" with whatever configs it can...Sure. No problem. Please find attached our kernel config for domUs. Did you see the mail from Jeremy? On 21.05.2007, at 15:13, Jeremy Fitzhardinge wrote:> 66 XENBUS: Device with no driver: device/vbd/51713 > 67 XENBUS: Device with no driver: device/vif/0 > > looks worrysome. Do you have netfront and blkfront compiled into > (your > kernel, either built-in or as modules in your initrd)? > > Though it seems you have a more basic problem with mounting the > initrd.I can say that we are using a kernel builf from the attached kernel config and a special initrd image for our domUs. The initrd image contains the modules xenblk.ko and xennet.ko. So netfront and blkfront are not compiled into the kernel. Our linux gurus made the kernel and the initrd image. I think I have to integrate those modules into the initrd image you are using and have to load them. But at this point I have no knowledge how to do this... Cheers, Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2007-May-22 09:33 UTC
Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Jan Michael wrote:> I can say that we are using a kernel builf from the attached kernel > config and a special initrd image for our domUs. > The initrd image contains the modules xenblk.ko and xennet.ko. So > netfront and blkfront are not compiled into the kernel.Modular is OK if they''re in the initrd you''re using.> Our linux gurus made the kernel and the initrd image. I think I have > to integrate those modules into the initrd image you are using and > have to load them. But at this point I have no knowledge how to do > this...It''s distro-dependent, but mkinitrd works for RH-type systems. Something like "mkinitrd --with=xenblk --with=xennet -v my-initrd.img <kernel version>". J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Michael
2007-May-23 18:05 UTC
Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Hello,
as I found out it is a little bit tricky to use one''s initrd. Because
Virtbench is overriding the init command in the domU configuration
file with rdinit and therefore it is not possible to load modules:
<virtbench-xen.cfg.in>
extra = "rdinit=/virtclient @VIRTBENCH_ID@ @SERVERIP@
@SERVERPORT@ /dev/xvda1 202 1 eth0 %s" %
string.replace(''@IP@'', ''.'',
''/'')
</virtbench-xen.cfg.in>
The template file virtbench-xen.cfg.in is used to create an
individual xen configuration file for each user domain.
Despite this I tried to get the virtclient working with my own
initrd. I was able to successfully load the xen network and block
device driver. At the end of the init script I could start virtclient
from the ramdisk.
So I modified the start_machine script in that way that it will
manipulate the init start script of the ramdisk and after that it
creates one initrd for each domU. The domU configuration uses now a
prepared ramdisk rather than this one from virtbench.
The benchmark passed with the following outcome:
Time for one context switch via pipe: 8734 (8640 - 9575)
Time for one Copy-on-Write fault: 5898 (5814 - 8963)
Time to exec client once: 573046 (565921 - 615390)
Time for one fork/exit/wait: 347687 (345750 - 362250)
Time to send 4 MB from host: 55785000 (27069625 - 315191500)
Time for one int-0x80 syscall: 370 (370 - 403)
Time for one syscall via libc: 376 (376 - 377)
Time to walk linear 64 MB: 1790875 (1711750 - 3332875)
Time to walk random 64 MB: 2254500 (2246000 - 2266250)
Time for one outb PIO operation: 721 (717 - 733)
DISABLED pte-update: glibc version is too old
Time to read from disk (256 kB): 18810406 (14266718 - 24088906)
Time for one disk read: 56343 (38593 - 201718)
DISABLED vmcall: not a VT guest
DISABLED vmmcall: not an SVM guest
Time to send 4 MB between guests: 94326750 (79872250 - 729306500)
Time for inter-guest pingpong: 130316 (119722 - 186511)
Time to sendfile 4 MB between guests: 134768000 (86528000 - 417646000)
Time to receive 1000 1k UDPs between guests: 26010000 (23384000 -
66784000)
Last question for today: What is the unit of these time values?
Thanks,
Jan
On 22.05.2007, at 11:33, Jeremy Fitzhardinge wrote:> Jan Michael wrote:
>> I can say that we are using a kernel builf from the attached kernel
>> config and a special initrd image for our domUs.
>> The initrd image contains the modules xenblk.ko and xennet.ko. So
>> netfront and blkfront are not compiled into the kernel.
>
> Modular is OK if they''re in the initrd you''re using.
>
>> Our linux gurus made the kernel and the initrd image. I think I have
>> to integrate those modules into the initrd image you are using and
>> have to load them. But at this point I have no knowledge how to do
>> this...
>
> It''s distro-dependent, but mkinitrd works for RH-type systems.
> Something like "mkinitrd --with=xenblk --with=xennet -v my-initrd.img
> <kernel version>".
>
> J
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jan Michael
2007-May-24 15:37 UTC
Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Hi Everybody, On 23.05.2007, at 20:05, Jan Michael wrote:> The benchmark passed with the following outcome: > > Time for one context switch via pipe: 8734 (8640 - 9575) > Time for one Copy-on-Write fault: 5898 (5814 - 8963) > Time to exec client once: 573046 (565921 - 615390) > Time for one fork/exit/wait: 347687 (345750 - 362250) > Time to send 4 MB from host: 55785000 (27069625 - 315191500) > Time for one int-0x80 syscall: 370 (370 - 403) > Time for one syscall via libc: 376 (376 - 377) > Time to walk linear 64 MB: 1790875 (1711750 - 3332875) > Time to walk random 64 MB: 2254500 (2246000 - 2266250) > Time for one outb PIO operation: 721 (717 - 733) > DISABLED pte-update: glibc version is too old > Time to read from disk (256 kB): 18810406 (14266718 - 24088906) > Time for one disk read: 56343 (38593 - 201718) > DISABLED vmcall: not a VT guest > DISABLED vmmcall: not an SVM guest > Time to send 4 MB between guests: 94326750 (79872250 - 729306500) > Time for inter-guest pingpong: 130316 (119722 - 186511) > Time to sendfile 4 MB between guests: 134768000 (86528000 - 417646000) > Time to receive 1000 1k UDPs between guests: 26010000 (23384000 - > 66784000)I didn''t had anything to do with benchmarking in the past, and especially not with virtualization benchmarks, so there are again some questions related to the results of the benchmarking test: 1. What can I read out of every single value which is listed above? Can you please give a short explenation? 2. What are the unit(s) of the measured values? 3. What is a good value and what is a bad value? On what does these measures depend on - hardware or software or both? 4. If I get a certain value like this one: Time for one context switch via pipe: 8734 (8640 - 9575). What can I do to improve/tune the performance or the values? 5. I googled through the web to find any results to compare with mine, but I couldn''t find anything. Do you have some? 6. In the README file is said that virtbench contains "low level" benchmarks. What do you consider as a "high level" benchmark? Ok. Enough of my questions so far. If you answere these ones I''ll may be have more afterwards. Thanks for your help, Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Petersson, Mats
2007-May-24 16:11 UTC
RE: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > Jan Michael > Sent: 24 May 2007 16:37 > To: Jeremy Fitzhardinge; Anthony Liguori; Rusty Russell; Xen > Mailing List > Cc: Alex Iribarren > Subject: Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support > > Hi Everybody, > > On 23.05.2007, at 20:05, Jan Michael wrote: > > The benchmark passed with the following outcome: > > > > Time for one context switch via pipe: 8734 (8640 - 9575) > > Time for one Copy-on-Write fault: 5898 (5814 - 8963) > > Time to exec client once: 573046 (565921 - 615390) > > Time for one fork/exit/wait: 347687 (345750 - 362250) > > Time to send 4 MB from host: 55785000 (27069625 - 315191500) > > Time for one int-0x80 syscall: 370 (370 - 403) > > Time for one syscall via libc: 376 (376 - 377) > > Time to walk linear 64 MB: 1790875 (1711750 - 3332875) > > Time to walk random 64 MB: 2254500 (2246000 - 2266250) > > Time for one outb PIO operation: 721 (717 - 733) > > DISABLED pte-update: glibc version is too old > > Time to read from disk (256 kB): 18810406 (14266718 - 24088906) > > Time for one disk read: 56343 (38593 - 201718) > > DISABLED vmcall: not a VT guest > > DISABLED vmmcall: not an SVM guest > > Time to send 4 MB between guests: 94326750 (79872250 - 729306500) > > Time for inter-guest pingpong: 130316 (119722 - 186511) > > Time to sendfile 4 MB between guests: 134768000 (86528000 - > 417646000) > > Time to receive 1000 1k UDPs between guests: 26010000 (23384000 - > > 66784000) > > I didn''t had anything to do with benchmarking in the past, and > especially not with virtualization benchmarks, so there are again > some questions related to the results of the benchmarking test: > > 1. What can I read out of every single value which is > listed above?The time it takes to perform the particular microbenchmark.> Can you please give a short explenation? > 2. What are the unit(s) of the measured values?Good question, and I don''t know the actual answer. I suspect they are clock-cycles or perhaps nanoseconds. It''s clearly not milliseconds or microseconds, so it''s a "very short time-unit".> 3. What is a good value and what is a bad value? On > what does these > measures depend on - hardware or software or both?They aren''t good or bad values as such - they are comparative numbers. There are no "absolute" good or bad values. If I say "ten seconds", that may be a good value if you''re running 100m. But it''s certainly a bad value for a computer running 10000 instructions, for example. Using these values, one could either compare one implementation of Xen with another, or compare two machines with different specs (e.g. different processors, different memory types, different network cards or disks, or whatever). The on with the higher numbers is the slower one.> 4. If I get a certain value like this one: Time for one > context > switch via pipe: 8734 (8640 - 9575). What can I do to improve/tune > the performance or the values?Like any other performance improvement, you''d have to figure out where the majority of time[1] is spent for this microbenchmark, and then try to improve that somehow. Repeatedly (unless it runs for a long time in itself) running this particular benchmark and running "oprofile" on the machine would be able to give a fair idea of where in the system the time is spent. The numbers in the bracket is the upper/lower numbers, the first number being the average of several runs.> 5. I googled through the web to find any results to > compare with > mine, but I couldn''t find anything. Do you have some?I don''t.> 6. In the README file is said that virtbench contains > "low level" > benchmarks. What do you consider as a "high level" benchmark?Low-level benchmark is similar to "microbenchmark". It tests ONE particular feature of the system in isolation. For example, the "context switch via pipe" is sending a message via a pipe from one process to another process, and measuring the time it takes from sending the message until it''s been received at the other end. This is a good way to measure very precise parts of a system, but improving this by 10%, 20% or 50% may serve no purpose if it isn''t a large portion of a higher level functionality. E.g. if you run the Blurg[2] web-server, it may not use pipes at all, so the performance of Blurg is completely unrelated of the performance of this particular benchmark. Some other functions in the microbenchmark are likely to have some effect on Blurg, but it may also be that a major portion of Blurg''s execution time isn''t in the OS/Hypervisor at all, so it doesn''t really make much difference at all. To give another type of example: We can measure the horse-power of a car-enging. There are several ways to do this. The most realistic is one that actually uses the car itself (such as a rolling-road), but we can also dismount the engine from the car and measure it without the gearbox, cooling fans, water pumps, and whatever else that can be "removed". This method will of course generate (somewhat) more power, but also less useful numbers. On the other hand, all of this is pointless if you can''t actually USE the power (e.g. the suspension isn''t good enough to go round corners, the brakes don''t work well, so if you don''t have half a kilometer to stop, you can''t use the maximum speed of the car, etc, etc) on the road/racetrack, right? So the BEST way to compare two cars would be to use the same (skilled) driver around a track or a road, to see which performs best. Microbenchmarks measure the engine-power, braking power, suspension springs, etc, etc. High level/application benchmarks measures the systems ability to perform a higher level task, such as web-serving, file-serving, complex calculation tasks, or some such. [1] Figuring out where the majority of time is spent is USUALLY the best place to start. However, there are cases where small distributed bits of code are the major part. In the past, I''ve seen cases where a function called many times got inlined, and eaach individual "call" of the function didn''t amount to much, but since it was called many times during the overall benchmark, it amounted to a noticable overhead. In another case, there was a "trace-function" that got called thousands of times a second, but since tracing was turned off, it didn''t actually do anything but return. This "nothing but return" was about 2% of the overall time of the "benchmark". However, the effect of actually CALLING the function (passing a bunch of parameters and often extracting those parameters from pointers/data structures) was taking about 15% of the overall time. Moving the check to see if there was any output to be done to outside the function call improved the overall performance by about 16%. Worth having, eh? [2] Blurg is a fictional web-server, not a product in real life, but for this example, it doesn''t really matter. -- Mats> > Ok. Enough of my questions so far. If you answere these ones > I''ll may > be have more afterwards. > Thanks for your help, > > Jan > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Michael
2007-Jun-06 08:51 UTC
Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
Hello Everyone, I got my hands on a HP blade with 64-bit Intel XEON processors. I could successfully start the benchmark but after a while client 1 closed the connection. Please see the full log below. Could you please help me to find the error? Is there a some kind of debugging mode I could use to get more details from both, client and server side? <virtbench execution> [root@somemachine virtbench-cern]# ./virtbench xen Bringing up machines11188 blocks 58.7% 11188 blocks 58.7% 11188 blocks 58.7% 11188 blocks 58.7% .... Time for one context switch via pipe: 7747 (5070 - 8202) Time for one Copy-on-Write fault: 4266 (4242 - 5865) Time to exec client once: 331171 (329757 - 332406) Time for one fork/exit/wait: 196031 (195781 - 196812) Time to send 4 MB from host: 37688750 (27595500 - 47665250) Time for one syscall via libc: 428 (417 - 652) Time to walk linear 64 MB: 2130375 (2123000 - 2168750) Time to walk random 64 MB: 2135000 (2130250 - 2140625) virtbench: client 1 closed connection sh: line 1: Using: command not found sh: line 2: Started: command not found virtbench: ''xen/stop_machine virtbench_3 Using config file "/tmp/virtbench-xen_3.cfg". Started domain virtbench_3 '' failed sh: line 1: Using: command not found sh: line 2: Started: command not found virtbench: ''xen/stop_machine virtbench_2 Using config file "/tmp/virtbench-xen_2.cfg". Started domain virtbench_2 '' failed Error: Domain ''virtbench_1'' does not exist. Error: ''xm destroy'' requires 1 arguments. sh: line 1: Using: command not found sh: line 2: Started: command not found virtbench: ''xen/stop_machine virtbench_1 Using config file "/tmp/virtbench-xen_1.cfg". Started domain virtbench_1 '' failed sh: line 1: Using: command not found sh: line 2: Started: command not found virtbench: ''xen/stop_machine virtbench_0 Using config file "/tmp/virtbench-xen_0.cfg". Started domain virtbench_0 '' failed </virtbench execution> Thank you very much, Jan Michael On 23.05.2007, at 20:05, Jan Michael wrote:> Hello, > > as I found out it is a little bit tricky to use one''s initrd. > Because Virtbench is overriding the init command in the domU > configuration file with rdinit and therefore it is not possible to > load modules: > > <virtbench-xen.cfg.in> > extra = "rdinit=/virtclient @VIRTBENCH_ID@ @SERVERIP@ > @SERVERPORT@ /dev/xvda1 202 1 eth0 %s" % string.replace(''@IP@'', > ''.'', ''/'') > </virtbench-xen.cfg.in> > > The template file virtbench-xen.cfg.in is used to create an > individual xen configuration file for each user domain. > Despite this I tried to get the virtclient working with my own > initrd. I was able to successfully load the xen network and block > device driver. At the end of the init script I could start > virtclient from the ramdisk. > > So I modified the start_machine script in that way that it will > manipulate the init start script of the ramdisk and after that it > creates one initrd for each domU. The domU configuration uses now a > prepared ramdisk rather than this one from virtbench. > > The benchmark passed with the following outcome: > > Time for one context switch via pipe: 8734 (8640 - 9575) > Time for one Copy-on-Write fault: 5898 (5814 - 8963) > Time to exec client once: 573046 (565921 - 615390) > Time for one fork/exit/wait: 347687 (345750 - 362250) > Time to send 4 MB from host: 55785000 (27069625 - 315191500) > Time for one int-0x80 syscall: 370 (370 - 403) > Time for one syscall via libc: 376 (376 - 377) > Time to walk linear 64 MB: 1790875 (1711750 - 3332875) > Time to walk random 64 MB: 2254500 (2246000 - 2266250) > Time for one outb PIO operation: 721 (717 - 733) > DISABLED pte-update: glibc version is too old > Time to read from disk (256 kB): 18810406 (14266718 - 24088906) > Time for one disk read: 56343 (38593 - 201718) > DISABLED vmcall: not a VT guest > DISABLED vmmcall: not an SVM guest > Time to send 4 MB between guests: 94326750 (79872250 - 729306500) > Time for inter-guest pingpong: 130316 (119722 - 186511) > Time to sendfile 4 MB between guests: 134768000 (86528000 - 417646000) > Time to receive 1000 1k UDPs between guests: 26010000 (23384000 - > 66784000) > > Last question for today: What is the unit of these time values? > > Thanks, > > Jan > > > On 22.05.2007, at 11:33, Jeremy Fitzhardinge wrote: >> Jan Michael wrote: >>> I can say that we are using a kernel builf from the attached kernel >>> config and a special initrd image for our domUs. >>> The initrd image contains the modules xenblk.ko and xennet.ko. So >>> netfront and blkfront are not compiled into the kernel. >> >> Modular is OK if they''re in the initrd you''re using. >> >>> Our linux gurus made the kernel and the initrd image. I think I have >>> to integrate those modules into the initrd image you are using and >>> have to load them. But at this point I have no knowledge how to do >>> this... >> >> It''s distro-dependent, but mkinitrd works for RH-type systems. >> Something like "mkinitrd --with=xenblk --with=xennet -v my-initrd.img >> <kernel version>". >> >> J > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Rusty Russell
2007-Jun-09 03:19 UTC
Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
On Thu, 2007-05-24 at 17:37 +0200, Jan Michael wrote:> I didn''t had anything to do with benchmarking in the past, and > especially not with virtualization benchmarks, so there are again > some questions related to the results of the benchmarking test: > > 1. What can I read out of every single value which is listed above? > Can you please give a short explenation? > 2. What are the unit(s) of the measured values?Hi Jan! Each one is in nanoseconds, shorter is better. All of them are run on processes within one randomly-chosen domU of the four (some are inter-guest test which run on two domUs)> > Time for one context switch via pipe: 8734 (8640 - 9575)Two processes within the domU, one is doing a read() waiting for the other to do a write(), then vice versa> > Time for one Copy-on-Write fault: 5898 (5814 - 8963)This measures the time for a page marked readonly to become writable when the guest writes to it.> > Time to exec client once: 573046 (565921 - 615390)This measures the client process execing itself.> > Time for one fork/exit/wait: 347687 (345750 - 362250)This measure the client process fork()ing, the child exiting, and the parent waiting for it.> > Time to send 4 MB from host: 55785000 (27069625 - 315191500)This measures network speed: 4MB TCP transfer from the virtbench process (dom0) to the client (domU).> > Time for one int-0x80 syscall: 370 (370 - 403) > > Time for one syscall via libc: 376 (376 - 377)These are the time taken to do a getppid() system call.> > Time to walk linear 64 MB: 1790875 (1711750 - 3332875) > > Time to walk random 64 MB: 2254500 (2246000 - 2266250)Memory walking.> > Time for one outb PIO operation: 721 (717 - 733)One io operation, roughly the time taken for a hypervisor entry & exit.> > DISABLED pte-update: glibc version is too oldThis test measures the time to update two page table entries, but required mremap() which is only in modern glibcs.> > Time to read from disk (256 kB): 18810406 (14266718 - 24088906)Read 256k from the block device.> > Time for one disk read: 56343 (38593 - 201718)Read a single block from the block device (ie. latency).> > DISABLED vmcall: not a VT guest > > DISABLED vmmcall: not an SVM guestThese only apply to fully-virtualized guests.> > Time to send 4 MB between guests: 94326750 (79872250 - 729306500)domU <-> domU 4MB TCP write.> > Time for inter-guest pingpong: 130316 (119722 - 186511)domU <-> domU TCP latency.> > Time to sendfile 4 MB between guests: 134768000 (86528000 - 417646000)domU <-> domU 4MB TCP write using sendfile().> > Time to receive 1000 1k UDPs between guests: 26010000 (23384000 - > > 66784000)Sending 1000 UDP packets from domU <-> domU. This benchmark is horribly unreliable and should probably be removed.> 3. What is a good value and what is a bad value? On what does these > measures depend on - hardware or software or both?Both... run "virtbench local" on the same hardware on a normal Linux kernel to see what native results are. This is really the target to aim for.> 4. If I get a certain value like this one: Time for one context > switch via pipe: 8734 (8640 - 9575). What can I do to improve/tune > the performance or the values?That would be Xen-specific, I''m not entirely sure how much that can be improved.> 5. I googled through the web to find any results to compare with > mine, but I couldn''t find anything. Do you have some?I do not release benchmark numbers myself; they''re quite dependent on particular hardware, and also virtualization technology is moving rapidly enough to make them quite obsolete. virtbench is mainly useful for spotting regressions, measuring code optimizations and explaining the results of higher-level benchmarks.> 6. In the README file is said that virtbench contains "low level" > benchmarks. What do you consider as a "high level" benchmark?Things like: kernbench, SDET, Spec, etc. I hope that helps, Rusty. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Rusty Russell
2007-Jun-09 03:57 UTC
Re: [Xen-devel] Re: [ANNOUNCE] virtbench now has xen support
On Wed, 2007-06-06 at 10:51 +0200, Jan Michael wrote:> Hello Everyone, > > I got my hands on a HP blade with 64-bit Intel XEON processors. I > could successfully start the benchmark but after a while client 1 > closed the connection. Please see the full log below. > > Could you please help me to find the error? Is there a some kind of > debugging mode I could use to get more details from both, client and > server side? > > <virtbench execution> > [root@somemachine virtbench-cern]# ./virtbench xen > Bringing up machines11188 blocks > 58.7% > 11188 blocks > 58.7% > 11188 blocks > 58.7% > 11188 blocks > 58.7% > .... > Time for one context switch via pipe: 7747 (5070 - 8202) > Time for one Copy-on-Write fault: 4266 (4242 - 5865) > Time to exec client once: 331171 (329757 - 332406) > Time for one fork/exit/wait: 196031 (195781 - 196812) > Time to send 4 MB from host: 37688750 (27595500 - 47665250) > Time for one syscall via libc: 428 (417 - 652) > Time to walk linear 64 MB: 2130375 (2123000 - 2168750) > Time to walk random 64 MB: 2135000 (2130250 - 2140625) > virtbench: client 1 closed connectionOK, this looks like look it got upset on the "outb" test. Unfortunately, redirecting standard input from /dev/null causes xen not to output any console at all. I''m not sure how to fix this, but you can enter names manually on the commandline to avoid running this benchmark.> sh: line 1: Using: command not found > sh: line 2: Started: command not found > virtbench: ''xen/stop_machine virtbench_3 > Using config file "/tmp/virtbench-xen_3.cfg". > Started domain virtbench_3 > '' failedIt seems that your xen/start_machine script printed out extra stuff, rahter than just the id to hand back to stop_machine. Cheers, Rusty. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel