Fedora xen - Apr 2007 - Benchmarking Report (Xen Virtualiztion , Para vs Fully)

Hi everyone ,

We are the students of seneca college toronto . We have developed a project
which performs the benchmarking tests over Xen machine and compare Para and
fully virtualiztions . Here is our final report of results of benchmarking Xen
tests. We used three different tools to perform the tests and posted the data
and graphs in our report.


The html format of our project is 
http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml


Any comment to out report is appreciable.

Thanks
Kiranjeet Beniwal

On Sat, 14 Apr 2007, Kiranjeet Kaur Beniwal wrote:
> Hi everyone ,
>
> We are the students of seneca college toronto . We have developed a project
which performs the benchmarking tests over Xen machine and compare Para and
fully virtualiztions . Here is our final report of results of benchmarking Xen
tests. We used three different tools to perform the tests and posted the data
and graphs in our report.
>
>
> The html format of our project is
> http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml
You only compared full vs para. Could you run the same benchmarks on the
same server in Iron mode, eg a non-xen kernel?

Most of our customers want to know the impact of "normal" vs
"virtual",
which this benchmark did not report on at all.

Paul
ps. say hi to Mike for me.

On Sa April 14 2007, Kiranjeet Kaur Beniwal wrote:
> The html format of our project is
> http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml
> Any comment to out report is appreciable.
Thanks, I looked at it only superficial and one thing irritated me: you 
switched the colors for Fully/Para in the "Disk/Filesystem IO"
Diagramm,
which I guess was not intentional.

Regards,
Till

On Sat, Apr 14, 2007 at 01:03:42PM -0400, Kiranjeet Kaur Beniwal
wrote:
> Hi everyone ,
> 
> We are the students of seneca college toronto . We have developed a project
> which performs the benchmarking tests over Xen machine and compare Para and
> fully virtualiztions . Here is our final report of results of benchmarking 
> Xen tests. We used three different tools to perform the tests and posted
the
> data and graphs in our report.
> 
> The html format of our project is 
> http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml
The pure-compute results look reasonable, but I don''t believe those
results for
Disk I/O in fully-virt for a second - Section 6, Disk I/O Unixbench is the exact
opposite of expectation - were results flipped?.  Based knowledge of the
architecture
of paravirt/ fullyvirt I/O subsystems, there is no way in the world fullyvirt
will ever outperform paravirt for network or disk. In previous benchmarks
I''ve
typically seen Paravirt have x2 the disk performance of fullyvirt, and have
x10 the network performance.

I''m not familiar with UnixBench enough to say whether it is good
benchmark
to use, or whether these were just runtime anomolies. I''d recommend
doing
some disk I/O tests using IOZone which is the pretty common standard for
disk benchmarking.

I''d also recommend doing some scaling tests using multiple concurrent
guests,
since people rarely use virtualization for just a single guest. When adding
additional concurrently executing guests I''d expect to see the gap
between
paravirt and full-virt increase even further. 

Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|

8:33pm Daniel P. Berrange said:
>
> The pure-compute results look reasonable, but I don''t believe
those results for
> Disk I/O in fully-virt for a second - Section 6, Disk I/O Unixbench is the
exact
> opposite of expectation - were results flipped?.  Based knowledge of the
architecture
>
And is it even fair to compare tap:aio: to file: disks without even a 
mention of the technical difference?

../C

On Sun, Apr 15, 2007 at 06:28:10PM -0700, Curtis Doty
wrote:
> 8:33pm Daniel P. Berrange said:
> >
> >The pure-compute results look reasonable, but I don''t believe
those
> >results for
> >Disk I/O in fully-virt for a second - Section 6, Disk I/O Unixbench is
the
> >exact
> >opposite of expectation - were results flipped?.  Based knowledge of
the
> >architecture
> >
> 
> And is it even fair to compare tap:aio: to file: disks without even a 
> mention of the technical difference?
It is as fair as you''ll get for file-backed VMs.
''file:'' for fully-virt
doesn''t have the same disadvantages / problems that
''file:'' does for
paravirt, since its actually better to compare ''tap:aio:'' for
PV with
''file:'' for FV that to compare ''file:'' with
PV with ''file:'' for FV.

It would certainly be worth doing 2 sets of disk I/O tests though, one
with file backed disks, and the other with raw partition / LVM backed
disks.

Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|