I saw the presentation "Virtualization of Linux Servers" at OLS last month and it had some nice comparisons of Xen performance vs a lot of other virtualization/container technologies: http://ols.fedoraproject.org/OLS/Reprints-2008/camargos-reprint.pdf As always with benchmarks, there are questions to ask and points to quibble, but overall Xen looks quite good... except on Dbench. Has anybody else run this benchmark on Xen and gotten better results? If not, any thoughts on why Xen (and all virt solutions) would do poorly on this benchmark? And whether Xen can/should be fixed? ==================================Thanks... for the memory I really could use more / My throughput's on the floor The balloon is flat / My swap disk's fat / I've OOM's in store Overcommitted so much (with apologies to the late great Bob Hope) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Aug 7, 2008 at 1:18 PM, Dan Magenheimer <dan.magenheimer@oracle.com> wrote:> I saw the presentation "Virtualization of Linux Servers" at > OLS last month and it had some nice comparisons of Xen > performance vs a lot of other virtualization/container > technologies: > > http://ols.fedoraproject.org/OLS/Reprints-2008/camargos-reprint.pdf >Thanks for pointing this one out.> As always with benchmarks, there are questions to ask and > points to quibble, but overall Xen looks quite good... > except on Dbench. Has anybody else run this benchmark > on Xen and gotten better results? If not, any thoughts > on why Xen (and all virt solutions) would do poorly on this > benchmark? And whether Xen can/should be fixed? >The original Xen paper [1] and our repeated research paper [2] showed a decent dbench "score" vs. native. http://www.cl.cam.ac.uk/research/srg/netos/papers/2003-xensosp.pdf http://web2.clarkson.edu/class/cs644/xen/files/repeatedxen-usenix04.pdf dbench is known to have a high standard deviation... some KVM results for dbench: http://virt.kernelnewbies.org/KVM/Performance dbench is an interesting test since it is basically an I/O test (Samba test) http://samba.org/ftp/tridge/dbench/README I don''t know if dbench is the best test to determine if Xen (or other virts) need to be fixed. Cheers, Todd> ==================================> Thanks... for the memory > I really could use more / My throughput''s on the floor > The balloon is flat / My swap disk''s fat / I''ve OOM''s in store > Overcommitted so much > (with apologies to the late great Bob Hope) > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > >-- Todd Deshane http://todddeshane.net check out our book: http://runningxen.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Todd Deshane wrote:> dbench is an interesting test since it is basically an I/O test (Samba test) > http://samba.org/ftp/tridge/dbench/READMEdbench is actually *not* an I/O test. It mostly stresses a filesystems interaction with the page cache. It''s heavily threaded and tends to scale okay but it''s rarely impacted heavily by the underlying storage systems I/O performance. It tends to demonstrate shadow page table SMP scalability more than I/O performance. The OLS paper referenced really had bad methodologies. If you read carefully, their host system was a 2-way system. They ran all of the guests UP though. Since they didn''t do a parallel make, it wasn''t very obvious for their "kernel build" but it became obvious with dbench since there were multiple threads. So native and "VServer" had access to both CPU cores whereas Xen, KVM, et al were only running on a single core. There''s no way dbench is 30% of native under Xen. That should have been a big red flag that something was wrong. Regards, Anthony Liguori> I don''t know if dbench is the best test to determine if Xen (or other > virts) need > to be fixed. > > Cheers, > Todd > > >> ==================================>> Thanks... for the memory >> I really could use more / My throughput''s on the floor >> The balloon is flat / My swap disk''s fat / I''ve OOM''s in store >> Overcommitted so much >> (with apologies to the late great Bob Hope) >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >> > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks, Anthony, very illuminating. However one would expect that the 1-cpu measurement would be about one-half of the 2-cpu measurement, not closer to one-fourth (27%), true? So even if the methodology is bad, the results may still be indicating something wrong that should be looked at? Todd, the references you provide are several years and several versions of Xen old. Have you seen any Xen dbench results on a more recent Xen version? While I agree that dbench might not be the best benchmark in the world, if some customers use and believe it (or, worse, believe the OLS paper), it would be nice to be able to set the record straight.> -----Original Message----- > From: Anthony Liguori [mailto:anthony@codemonkey.ws] > Sent: Thursday, August 07, 2008 1:50 PM > To: deshantm@gmail.com > Cc: dan.magenheimer@oracle.com; Xen-Devel (E-mail) > Subject: Re: Xen performance and Dbench > > > Todd Deshane wrote: > > dbench is an interesting test since it is basically an I/O > test (Samba test) > > http://samba.org/ftp/tridge/dbench/README > > > dbench is actually *not* an I/O test. It mostly stresses a > filesystems > interaction with the page cache. It''s heavily threaded and tends to > scale okay but it''s rarely impacted heavily by the underlying storage > systems I/O performance. It tends to demonstrate shadow page > table SMP > scalability more than I/O performance. > > The OLS paper referenced really had bad methodologies. If you read > carefully, their host system was a 2-way system. They ran all of the > guests UP though. Since they didn''t do a parallel make, it > wasn''t very > obvious for their "kernel build" but it became obvious with > dbench since > there were multiple threads. > > So native and "VServer" had access to both CPU cores whereas Xen, KVM, > et al were only running on a single core. There''s no way > dbench is 30% > of native under Xen. That should have been a big red flag that > something was wrong. > > Regards, > > Anthony Liguori > > > I don''t know if dbench is the best test to determine if Xen > (or other > > virts) need > > to be fixed. > > > > Cheers, > > Todd > > > > > >> ==================================> >> Thanks... for the memory > >> I really could use more / My throughput''s on the floor > >> The balloon is flat / My swap disk''s fat / I''ve OOM''s in store > >> Overcommitted so much > >> (with apologies to the late great Bob Hope) > >> _______________________________________________ > >> Xen-devel mailing list > >> Xen-devel@lists.xensource.com > >> http://lists.xensource.com/xen-devel > >> > >> > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Aug 7, 2008 at 6:23 PM, Dan Magenheimer <dan.magenheimer@oracle.com> wrote:> Thanks, Anthony, very illuminating. However one would > expect that the 1-cpu measurement would be about one-half > of the 2-cpu measurement, not closer to one-fourth (27%), > true? So even if the methodology is bad, the results > may still be indicating something wrong that should > be looked at? > > Todd, the references you provide are several years and > several versions of Xen old. Have you seen any Xen > dbench results on a more recent Xen version? While > I agree that dbench might not be the best benchmark > in the world, if some customers use and believe it > (or, worse, believe the OLS paper), it would be nice > to be able to set the record straight. >I don''t know of anything on dbench and Xen. I don''t think it would be too hard to try to re-run their Xen dbench numbers. We could then not make the mistakes that Anthony pointed out. More investigation is definitely needed. Cheers, Todd _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel