Hi Folks, I was wondering if anyone has an pointers/suggestions on how I might increase disk performance of a HVM zvol-backed DomU? - this is my first DomU, so hopefully its something obvious Running bonnie++ shows the DomU''s performance to be 3 orders of magnitude worse than Dom0''s, which itself is half as good as when not running xVM at all (see bottom for bonnie++ results) I''m using a 2*raidz2 SAN for the DomU backing. I''m also using a separate disk (a compact flash) for a ZFS-based root disk. The system has a single quad-core opteron with 8G phyical memory, but when I boot Dom0, I pass "dom0_mem=4G dom0_max_vcpus=2 dom0_vcpus_pin=true" on the "kernel$" line to limit how much mem/cpu the Dom0 can use. When I created my DomU, I first created the zvol: zfs create -V 4G tank/puffy-root-disk and then installed the OS with something like this: virt-install --name puffy --hvm --ram 512 --vnc \ --file /dev/zvol/dsk/tank/puffy-root-disk \ --cdrom /tank/install43_amd64.iso --noautoconsole \ --os-type=unix --os-variant=openbsd4 --vcpus=1 BTW, I really didn''t have to run bonnie++ to know the performance wasn''t good since: 1) the install took ~40 minutes (normally ~2 minutes) 2) `xm top` showed the DomU as "blocked" almost the whole time I really haven''t found much info on ways to improve this, does anyone have any pointers? PS: bonnie++ output below! Thanks, Kent Bonnie++ on plain system (not Dom0) Version 1.03d ------ Sequential Output ------ --- Sequential Input --- --Random- -Per Char- --Block-- -Rewrite- --Per Char-- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP san 16G 74241 93 304695 80 199290 57 74182 99 601348 69 1812 5 ------ Sequential Create ------ -------- Random Create -------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 16 23050 99 +++++ +++ +++++ +++ 22865 99 +++++ +++ 32317 92 Bonnie++ on Dom0 Version 1.03d ------ Sequential Output ------ --- Sequential Input --- --Random- -Per Char- --Block-- -Rewrite- --Per Char-- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP san 8G 67051 85 166574 59 126454 63 71058 96 362940 76 1645 11 ------ Sequential Create ------ -------- Random Create -------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 16 15815 99 +++++ +++ 22903 99 15222 99 +++++ +++ 22339 99 Bonnie++ in zvol-backed DomU (openbsd 4.3, hvm) Version 1.03 ------ Sequential Output ------ --- Sequential Input --- --Random- -Per Char- --Block-- -Rewrite- --Per Char-- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP puffy 300M 451 0 295 0 184 0 976 4 1521 2 26.8 1 ------ Sequential Create ------ -------- Random Create -------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 16 8 1 +++++ +++ 16 1 8 1 +++++ +++ 10 0
David Edmondson
2008-Sep-25 11:40 UTC
Re: Help with b97 HVM zvol-backed DomU disk performance
On Thu, Sep 25, 2008 at 07:06:22AM -0400, Kent Watsen wrote:> I was wondering if anyone has an pointers/suggestions on how I might > increase disk performance of a HVM zvol-backed DomU? - this is my > first DomU, so hopefully its something obviousIf you don''t have PV disk drivers for the HVM guest then performance will be poor - dom0 is emulating an IDE controller so that the guest can believe that it has one. This is never going to perform very well. Are there PV drivers for OpenBSD?
David Edmondson wrote: On Thu, Sep 25, 2008 at 07:06:22AM -0400, Kent Watsen wrote: I was wondering if anyone has an pointers/suggestions on how I might increase disk performance of a HVM zvol-backed DomU? - this is my first DomU, so hopefully its something obvious If you don''t have PV disk drivers for the HVM guest then performance will be poor - dom0 is emulating an IDE controller so that the guest can believe that it has one. This is never going to perform very well. Are there PV drivers for OpenBSD? No, unfortunately, the OpenBSD project does not yet support PV drivers, but is lack of PV really suppose to be this bad? I mean, its practically unusable. For examples, here is a large file copy test I performed (note: its GE over cat5, which is likely the bottleneck for the "Dom0" tests) Copy large file from Desktop to Dom (using `scp`) : Dom0: ~27136KB/s DomU: ~322KB/s Copy large file from Dom to Desktop (using `scp`): Dom0: ~28441KB/s DomU: ~662KB/s BTW, My goal is to host web and mail servers from my house. One good news item with this is that my FIOS-based ISP speeds are 1933Kb/s download and 231Kb/s upload, so at least the disk performance would be the bottleneck in serving up the web content... Thanks, Kent
David Edmondson
2008-Sep-26 05:16 UTC
Re: Help with b97 HVM zvol-backed DomU disk performance
On Thu, Sep 25, 2008 at 11:59:01AM -0400, Kent Watsen wrote:> Are there PV drivers for OpenBSD? > > No, unfortunately, the OpenBSD project does not yet support PV drivers, but is > lack of PV really suppose to be this bad? I mean, its practically unusable. > For examples, here is a large file copy test I performed (note: its GE over > cat5, which is likely the bottleneck for the "Dom0" tests) > > Copy large file from Desktop to Dom (using `scp`) : > Dom0: ~27136KB/s > DomU: ~322KB/s > > Copy large file from Dom to Desktop (using `scp`): > Dom0: ~28441KB/s > DomU: ~662KB/sThat''s worse that I''ve seen with other HVM guests, where 30Mb/s was about the norm. If this was just disk or net then I''d suspect that the device emulation simply didn''t match up well with the OpenBSD driver, but given that you''re seeing both as slow it may well be a broader problem. In your research, have you seen reports of OpenBSD running under HVM on other ''based on the work of the Xen open source community'' platforms?
David Edmondson wrote:
That''s
worse that I''ve seen with other HVM guests, where 30Mb/s was about the
norm.
Hmmm, that would be about 2.5 times faster than what I''m seeing... I
wish my OpenBSD HVM DomU disk performance was that good...
If this was just disk or net then I''d suspect that the device
emulation simply didn''t match up well with the OpenBSD driver, but
given that you''re seeing both as slow it may well be a broader
problem.
I hadn''t really said anything about the DomU''s networking
performance
but, since you mention it, using `iperf` shows I have 8.4Mbit/sec
upload and 6.7Mbit/sec download between my OpenBSD DomU and my
Desktop. This means that my networking throughput is about 10 times
faster than my file-transfer tests, where the disk was [likely] the
bottleneck. However, even these networking numbers are less than what
you say other HVM DomU''s file-copies. BTW, running `iperf` on Dom0
gives 400Mbit/sec upload and 800Mbits/sec download to my Desktop. That
is, the OpenBSD DomU''s networking performance is about 80 times slower
than the Dom0''s
Is there any suggestion for matching the "device emulation" better or,
pointers toward what can be done if it is a "broader problem"
In your research, have you seen reports of OpenBSD running under HVM
on other ''based on the work of the Xen open source community''
platforms?
I have searched; there a few reports of running OpenBSD HVM on various
Dom0 with all saying that performance was not great, but because those
posts didn''t include performance numbers, its hard to say how much they
resemble my situation.
BTW, I just installed CentOS with PV and had much better results:
Bonnie:
Version 1.03 ------ Sequential Output ------
--- Sequential Input --- --Random-
-Per Char- --Block-- -Rewrite- --Per Char--
--Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP
K/sec %CP K/sec %CP
centos 1G 33103 96 98774
37 36682 8 35704
82 477729 39
+++++ +++
------ Sequential Create ------ -------- Random
Create --------
-Create-- --Read--- -Delete-- -Create--
--Read--- -Delete--
files K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec
%CP K/sec %CP
16 22912 98 +++++ +++ +++++
+++ 21896 99 +++++
+++ +++++ +++
File transfer tests:
Copy large file from Desktop to Dom (using `scp`) :
Dom0: ~27136KB/s
DomU: ~24780KB/s (CentOS-based)
Copy large file from Dom to Desktop (using `scp`):
Dom0: ~28441KB/s
DomU: ~22732KB/s (CentOS-based)
Iperf tests:
938 Mbits/sec
BTW, I am confused by that last test - if true, then PV CentOS guest
networking is 30% faster than OpenSolaris''s Dom0''s - weird. I
ran the
test multiple times, as well as the tests again the Dom0 with
consistent results, so I''m not sure how its possible...
Thanks,
Kent