CentOS virt - Aug 2008 - iowait / Perfomance problems with xen on drbd (centos 5.2)

Hi,

we have 2 servers on centos5.2 in a cluster with the redhat cman in the
configuration

SERVER1 > LVM \
CMAN           DRBD > XEN
SERVER2 > LVM /

For each xen we create a new local LVM on each node, put them in a drbd
and install an os (debian 4 or ubuntu 8). The xens are running as pvm.

Everything works fine since over 4 months now.

But we have some performance-Problems:

Writing on the Xen-devices produces an iowait about 50 to 60% on the
xen-processors.

we have 3 szenarios tested:

a) Xen on DRBD
b) Xen on DRBD but disconnected
c) direct mounted DRBD

You can see the difference in Write pro Char and Write per Block.

a: 3792 / 4011
b: 52037 / 103777
c: 57135 / 325002

See bonnie_result.txt for more data.

We attached our drbd.conf and a xen-config for your information.

The Server are each 2xDual-Core AMD Opteron(tm) Processor 2214 HE with
32 Gb RAM and 2TB-Harddrive on Raid5 running von CentOS 5.2. The
XEN-DomUs are Debian 4 or Ubuntu 8. The NICs are bonded Intel 1GBit. For
the DRBD we have an own connection on a seperately subnet.

Though we are wondering what there happens. Any Idea?

And: YES, we tried a some configurations in the last past months before
we are asking you today  :)

Best regards an Thank you in advance
Jan


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: drbd.conf.txt
URL:
<http://lists.centos.org/pipermail/centos-virt/attachments/20080818/db6fd1e2/attachment-0012.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: xen_config.txt
URL:
<http://lists.centos.org/pipermail/centos-virt/attachments/20080818/db6fd1e2/attachment-0013.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: bonnie_result.txt
URL:
<http://lists.centos.org/pipermail/centos-virt/attachments/20080818/db6fd1e2/attachment-0014.txt>

<p>Hello, Jan.<br /><br /><br />If you are using 2.6.18
kernel you can disable DEVFS.<br /><br />Follow this steps to ensure
that you don't have forgot anything.<br /><br
/>http://wiki.xen-br.org/index.php?title=Xen-ha<br /><br /><br
/><br />Em 15:30, <strong><span>Jan
Kellermann  </span></strong> escreveu: </p><blockquote
style="padding-left: 1ex; margin: 0pt 0pt 0pt 0.8ex; border-left: #6868cc
2px solid"><br />Hi,<br /><br />we have 2 servers on
centos5.2 in a cluster with the redhat cman in the<br
/>configuration<br /><br />SERVER1 > LVM \<br />CMAN
DRBD > XEN<br />SERVER2 > LVM /<br /><br />For each xen
we create a new local LVM on each node, put them in a drbd<br />and
install an os (debian 4 or ubuntu 8). The xens are running as pvm.<br
/><br />Everything works fine since over 4 months now.<br
/><br />But we have some performance-Problems:<br /><br
/>Writing on the Xen-devices produces an iowait about 50 to 60% on the<br
/>xen-processors.<br /><br />we ha
 ve 3 szenarios tested:<br /><br />a) Xen on DRBD<br />b) Xen
on DRBD but disconnected<br />c) direct mounted DRBD<br /><br
/>You can see the difference in Write pro Char and Write per Block.<br
/><br />a: 3792 / 4011<br />b: 52037 / 103777<br />c: 57135
/ 325002<br /><br />See bonnie_result.txt for more data.<br
/><br />We attached our drbd.conf and a xen-config for your
information.<br /><br />The Server are each 2xDual-Core AMD
Opteron(tm) Processor 2214 HE with<br />32 Gb RAM and 2TB-Harddrive on
Raid5 running von CentOS 5.2. The<br />XEN-DomUs are Debian 4 or Ubuntu 8.
The NICs are bonded Intel 1GBit. For<br />the DRBD we have an own
connection on a seperately subnet.<br /><br />Though we are
wondering what there happens. Any Idea?<br /><br />And: YES, we
tried a some configurations in the last past months before<br />we are
asking you today :)<br /><br />Best regards an Thank you in
advance<br />Jan<br /><br /><br /><br /><br
/></blockquote>