Xen users - Sep 2011 - [XCP] primary/primary DRBD 8.4.0-1 LVM-based shared SR (xcp 1.1) preformance tuning

Hi all,

we have followed the very good HOWTO by
http://wherethebitsroam.com/blogs/jeffw/drbd-xcp-05 and set up DRBD on
XCP 1.1 in primary/primary mode.

It works fine, but I am wondering how to squeeze more performance out of
the system (we currently use a crossover GB Ethernet connection). When
writing a 1 GB file on a guest I get write performance of about 5MB/s
(idle). We have disabled all swap partitions. The Disks are capable of
writing at much higher rate, as well the network connection allows a
much higher rate.

Because of active/active we have to use protocol C, but I am not sure if
this is the real bottleneck.

What are your experiences with performance improving DRBD performance.
Do you have any tricks?

When looking into performance tuning DRBD, I found the max-bio-bvecs
settings as a performance factor. I set it to 1 (which is slow). Is this
problem still the case (we are using DRBD version 8.4.0-1)? Can one
enforce the alignment settings for ext3/ext4?

    # In some special circumstances the device mapper stack manages to
    # pass BIOs to DRBD that violate the constraints that are set forth
    # by DRBD''s merge_bvec() function and which have more than one
bvec.
    # A known example is:
    # phys-disk -> DRBD -> LVM -> Xen -> missaligned partition (63)
->
DomU FS
    # Then you might see "bio would need to, but cannot, be split:" in
    # the Dom0''s kernel log.
    # The best workaround is to proper align the partition within
    # the VM (E.g. start it at sector 1024). (Costs 480 KiByte of storage)
    # Unfortunately the default of most Linux partitioning tools is
    # to start the first partition at an odd number (63). Therefore
    # most distribution''s install helpers for virtual linux machines
will
    # end up with missaligned partitions.
    # The second best workaround is to limit DRBD''s max bvecs per BIO
    # (= max-bio-bvecs) to 1. (Costs performance).
    # max-bio-bvecs 1;

Best,
Jakob


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> What are your experiences with performance improving DRBD performance.
> Do you have any tricks?
> 
Not any tricks, but if you want a comparison I''m just setting up the
following:

2 x DL165G7 servers with 2 x 2TB disks linked together via 2 x 1GB
Ethernet bonded in ''round robin'' mode. Running DRBD across the
bonded
link and then exporting via iSCSI across another 1GB link (will be
bonded to another link - just haven''t done it yet)

2 x DL165G7 servers running VM''s. I haven''t set up the
VM''s yet but I do
have an OCFS2 volume mounted on the iSCSI, and I''m getting around
37MB/s
with the following:

# dd if=/dev/zero of=test.bin bs=1M count=4096 conv=fdatasync,fsync
4096+0 records in
4096+0 records out
4294967296 bytes (4.3 GB) copied, 113.314 s, 37.9 MB/s

I assume OCFS2 adds some overhead... and I''m hoping I can tune it a bit
higher. hdparm on the iscsi volume gives the theoretical GB/s limit of
100MBit/s raw read speed so I think the data pipe itself is pretty good.

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users