Xen users - Feb 2012 - Terribad "file:" performance (vs reasonable "phy:" perf)

Howdy,

I''ve done a bit more research, and found some answers to my earlier
question about qemu-dm.  It would seem that a 64-bit 3.x kernel does not have
blktap drivers available.  And, according to the docs in
docs/misc/xl-disk-configuration.txt, when a domU disk is specified as:

	file:/path/to/file

the rule seems to be: "For block device type ''file'' with
format raw or when no format specfied, tapdisk2 is used when present otherwise
qemu fallback option is used."

I dug through some older sources that suggested using the "phy:"
block-device-type with a loop device (e.g., /dev/loop0).  This worked, and gave
an order-of-magnitude improvement.  Here''s what I''ve observed:
on a quad-core i5-2400 with a single sata HD, I see this type of performance:

  * A physical disk partition           - phy:/dev/sda7       - ~55 MB/s
  * A loop device using the phy: driver - phy:/dev/loop0      - ~35 MB/s
  * A qemu-dm fallback                  - file:/xen/domU.img  - < 3 MB/s (!)

Before pursuing this any further, it would be reassuring to hear that other
people have seen similar results, or if I''m barking up the wrong tree. 
Specifically, my configuration is Xen-4.1.2 on a 64-bit Linux-3.1.0 dom0
(dom0_mem=2048), with a 64-bit Linux-3.1.0 domU configured as a PV host. 
Neither host nor guest is multilib or has 32-bit libraries.  And, I''m
using xl (not xm) to start my instance.  Have the collective "you"
seen numbers like these, or is this a sign that I''m missing something?

In addition, even though that same doc suggests that these "device type
specifiers" like "file:" or "phy:" are deprecated,
until a blktap driver is ported to a pure-64-bit environment, they are still
necessary, right?  My 64-bit build doesn''t have blktapctl (or whatever
that daemon is called), and when I leave out the type specifier, the domU hangs
on waiting for XENBUS, which--along with the various log outputs--seemed to
indicate that it was stuck trying to load a blktap driver which doesn''t
exist.

In short, I''d like to have a reasonable assurance that my choice for
non-fixed partitions (along with my other system constraints) is using a
"phy:" device type with either: 1) loop devices I create by hand at
the start of each VM, or 2) using LVM, which creates virtual block devices.

Are those my choices aside from qemu-dm (which appears to suck wind)?

	Q

Hi,

I don''t have sufficient testing experience, but I do have experience
concerning loop (randomly used it a few years from Xen2 to Xen3). Do
NOT USE loop if you have any valuable data in the virtual machines.
There is enough written about it in the archives here and most Xen books.

Florian

On Feb 3, 2012, at 2:49 PM, Florian Heigl wrote:
> I don''t have sufficient testing experience, but I do have
experience
> concerning loop (randomly used it a few years from Xen2 to Xen3). Do
> NOT USE loop if you have any valuable data in the virtual machines.
> There is enough written about it in the archives here and most Xen books.
Thanks for this info.  Do you happen to have some URLs I could look at (or
suggest a search term for me) to show that /dev/loopN is bad?  From my own
testing, I would imagine it might have to do with sync semantics; I''d
be very interested in seeing what the specific concerns are.

There seem to be a few alternatives to loop:

	1: blktap
	2: qemu-dm
	3: phy:/dev/sdXY    (blkback)
	4: LVM              (also blkback)

1: I don''t believe I have access to the blktap driver.  For anyone else
looking at disk performance issues and stumbles onto this message, here is my
setup (which will probably not resemble yours):
> Specifically, my configuration is Xen-4.1.2 on a 64-bit Linux-3.1.0 dom0
(dom0_mem=2048), with a 64-bit Linux-3.1.0 domU configured as a PV host. 
Neither host nor guest is multilib or has 32-bit libraries.  And, I''m
using xl (not xm) to start my instance.
2: Performance seems abysmal.

3: Yes, this is a potential last resort, but a fixed-partition scheme is truly
horrid from a management perspective.

4: A new avenue to investigate.

===
Is there an option I''m unaware of...?

	Q