Xen users - Jan 2008 - Re; TAP:AIO driver

Hi, 

Is there any reason / instance where TAP:AIO might not work where FILE would
work?

For instance is TAP:AIO designed to work on network filesystems or local devices
only?

I''ve been told that FILE is bad, however, is this in the context of
local devices, or also in the context of network filesystems ... ?

tia 
Gareth. 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> Is there any reason / instance where TAP:AIO might not work where FILE
> would work?
It was never a question of file flat-out not working; just that in some cases 
it performs badly in various ways and causes instability.
> For instance is TAP:AIO designed to work on network filesystems or local
> devices only?
Both.
> I''ve been told that FILE is bad, however, is this in the context
of local
> devices, or also in the context of network filesystems ... ?
tap:aio is recommended generally, I believe.

On a local filesystem it has the advantage of not using up your loop devices.  
It may have better performance but I''ve not seen any numbers recently,
so I''m
not sure.  It may also have better memory use characteristics (see below).

The worst configuration I''m aware of for file: devices is on a network 
filesystem (NFS being the specific source of complaints but I imagine similar 
issues could arise for others).  Using the loop device (which is what file: 
does) on a file on an NFS filesystem is apparently a really good way to 
provoke OOM (out of memory) conditions in dom0''s kernel, which will
probably
then start killing things randomly to free up memory.  Not good.

Using file-based VBDs on an NFS filesystem is a bit gross anyway and not 
something I''d recommend; but my understanding is that if you really
want to
do this, tap will be more stable.

(slightly confusingly, the same arguments probably don''t all apply to
HVM
guests without the PV drivers installed, since I don''t think file:/ in
that
case actually uses the loop device, although it probably does set it up)

tap also has the advantage of having some support for copy-on-write disk 
images using the qcow format, which might be something you want to look into 
if you have lots of domains based on a common base image.

Cheers,
Mark

-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Mmm, tvm.

For the record AIO does not work on my networkfilesystem (gluster) but my
configuration is yielding 67Mb/sec over a network filsystem .vs. about 75Mb/sec
on the local disk the filesystem is running off. (using file:)

If I try to use AIO it goes into a wait state trying to allocate the block
device on boot, counts down (on the screen) from 300 seconds .. then fails ..

.. Any idea why ?

Gareth.


----- Original Message -----
step 3.: "Mark Williamson" <mark.williamson@cl.cam.ac.uk>
To: xen-users@lists.xensource.com
Cc: "Gareth Bult" <gareth@encryptec.net>
Sent: 28 January 2008 15:31:02 o''clock (GMT) Europe/London
Subject: Re: [Xen-users] Re; TAP:AIO driver
> Is there any reason / instance where TAP:AIO might not work where FILE
> would work?
It was never a question of file flat-out not working; just that in some cases 
it performs badly in various ways and causes instability.
> For instance is TAP:AIO designed to work on network filesystems or local
> devices only?
Both.
> I''ve been told that FILE is bad, however, is this in the context
of local
> devices, or also in the context of network filesystems ... ?
tap:aio is recommended generally, I believe.

On a local filesystem it has the advantage of not using up your loop devices.  
It may have better performance but I''ve not seen any numbers recently,
so I''m
not sure.  It may also have better memory use characteristics (see below).

The worst configuration I''m aware of for file: devices is on a network 
filesystem (NFS being the specific source of complaints but I imagine similar 
issues could arise for others).  Using the loop device (which is what file: 
does) on a file on an NFS filesystem is apparently a really good way to 
provoke OOM (out of memory) conditions in dom0''s kernel, which will
probably
then start killing things randomly to free up memory.  Not good.

Using file-based VBDs on an NFS filesystem is a bit gross anyway and not 
something I''d recommend; but my understanding is that if you really
want to
do this, tap will be more stable.

(slightly confusingly, the same arguments probably don''t all apply to
HVM
guests without the PV drivers installed, since I don''t think file:/ in
that
case actually uses the loop device, although it probably does set it up)

tap also has the advantage of having some support for copy-on-write disk 
images using the qcow format, which might be something you want to look into 
if you have lots of domains based on a common base image.

Cheers,
Mark

-- 
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users