Xen devel - Oct 2004 - Block Device Cache Coherency

My understanding of the Xen block device backend is that requests from
xenU domains are passed off rather directly to the underlying block
device driver in domain-0, without going through the page/buffer cache.
This leads me to wonder: are situations like the following possible?

  - Domain-0 initializes a disk partition for a new domain.  Some of the
    data is buffered in memory and not immediately flushed to disk.  The
    new domain is started, requests data from the disk, and reads the
    old data because the new data hasn''t hit disk yet.
  - Domain-0 had earlier read some data from a disk partition used by
    domain-1 and has this data cached.  Domain-1 writes some changes to
    disk, then shuts down.  A program in domain-0 wishes to look at the
    filesystem used by domain-1.  The changes made by domain-1 have been
    written to disk, but domain-0 doesn''t realize its cached data is
    stale and reuses the old data.

Both are cases where there isn''t any concurrent access to the device,
but domain-0 could do the wrong thing because it doesn''t realize that
domain-1 needs to use or has recently changed the same data.

Does the driver domain (domain-0) do something to prevent cases such as
the above happening, or is this a possible concern?  (I haven''t
observed
any problems along these lines in real use, but am curious.)

In a related question, if LVM is being used to share a device using
copy-on-write partitions (or even if the same device is being shared
read-only between several domains), will each read from a different
domain require a separate read from disk?  Is there some way to have,
say, domain-0 store some of the data in the page cache so that
subsequent requests for the same physical sectors can be satisfied
immediately, in return for giving up some performance and resource
isolation?

Thanks, and sorry if I''m managing to completely misinterpret how the
device drivers work and asking meaningless questions.

--Michael Vrable


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

> Both are cases where there isn''t any concurrent access to the
device,
> but domain-0 could do the wrong thing because it doesn''t realize
that
> domain-1 needs to use or has recently changed the same data.
> 
> Does the driver domain (domain-0) do something to prevent cases such as
> the above happening, or is this a possible concern?  (I haven''t
observed
> any problems along these lines in real use, but am curious.)
A good question.

You should never have a disc partition mounted by multiple domains,
unless all domains have read-only access. If you follow this rule then
you can''t end up with inconsistent stale buffer-cache pages because
they get flushed on umount.

To this end, I think xend does multi-reader single-writer access
control on disc partitions (at least, it used to).
> In a related question, if LVM is being used to share a device using
> copy-on-write partitions (or even if the same device is being shared
> read-only between several domains), will each read from a different
> domain require a separate read from disk?  Is there some way to have,
> say, domain-0 store some of the data in the page cache so that
> subsequent requests for the same physical sectors can be satisfied
> immediately, in return for giving up some performance and resource
> isolation?
Another good question. :-)

There is no shared buffer cache amongst domains, so every domain must
fill its cache separately and this will require a separate set of disc
reads for each domain, even for common data.

If you use a loopback-mounted file (''losetup'') or NFS to
access
filesystems then you might end up going through DOM0''s buffer cache,
which could get you some benefit. BUT the fact would remain that you
still have multiple copies of the shared data, in other domains''
buffer caches (i.e., you''d save some disc accesses, but
wouldn''t save
any memory).

What would be fun is to come up with a proper shared buffer cache that
allows shared read access amongst domains to a single shared page,
and have a scheme for copy-on-write.

Lots of opportunity for fun and games here: e.g., how to account
shared memory, how to size the shared buffer cache, and doing a nice
comparisons against Xen''s current strong isolation model.

 -- Keir


-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
Use IT products in your business? Tell us what you think of them. Give us
Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel