Libguestfs - Feb 2021 - [RFC nbdkit PATCH] multi-conn: New filter

On 2/24/21 11:33 AM, Eric Blake wrote:
> Implement a TODO item of emulating multi-connection consistency via
> multiple plugin flush calls to allow a client to assume that a flush
> on a single connection is good enough.  This also gives us some
> fine-tuning over whether to advertise the bit, including some setups
> that are unsafe but may be useful in timing tests.
> 
> Testing is interesting: I used the sh plugin to implement a server
> that intentionally keeps a per-connection cache.
> 
> Note that this filter assumes that multiple connections will still
> share the same data (other than caching effects); effects are not
> guaranteed when trying to mix it with more exotic plugins like info
> that violate that premise.
> ---
> 
> I'm still working on the test; the sh plugin is good enough that it
> does what I want when playing with it manually, but I still need to
> write up various scenarios in test-multi-conn.sh to match what I've
> played with manually.
> 
> I'm open to feedback on the set of options I've exposed during
.config
> (too many, not enough, better names?)  Right now, it is:
> multi-conn-mode=auto|plugin|disable|emulate|unsafe
> multi-conn-track-dirty=fast|connection|off
Another thing I thought about: should I add a knob that lets multi-conn
know whether export names are significant?  When set, it only has to
maintain consistency by flushing on other connections visiting the same
export name, and not all connections.


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

On Wed, Feb 24, 2021 at 11:40:09AM -0600, Eric Blake
wrote:
> On 2/24/21 11:33 AM, Eric Blake wrote:
> > Implement a TODO item of emulating multi-connection consistency via
> > multiple plugin flush calls to allow a client to assume that a flush
> > on a single connection is good enough.  This also gives us some
> > fine-tuning over whether to advertise the bit, including some setups
> > that are unsafe but may be useful in timing tests.
> > 
> > Testing is interesting: I used the sh plugin to implement a server
> > that intentionally keeps a per-connection cache.
> > 
> > Note that this filter assumes that multiple connections will still
> > share the same data (other than caching effects); effects are not
> > guaranteed when trying to mix it with more exotic plugins like info
> > that violate that premise.
> > ---
> > 
> > I'm still working on the test; the sh plugin is good enough that
it
> > does what I want when playing with it manually, but I still need to
> > write up various scenarios in test-multi-conn.sh to match what
I've
> > played with manually.
> > 
> > I'm open to feedback on the set of options I've exposed during
.config
> > (too many, not enough, better names?)  Right now, it is:
> > multi-conn-mode=auto|plugin|disable|emulate|unsafe
> > multi-conn-track-dirty=fast|connection|off
The patch itself is fine.  Was there a particular reason to post it as
an RFC?  I think it should be fine as it is.  The one though was ...
> Another thing I thought about: should I add a knob that lets multi-conn
> know whether export names are significant?  When set, it only has to
> maintain consistency by flushing on other connections visiting the same
> export name, and not all connections.
... While there are only a few plugins where the exportname is vital
(eg. tmpdisk), and there are a few more where the exportname can be
significant but is not usually, it might be worth this extra knob.

I'm not sure how to implement it though - the filter only sees
exportnames flying past so it must remember ones it has seen.  But
then I think the problem is you may be in a position where you need to
issue a flush, but you no longer have the next_ops struct(?)  (It
would be the same kind of problem as the background threads TODO)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top