search for: can_cache

...dkit-python-plugin.pod b/plugins/python/nbdkit-python-plugin.pod index 0fd4dcb..2bc4722 100644 --- a/plugins/python/nbdkit-python-plugin.pod +++ b/plugins/python/nbdkit-python-plugin.pod @@ -212,6 +212,13 @@ contents will be garbage collected. def can_trim(h): # return a boolean +=item C<can_cache> + +(Optional) + + def can_cache(h): + # return a boolean + =item C<pread> (Required) @@ -293,6 +300,19 @@ because there is nothing to optimize if S<C<flags & nbdkit.FLAG_MAY_TRIM>> is false), use S<C<nbdkit.set_error (errno.EOPNOTSUPP)>>. +=item C<...

Since v1: - rework .can_cache to be tri-state, with default of no advertisement (ripple effect through other patches) - add a lot more patches in order to round out filter support And in the meantime, Rich pushed NBD_CMD_CACHE support into libnbd, so in theory we now have a way to test cache commands through the entire stack....

Implementing extents requires some coordination for the Rust code to call back into libnbdkit; I'm not familiar with Rust enough to do that. But with placeholders for those slots, implementing can_cache/cache is trivial. This improves the situation mentioned in commit 031fae85. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/rust/nbdkit-rust-plugin.pod | 10 ++++++++++ plugins/rust/src/lib.rs | 13 +++++++++++++ 2 files changed, 23 insertions(+) diff --git a/plugins...

I'm still working my way through the filters before this series will be complete, but this is enough of a start to at least get some feedback on the idea of implementing another NBD protocol extension. Eric Blake (9): server: Internal hooks for implementing NBD_CMD_CACHE plugins: Add .cache callback file, split: Implement .cache with posix_fadvise nbd: Implement NBD_CMD_CACHE

...guages we can detect if (eg) a C<pwrite> method is defined and synthesize an appropriate response if no actual C<can_write> method is defined. @@ -243,13 +243,20 @@ The script should exit with code C<0> for true or code C<3> for false. =item C<can_fua> +=item C<can_cache> + /path/to/script can_fua <handle> + /path/to/script can_cache <handle> -This controls Forced Unit Access (FUA) behaviour of the core server. +These control Forced Unit Access (FUA) and caching behaviour of the +core server. -Unlike the other C<can_*> callbacks, this one i...

On 8/16/19 12:08 PM, Eric Blake wrote: > Implementing extents requires some coordination for the Rust code to > call back into libnbdkit; I'm not familiar with Rust enough to do > that. But with placeholders for those slots, implementing > can_cache/cache is trivial. This improves the situation mentioned in > commit 031fae85. > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- > +++ b/plugins/rust/src/lib.rs > @@ -93,6 +93,15 @@ pub struct Plugin { > pub magic_config_key: *const c_char, > > pu...

...l the next few patches expose and implement new callbacks for filters and plugins, our initial implementation for NBD_CMD_CACHE is to just blindly advertise the feature, and call into .pread with an ignored buffer. Note that for bisection reasons, this patch treats any use of a filter as a forced .can_cache of false to bypass the filter's caching; once all affected filters are patched to handle cache requests correctly, a later patch will then switch the filter default to passthrough for the sake of remaining filters. Signed-off-by: Eric Blake <eblake@redhat.com> --- Note: we could also c...

...PM, Eric Blake wrote: > On 8/16/19 12:08 PM, Eric Blake wrote: >> Implementing extents requires some coordination for the Rust code to >> call back into libnbdkit; I'm not familiar with Rust enough to do >> that. But with placeholders for those slots, implementing >> can_cache/cache is trivial. This improves the situation mentioned in >> commit 031fae85. >> >> Signed-off-by: Eric Blake <eblake@redhat.com> >> --- > And of course, if you want to actually implement extents, and figure out > how to expose C-based nbdkit functions to be c...

...t; > On 8/16/19 12:08 PM, Eric Blake wrote: > >> Implementing extents requires some coordination for the Rust code to > >> call back into libnbdkit; I'm not familiar with Rust enough to do > >> that. But with placeholders for those slots, implementing > >> can_cache/cache is trivial. This improves the situation mentioned in > >> commit 031fae85. > >> > >> Signed-off-by: Eric Blake <eblake@redhat.com> > >> --- > > > And of course, if you want to actually implement extents, and figure out > > how to expo...

...1 insertions(+), 1 deletion(-) diff --git a/plugins/ocaml/ocaml.c b/plugins/ocaml/ocaml.c index 4447d7f..f664a7f 100644 --- a/plugins/ocaml/ocaml.c +++ b/plugins/ocaml/ocaml.c @@ -128,6 +128,9 @@ static value can_multi_conn_fn; static value can_extents_fn; static value extents_fn; +static value can_cache_fn; +static value cache_fn; + /*----------------------------------------------------------------------*/ /* Wrapper functions that translate calls from C (ie. nbdkit) to OCaml. */ @@ -638,6 +641,48 @@ extents_wrapper (void *h, uint32_t count, uint64_t offset, uint32_t flags, CAMLreturnT (int,...

It's easy to expose new callbacks to sh plugins. I have no idea how any plugin would actually usefully cache anything in shell (you can't really store it in a shell variable, since a new shell process is started for each command); but if nothing else, this allows a user to implement .can_cache returning true to bypass nbdkit's normal fallback to .pread. The shell plugin, coupled with Rich's work on libnbd as a client-side library for actually exercising calls to NBD_CMD_CACHE, will be a useful way to prove that cache commands even make it through the stack. (Remember, qemu 3.0 w...

...and implement new callbacks for > filters and plugins, our initial implementation for NBD_CMD_CACHE is > to just blindly advertise the feature, and call into .pread with an > ignored buffer. > > Note that for bisection reasons, this patch treats any use of a filter > as a forced .can_cache of false to bypass the filter's caching; once > all affected filters are patched to handle cache requests correctly, a > later patch will then switch the filter default to passthrough for the > sake of remaining filters. > > Signed-off-by: Eric Blake <eblake@redhat.com> &g...

...that the cache of can_zero answers the filter semantics (whether we advertise zero support in eflags) and not the plugin semantics (whether we should attempt .zero or just automatically fall back to .pwrite); that will be improved in the next patch. In the cow filter, adding a call to next_ops->can_cache in .prepare means that .cache no longer has to worry about can_cache failing (the can_FOO methods do not guarantee easy access to a sane errno value). Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-filter.pod | 9 +-- docs/nbdkit-plugin.pod | 6 +- server/in...

...False) + +def can_fua(h): + fua = cfg.get ('can_fua', "none") + if fua == "none": + return nbdkit.FUA_NONE + elif fua == "emulate": + return nbdkit.FUA_EMULATE + elif fua == "native": + return nbdkit.FUA_NATIVE + +def can_cache(h): + cache = cfg.get ('can_cache', "none") + if cache == "none": + return nbdkit.CACHE_NONE + elif cache == "emulate": + return nbdkit.CACHE_EMULATE + elif cache == "native": + return nbdkit.CACHE_NATIVE + +def pread(...

...plugin), ._api_version = NBDKIT_API_VERSION, + ._thread_model = NBDKIT_THREAD_MODEL_PARALLEL, /* The following field is used as a canary to detect whether the * OCaml code started up and called us back successfully. If it's @@ -131,6 +132,8 @@ static value extents_fn; static value can_cache_fn; static value cache_fn; +static value thread_model_fn; + /*----------------------------------------------------------------------*/ /* Wrapper functions that translate calls from C (ie. nbdkit) to OCaml. */ @@ -683,18 +686,30 @@ cache_wrapper (void *h, uint32_t count, uint64_t offset, uint3...

v1: https://www.redhat.com/archives/libguestfs/2019-November/msg00153.html v2: - Fix implementation of can_cache. - Add implementation of can_fua. - Add a very thorough test suite which tests every command + flag combination.

...g.get ('can_fua', "none") > + if fua == "none": > + return nbdkit.FUA_NONE > + elif fua == "emulate": > + return nbdkit.FUA_EMULATE > + elif fua == "native": > + return nbdkit.FUA_NATIVE > + > +def can_cache(h): > + cache = cfg.get ('can_cache', "none") > + if cache == "none": > + return nbdkit.CACHE_NONE > + elif cache == "emulate": > + return nbdkit.CACHE_EMULATE > + elif cache == "native": > + retur...

v3 was here: https://www.redhat.com/archives/libguestfs/2019-November/msg00209.html In v4: - Rebase on top of current master. Includes various fixes and updates required because of Nir's patches that went into master. - Fix api_version() -> API_VERSION in patch 2 noted previously on the mailing list. Rich.

...onfig_key>, -C<can_extents>, -C<extents>. +C<magic_config_key>. These are not yet supported. diff --git a/plugins/python/python.c b/plugins/python/python.c index 398473f5..585cd9e6 100644 --- a/plugins/python/python.c +++ b/plugins/python/python.c @@ -1020,6 +1020,79 @@ py_can_cache (void *handle) return NBDKIT_CACHE_NONE; } +static int +py_can_extents (void *handle) +{ + ACQUIRE_PYTHON_GIL_FOR_CURRENT_SCOPE; + return boolean_callback (handle, "can_extents", "extents"); +} + +static int +py_extents (void *handle, uint32_t count, uint64_t offset, +...

v2 was here: https://www.redhat.com/archives/libguestfs/2019-November/msg00163.html I pushed patch 1 (with spelling fix), patch 4 and patch 5 since those were previously ACKed on the list. Differences in v3: - Add error checking to PyModule_AddIntConstant. - Use API_VERSION constant instead of function. - Add max API version supported to --dump-plugin output. - Print API_VERSION selected by