search for: noextents

tmpfs has a known bug of O(n^2) behavior with lseek(SEEK_HOLE); this is one situation where the noextents filter can come in handy to avoid the performance penalty of exposing accurate extents. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/noextents/nbdkit-noextents-filter.pod | 7 ++++++- plugins/file/nbdkit-file-plugin.pod | 9 ++++++++- 2 files changed, 14 insertions(+)...

...the image contains holes, and whether it is known to read as all zeroes. Most filters just pass through the plugin's result (as it is only checked once when first connecting), but we can do useful things in log (mention the setting), extentlist (answer based on the extents we are advertising), noextents (suppress the advertisement), and truncate (advertise sparse support for our added tail hole). Expand a comment in the cache filter about potential future enchnancements. Initial read-only zero status is easy to add through the extentlist filter (supply a filter list with no entries and the entir...

...quested SR, there is still a case to be made for allowing a filter to prevent SR, at least for testing purposese (such as comparison on what a client does with no block status vs. a block status that always reports allocated). Enhance the filter API to allow an SR inhibit, and wire up the existing noextents filter to expose this option. In particular, doing this found that 'qemu-nbd --list' from qemu 4.2 is rather picky: it hangs up on a server that replies with NBD_REP_ERR_POLICY, rather than silently proceeding without SR support (at least libnbd is more tolerant). Signed-off-by: Eric Blak...

...erver that replies with > NBD_REP_ERR_POLICY, rather than silently proceeding without SR support > (at least libnbd is more tolerant). > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- > > Looks much different as a command line option instead of a hack to > the noextents filter, but I like the result a lot better. > > I'm open to bike-shed opinions on the option name, or even if we want > to emulate a tri-state option --protocol=[old|no-sr|new] (with -o and > -n remaining synonyms for 2 of the 3 options for back-compat). I > purposefully did no...

...4.2 is rather picky: it hangs up on a server that replies with NBD_REP_ERR_POLICY, rather than silently proceeding without SR support (at least libnbd is more tolerant). Signed-off-by: Eric Blake <eblake@redhat.com> --- Looks much different as a command line option instead of a hack to the noextents filter, but I like the result a lot better. I'm open to bike-shed opinions on the option name, or even if we want to emulate a tri-state option --protocol=[old|no-sr|new] (with -o and -n remaining synonyms for 2 of the 3 options for back-compat). I purposefully did not burn a short-option let...

On Mon, Feb 10, 2020 at 03:43:56PM -0600, Eric Blake wrote: > diff --git a/filters/noextents/noextents.c b/filters/noextents/noextents.c > index e6ac33b..091f30b 100644 > --- a/filters/noextents/noextents.c > +++ b/filters/noextents/noextents.c > @@ -1,5 +1,5 @@ > /* nbdkit > - * Copyright (C) 2019 Red Hat Inc. > + * Copyright (C) 2019-2020 Red Hat Inc. > * >...

Similar to the existing fua, nozero and noextents filters, add a filter to make it easy to override the basic caching functionality, in part to facilitate timing tests of whether a plugin's cache implementation is worthwhile. A worthwhile test to add to the testsuite would connect the log filter both before and after the nocache filter, to pr...

...reduce thread model dynamically noparallel: Implement new filter filters: Use only .thread_model, not THREAD_MODEL docs/nbdkit-filter.pod | 45 ++++++---- filters/fua/nbdkit-fua-filter.pod | 1 + filters/nocache/nbdkit-nocache-filter.pod | 1 + filters/noextents/nbdkit-noextents-filter.pod | 1 + .../noparallel/nbdkit-noparallel-filter.pod | 66 ++++++++++++++ filters/nozero/nbdkit-nozero-filter.pod | 1 + configure.ac | 2 + include/nbdkit-filter.h | 7 +- filters/cow/cow.c...

...upport is disabled. To work around this you can use nbdkit --readonly. However I don't understand why that would be necessary, except perhaps it's just an undocumented limitation of VDDK. For all the cases _we_ care about we're using --readonly, so that's lucky. (3) Using nbdkit-noextents-filter and nbdkit-stats-filter we can nicely measure the benefits of extents: With noextents (ie. force full copy): elapsed time: 323.815 s read: 8194 ops, 17179869696 bytes, 4.24437e+08 bits/s Without noextents (ie. rely on qemu-img skipping sparse bits): elapsed time: 237.41 s read: 8...

...include/nbdkit-filter.h | 6 ++--- filters/cow/cow.c | 2 -- filters/delay/delay.c | 2 -- filters/error/error.c | 2 -- filters/fua/fua.c | 2 -- filters/log/log.c | 2 -- filters/nocache/nocache.c | 2 -- filters/noextents/noextents.c | 2 -- filters/noparallel/noparallel.c | 2 -- filters/nozero/nozero.c | 2 -- filters/offset/offset.c | 2 -- filters/partition/partition.c | 2 -- filters/rate/rate.c | 2 -- filters/readahead/readahead.c | 2 -- filters/stats/stats.c...

I'm very lukewarm on this one. Why is this a plugin-level feature? For other protocol things we have used command line options (eg. -o / -n and the various TLS options). Also I understand that this found a bug in qemu-nbd which is great, but why else would it ever be needed? Maybe we can control this with some kind of command line protocol flags, and try to emphasize that, like the -D flag,

...ents/cacheextents.c | 1 - filters/cow/cow.c | 1 - filters/delay/delay.c | 1 - filters/error/error.c | 1 - filters/fua/fua.c | 1 - filters/log/log.c | 1 - filters/nocache/nocache.c | 1 - filters/noextents/noextents.c | 1 - filters/noparallel/noparallel.c | 1 - filters/nozero/nozero.c | 1 - filters/offset/offset.c | 1 - filters/partition/partition.c | 1 - filters/rate/rate.c | 1 - filters/readahead/readahead.c | 1 - filters/s...

...hild = ‘--run’ command so when the second child exits, the monitoring process (which is doing nothing except waiting for the second child to exit) can kill nbdkit. If VDDK cannot handle this situation (and I'm just guessing that this is the bug) then VDDK has a bug. > >(3) Using nbdkit-noextents-filter and nbdkit-stats-filter we can > >nicely measure the benefits of extents: > > > >With noextents (ie. force full copy): > > > > elapsed time: 323.815 s > > read: 8194 ops, 17179869696 bytes, 4.24437e+08 bits/s > > > >Without noextents (ie. rely...

...10 ++- docs/nbdkit-plugin.pod | 23 ++++++ filters/cache/blk.c | 7 +- filters/extentlist/extentlist.c | 28 +++++++ filters/log/log.c | 9 +- filters/log/nbdkit-log-filter.pod | 2 +- filters/noextents/nbdkit-noextents-filter.pod | 11 +-- filters/noextents/noextents.c | 18 +++- filters/truncate/truncate.c | 15 +++- include/nbdkit-filter.h | 8 +- include/nbdkit-plugin.h | 5 +- plugins/data/data.c...

...;s just an >undocumented limitation of VDDK. For all the cases _we_ care about >we're using --readonly, so that's lucky. > It might've been a safe measure for multiple accesses or something similar. Or "we'll implement that later" symptome. >(3) Using nbdkit-noextents-filter and nbdkit-stats-filter we can >nicely measure the benefits of extents: > >With noextents (ie. force full copy): > > elapsed time: 323.815 s > read: 8194 ops, 17179869696 bytes, 4.24437e+08 bits/s > >Without noextents (ie. rely on qemu-img skipping sparse bits): &gt...

The retry filter defaults to 5 retries, but when run with verbose details produces some confusing output: $ rm -f /tmp/inject; (sleep 5s; touch /tmp/inject)& ./nbdkit -f -v -U - \ null 1G --filter=retry --filter=noextents --filter=error error-rate=100% \ error-file=/tmp/inject --filter=delay rdelay=1 \ --run 'qemu-img convert $nbd out.img' ... nbdkit: null[1]: debug: noextents: pread count=2097152 offset=14680064 nbdkit: null[1]: debug: error: pread count=2097152 offset=14680064 nbdkit: null[1]: error: i...

...04:00:22PM -0500, Eric Blake wrote: >> The retry filter defaults to 5 retries, but when run with verbose >> details produces some confusing output: >> >> $ rm -f /tmp/inject; (sleep 5s; touch /tmp/inject)& ./nbdkit -f -v -U - \ >> null 1G --filter=retry --filter=noextents --filter=error error-rate=100% \ >> error-file=/tmp/inject --filter=delay rdelay=1 \ >> --run 'qemu-img convert $nbd out.img' >> ... >> nbdkit: null[1]: debug: noextents: pread count=2097152 offset=14680064 >> nbdkit: null[1]: debug: error: pread count=20...

...this situation (and I'm just guessing that this > >is the bug) then VDDK has a bug. > > > > Sure, but having a workaround could be nice, if it's not too much work. Patches welcome, but I suspect there's not a lot we can do in nbdkit > >>>(3) Using nbdkit-noextents-filter and nbdkit-stats-filter we can > >>>nicely measure the benefits of extents: > >>> > >>>With noextents (ie. force full copy): > >>> > >>> elapsed time: 323.815 s > >>> read: 8194 ops, 17179869696 bytes, 4.24437e+08 bits/s...

...es for the APIs? Is the Init called before the first child forks off? >If VDDK cannot handle this situation (and I'm just guessing that this >is the bug) then VDDK has a bug. > Sure, but having a workaround could be nice, if it's not too much work. >> >(3) Using nbdkit-noextents-filter and nbdkit-stats-filter we can >> >nicely measure the benefits of extents: >> > >> >With noextents (ie. force full copy): >> > >> > elapsed time: 323.815 s >> > read: 8194 ops, 17179869696 bytes, 4.24437e+08 bits/s >> > >>...

...c Blake wrote: > >>The retry filter defaults to 5 retries, but when run with verbose > >>details produces some confusing output: > >> > >>$ rm -f /tmp/inject; (sleep 5s; touch /tmp/inject)& ./nbdkit -f -v -U - \ > >> null 1G --filter=retry --filter=noextents --filter=error error-rate=100% \ > >> error-file=/tmp/inject --filter=delay rdelay=1 \ > >> --run 'qemu-img convert $nbd out.img' > >>... > >>nbdkit: null[1]: debug: noextents: pread count=2097152 offset=14680064 > >>nbdkit: null[1]: debug: e...