search for: get_siz

...was given with a description in .list_exports. Note, however, that while we map "" into a canonical name prior to .open (as the plugin is easier to write if it can just assume the client already passed in the canonical name), we don't need a description until after .open, alongside .get_size. Note that in general, we cannot require the description given (or omitted) in .list_exports to match the description returned by .export_description. Whether a client asks for a listing of available exports is orthogonal to whether a client requests a description on a single export. The recent...

We don't know how long a plugin's .get_size() will take, but we also documented that it shouldn't change per connection and therefore can be cached. It's not hard to see that we have to consult the size per connection (see commit b3a43ccd for a test that purposefully exposes different sizes to separate clients), nor to search the c...

...e superblock data in the image on close you can do it in >> -your C<.finalize> method (calling the plugin's C<.pwrite> method). >> -Doing these things in the filter's C<.open> or C<.close> method is not >> -possible. >> +the plugin's C<.get_size> and C<.pread> methods via C<next_ops>). Or >> +if you need to cleanly update superblock data in the image on close >> +you can do it in your C<.finalize> method (calling the plugin's >> +C<.pwrite> method). Doing these things in the filter's...

The truncate filter tried to be careful to lock access to setting or reading the real_size variable learned from calling next_ops->get_size, in anticipation of not behaving incorrectly if the NBD protocol makes dynamic resize supported, and where the global variable could serve as a cache rather than having to always call next_ops->get_size to recompute things. However, nbdkit-plugin.pod already documents that .get_size and other o...

...we should only translate the starting offset, and let the plugin report for at least as many bytes as our range permits. Otherwise, short-changing the range causes bad behavior such as returning 0 extents, or even failing the creation of an extents tracker: $ cat script case "$1" in get_size) echo 1m;; can_extents) ;; extents) echo 0 1m;; *) exit 2 ;; esac $ nbdkit -U - --filter=offset sh script offset=64k \ --run 'qemu-io -r -f raw -c map $nbd' nbdkit: sh[1]: error: extents: plugin must return at least one extent 896 KiB (0xe0000) bytes allocated at offset 0...

While working on adding assertions to pthread_mutex_lock calls, I noticed that the truncate filter's use of mutex didn't really protect us, and isn't really necessary. Cleaning that up also spotted a couple of other potential cleanups. Eric Blake (4): filters: Drop useless .open callbacks truncate: Fix corruption when plugin changes per-connection size truncate: Test for safe

...-> unit) option; preconnect : (bool -> unit) option; + list_exports : (bool -> bool -> export list) option; + default_export : (bool -> bool -> string) option; open_connection : (bool -> 'a) option; (* required *) close : ('a -> unit) option; get_size : ('a -> int64) option; (* required *) + export_description : ('a -> string) option; can_cache : ('a -> cache_flag) option; can_extents : ('a -> bool) option; diff --git a/plugins/ocaml/NBDKit.ml b/plugins/ocaml/NBDKit.ml index 9ce3bf3e..1d014934...

...-> unit) option; preconnect : (bool -> unit) option; + list_exports : (bool -> bool -> export list) option; + default_export : (bool -> bool -> string) option; open_connection : (bool -> 'a) option; (* required *) close : ('a -> unit) option; get_size : ('a -> int64) option; (* required *) + export_description : ('a -> string) option; can_cache : ('a -> cache_flag) option; can_extents : ('a -> bool) option; diff --git a/plugins/ocaml/NBDKit.ml b/plugins/ocaml/NBDKit.ml index 9ce3bf3e..1823fc71...

...quot;embedded '%s%s' from within ext2 image: %s", + slash, fname, base); + else + asprintf (&desc, "embedded '%s%s' from within ext2 image", slash, fname); + return nbdkit_strdup_intern (desc); +} + /* Get the disk size. */ static int64_t ext2_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) @@ -412,23 +431,24 @@ ext2_flush (struct nbdkit_next_ops *next_ops, void *nxdata, */ static struct nbdkit_filter filter = { - .name = "ext2", - .longname = "nbdkit ext2 filter", - .load...

...%04x" $eflags + for f in $all_flags; do + [ $(( eflags & ${!f} )) -ne 0 ] && echo -n " $f" + done + echo +} + +fail () +{ + echo "$@" + exit 1 +} + +# no -r +# can_write=false + +do_nbdkit <<'EOF' +case "$1" in + get_size) echo 1M ;; + *) exit 2 ;; +esac +EOF + +[ $eflags -eq $(( HAS_FLAGS|READ_ONLY )) ] || + fail "expected HAS_FLAGS|READ_ONLY" + +# -r +# can_write=false + +do_nbdkit -r <<'EOF' +case "$1" in + get_size) echo 1M ;; + *) exit 2 ;; +esac +EOF + +[ $efla...

...). Or if you need to -cleanly update superblock data in the image on close you can do it in -your C<.finalize> method (calling the plugin's C<.pwrite> method). -Doing these things in the filter's C<.open> or C<.close> method is not -possible. +the plugin's C<.get_size> and C<.pread> methods via C<next_ops>). Or +if you need to cleanly update superblock data in the image on close +you can do it in your C<.finalize> method (calling the plugin's +C<.pwrite> method). Doing these things in the filter's C<.open> or +C<.cl...

..._t count, uint64_t offset, uint32_t flags, int *err); > > This has some advantages: > > - It's explicit. > > - It means we preserve the convention that -1 = error > used everywhere else in nbdkit. > > - Better for APIs that actually return something like get_size. > > - Can be both read and written. Seems like it might be reasonable enough, even though it's a bit more verbose; I can respin my series along those lines for comparison. Note that get_size() doesn't send an error over the wire to the client, so there, accessing errno in the...

...d: /tmp/nbdkitpKY00D/inline-script.sh open: status 0 nbdkit: sh[1]: debug: sh: open returned handle 0x2aa2a0ee2a0 nbdkit: sh[1]: debug: retry: open returned handle 0x2aa2a10e5c0 nbdkit: sh[1]: debug: sh: prepare readonly=0 nbdkit: sh[1]: debug: retry: prepare readonly=0 nbdkit: sh[1]: debug: retry: get_size nbdkit: sh[1]: debug: sh: get_size nbdkit: sh[1]: debug: calling: /tmp/nbdkitpKY00D/inline-script.sh get_size "" nbdkit: sh[1]: debug: completed: /tmp/nbdkitpKY00D/inline-script.sh get_size: status 0 nbdkit: sh[1]: debug: retry: can_write nbdkit: sh[1]: debug: sh: can_write nbdkit: sh[1]...

...e.gz > nbdkit curl --filter=gzip https://example.com/disk.gz > > Because of the nature of the gzip format which is not blocked based > and thus not seekable, this filter caches the whole uncompressed file > in a hidden temporary file. This is required in order to implement > .get_size. See this link for a more detailed explanation: > https://stackoverflow.com/a/9213826 > > This commit deprecates nbdkit-gzip-plugin and suggests removal in > nbdkit 1.26. > --- Nice - this one seems like a fairly straight-forward conversion. > +++ b/filters/tar/nbdkit-tar-fi...

On 12/14/18 4:16 PM, Richard W.M. Jones wrote: > This implements something like a readonly 1MB disk reading as zeroes: > > nbdkit sh script=- <<'EOF' > case "$1" in > get_size) echo 1M ;; > pread) dd if=/dev/zero count=$3 iflag=count_bytes ;; > *) exit 2 ;; > esac > EOF > > Use of "-" is analogous to reading passwords from stdin. > --- > @@ -26,6 +30,19 @@ like this: > You may have to add further C<key=value> a...

...ying size when there was no range, and even when there is a range, we were not careful about integer overflow (offset=5E range=5E happily claims to export a 5E image; but all bets are off if you later try to access beyond the real underlying size). And in several cases, such as if the plugin's get_size fails but range was not provided, we are not returning -1 for failure. Fixes: 3db69f56 Signed-off-by: Eric Blake <eblake@redhat.com> --- The first post-1.14 bugfix. Too bad I didn't spot it earlier today. filters/offset/offset.c | 25 ++++++++++++------------- 1 file changed, 12 ins...

...ne-script.sh open: status 0 > nbdkit: sh[1]: debug: sh: open returned handle 0x2aa2a0ee2a0 > nbdkit: sh[1]: debug: retry: open returned handle 0x2aa2a10e5c0 > nbdkit: sh[1]: debug: sh: prepare readonly=0 > nbdkit: sh[1]: debug: retry: prepare readonly=0 > nbdkit: sh[1]: debug: retry: get_size > nbdkit: sh[1]: debug: sh: get_size > nbdkit: sh[1]: debug: calling: /tmp/nbdkitpKY00D/inline-script.sh get_size "" > nbdkit: sh[1]: debug: completed: /tmp/nbdkitpKY00D/inline-script.sh get_size: status 0 > nbdkit: sh[1]: debug: retry: can_write > nbdkit: sh[1]: debug: sh...

...image: %s", > + slash, fname, base); > + else > + asprintf (&desc, "embedded '%s%s' from within ext2 image", slash, fname); > + return nbdkit_strdup_intern (desc); > +} > + > /* Get the disk size. */ > static int64_t > ext2_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) > @@ -412,23 +431,24 @@ ext2_flush (struct nbdkit_next_ops *next_ops, void *nxdata, > */ > > static struct nbdkit_filter filter = { > - .name = "ext2", > - .longname = "nbdki...

...nbdkit file --filter=gzip file.gz nbdkit curl --filter=gzip https://example.com/disk.gz Because of the nature of the gzip format which is not blocked based and thus not seekable, this filter caches the whole uncompressed file in a hidden temporary file. This is required in order to implement .get_size. See this link for a more detailed explanation: https://stackoverflow.com/a/9213826 This commit deprecates nbdkit-gzip-plugin and suggests removal in nbdkit 1.26. --- filters/gzip/nbdkit-gzip-filter.pod | 85 +++++++ filters/tar/nbdkit-tar-filter.pod | 7 +- plugins/gzip/nbdkit-gzip-plugin...

This picks up python and ocaml. Some of our languages are lacking a number of bindings (for example, lua and perl lack .extents, so I didn't have anything to copy from), and I felt less comfortable with golang and rust. But for python and ocaml, I was able to test a working implementation. Eric Blake (2): python: Implement .list_exports and friends ocaml: Implement .list_exports and