similar to: [PATCH nbdkit] log: Decode the extent type in output.

On 4/1/19 6:53 AM, Richard W.M. Jones wrote: > Instead of printing something like ‘type=0’ or ‘type=3’, this changes > the output to show the hole and zero flags separately. For example: > > $ ./nbdkit -U - --filter=log sh - logfile=/dev/stdout \ > --run 'qemu-img map $nbd' <<'EOF' > case "$1" in > get_size) echo 1M ;; >

This filter caches the last result of the extents() call and offers a nice speed-up for clients that only support req_one=1 in combination with plugins like vddk, which has no overhead for returning information for multiple extents in one call, but that call is very time-consuming. Quick test showed that on a fast connection and a sparsely allocated 16G disk with a OS installed `qemu-img map`

On 12/14/18 4:16 PM, Richard W.M. Jones wrote: > This implements something like a readonly 1MB disk reading as zeroes: > > nbdkit sh script=- <<'EOF' > case "$1" in > get_size) echo 1M ;; > pread) dd if=/dev/zero count=$3 iflag=count_bytes ;; > *) exit 2 ;; > esac > EOF > > Use of "-" is analogous to reading

While iflag=count_bytes combined with bs > 1 allows for more efficient operation, it is a feature of GNU dd, and not present on other implementations such as BSD. Sticking to just POSIX features makes things more portable. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-loop.pod | 2 +- docs/nbdkit.pod | 4 ++--

A devious plugin can cause an 8-fold increase in the reply size in relation to the request size, by alternating status every byte of the request. In the worst case, this can cause the client to reject our response and kill the connection because the response is too large. What's more, it consumes a lot of memory to track that many extents. Let's put an upper bound on the maximum number of

On Fri, Mar 08, 2019 at 10:50:35AM -0600, Eric Blake wrote: > > int extents (void *handle, uint32_t count, uint64_t offset, > > uint32_t flags /* always 0? */, > > Unless we support the REQ_ONE flag (where the client specifically wants > only one extent returned). ... > > nbdkit would need to do some massaging on this to get it into the > > right

On Mon, Mar 11, 2019 at 02:53:55PM -0500, Eric Blake wrote: >On 3/11/19 2:44 PM, Richard W.M. Jones wrote: >> On Fri, Mar 08, 2019 at 10:50:35AM -0600, Eric Blake wrote: >>>> int extents (void *handle, uint32_t count, uint64_t offset, >>>> uint32_t flags /* always 0? */, >>> >>> Unless we support the REQ_ONE flag (where the client

On Wed, Aug 02, 2023 at 08:50:21PM -0500, Eric Blake wrote: > Previously, we had not been doing any validation of server extent > responses, which means a client query at an offset near the end of the > export can result in a buggy server sending a response longer than the > export length and potentially confusing the client. The NBD spec also > says that an extent length should be

On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > This uses a new API VixDiskLib_QueryAllocatedBlocks provided in > VDDK >= 6.7. > > Thanks: Martin Kletzander. > --- > plugins/vddk/vddk-structs.h | 15 +++- > plugins/vddk/vddk.c | 138 ++++++++++++++++++++++++++++++++++++ > 2 files changed, 152 insertions(+), 1 deletion(-) > > +static int >

I've posted a couple of patches towards the ultimate goal of implementing NBD_CMD_BLOCK_STATUS / base:allocation in nbdkit. Before I can do the final patch I think we need to discuss how this would be exposed to plugins since at the end of the day they need to implement the feature. Background reading: - preparatory patches:

On Mon, Mar 25, 2019 at 05:34:25PM +0000, Richard W.M. Jones wrote: > On Sat, Mar 23, 2019 at 02:29:54PM -0500, Eric Blake wrote: > > On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > ... > > > + for (i = 0; i < block_list->numBlocks; ++i) { > > > + uint64_t offset, length; > > > + > > > + offset = block_list->blocks[i].offset *

On 9/12/19 4:01 PM, Richard W.M. Jones wrote: > --- > tests/Makefile.am | 3 ++ > tests/test-export-name.sh | 86 +++++++++++++++++++++++++++++++++++++++ > 2 files changed, 89 insertions(+) > +# Create a sh plugin which echos the export name back in the device. echoes? One of those weird words with dual acceptable spellings, where it depends on who you ask for which

Typical output: 2019-03-26 18:22:17.287434 connection=2 Extents id=2 offset=0x0 count=0x40000000 req_one=1 ... 2019-03-26 18:22:17.290040 connection=2 ...Extents id=2 extents=[{ offset=0x0, length=0x250000, type=0 }, { offset=0x250000, length=0x7db0000, type=3 }, { offset=0x8000000, length=0x8000, type=0 }, { offset=0x8008000, length=0x38000, type=3 }, { offset=0x8040000, length=0x208000, type=0

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

Typical output: 2019-03-26 18:22:17.287434 connection=2 Extents id=2 offset=0x0 count=0x40000000 req_one=1 ... 2019-03-26 18:22:17.290040 connection=2 ...Extents id=2 extents=[{ offset=0x0, length=0x250000, type=0 }, { offset=0x250000, length=0x7db0000, type=3 }, { offset=0x8000000, length=0x8000, type=0 }, { offset=0x8008000, length=0x38000, type=3 }, { offset=0x8040000, length=0x208000, type=0

On 4/23/19 4:49 PM, Richard W.M. Jones wrote: >> >> ...to here, after the final nbdkit_add_extent, so that we can return a >> larger extent than the client's request. I remember when I originally >> asked, you declined due to odd interactions with REQ_ONE semantics, but >> since then, we changed how add_extent() works. Does it work now to defer >> the

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.

The previous incomplete patch was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00049.html based on earlier discussion here: https://www.redhat.com/archives/libguestfs/2019-September/msg00047.html In v2: - The previous patch was incomplete. This version completes it by adding tests and extending nbdkit-sh-plugin. - nbdkit_export_name now returns NULL for error,

On 3/19/19 11:34 AM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- > + void nbdkit_extents_free (struct nbdkit_extents_map *); > + > +Frees an existing extents map. Is this like free() where it is safe to call on NULL? > + > +=head3 Iterating over nbdkit_extents_map