similar to: [PATCH nbdkit 0/3] msys2 support for review

From: Yifan Gu <gyf304@gmail.com> This patch adds in indirection for API functions, as PE DLL loader does not resolve undefined symbols. This patch only includes header changes. --- include/nbdkit-common.h | 133 +++++++++++++++++++++++++++++++++++++++- include/nbdkit-compat.h | 97 +++++++++++++++++++++++++++++ include/nbdkit-filter.h | 39 ++++++++++++ include/nbdkit-plugin.h | 27

From: Yifan Gu <gyf304@gmail.com> Windows PE DLL needs function indirection, this patch injects API functions for plugins / filters if on Windows. Note that nbdkit_peer_name is explicitly disabled due to potential ABI incompatibilities. --- server/main.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/server/main.c b/server/main.c

Eric: Yifan Gu has posted a few patches for mingw support. My comments below. https://github.com/gyf304/nbdkit/commit/a37c4ca6546dfc4e96e305af97b62e5a9d6174ca * I think the SHARED_LDFLAGS idea is good. I pushed a slightly different take on the idea here: https://github.com/libguestfs/nbdkit/commit/1d634009ab8e43592065ec469df6312400525cc8 It's slightly different from what Yifan

On 3/24/20 3:12 PM, Eric Blake wrote: >> (For non-mingw platforms) this breaks the source API promises rather >> seriously, so if I understand your proposal correctly I don't think >> this is a good idea.  It's possibly something we can consider for >> internal plugins, or for the V3 API. > > How does it break API to request that someone link against a

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

Ultimately, both the blocksize and swab filters want to return aligned extents to the client. I'm posting this as a snapshot of my work in progress on how I plan to get there (it's not quite working yet, but I'm done for today and wanted to at least document my ideas). I might also add a convenience function for nbdkit_extents_offset, since we have a number of filters that repeat the

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.

On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- I see you've already made a couple of tweaks to your block-status branch since posting this (skipping 0-legnth add_extent, and coalescing consecutive add_extents with the same type), but here's some more review:

This filter caches the last result of the extents() call and offers a nice speed-up for clients that only support req_on=1 in combination with plugins like vddk, which has no overhead for returning information for multiple extents in one call, but that call is very time-consuming. Quick test showed that on a fast connection and a sparsely allocated 16G disk with a OS installed `qemu-img map` runs

I pushed this one already. It should go into the stable-1.12 branch too. Rich.

This filter caches the last result of the extents() call and offers a nice speed-up for clients that only support req_one=1 in combination with plugins like vddk, which has no overhead for returning information for multiple extents in one call, but that call is very time-consuming. Quick test showed that on a fast connection and a sparsely allocated 16G disk with a OS installed `qemu-img map`

On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > Allows you to safely use nbdkit-offset-filter on top of a plugin > supporting extents. > --- > filters/offset/offset.c | 35 +++++++++++++++++++++++++++++++++++ > 1 file changed, 35 insertions(+) If we don't change our minds on the interface in patch 1, then this looks correct. > +/* Extents. */ > +static int >

This fix isn't exhaustive but it fixes some obvious problems in the filters. Rich.

There's more that can be done (in particular, use of CLEANUP_FREE), but this is enough to at least see if I'm on the right track. I couldn't figure out an obvious difference between common/include and common/utils, but it looks like the former is for things that are inlineable via .h only, while the latter is when you need to link in a convenience library, so this landed in the

When querying the extents of the underlying plugin, we should only translate the starting offset, and let the plugin report for at least as many bytes as our range permits. Otherwise, short-changing the range causes bad behavior such as returning 0 extents, or even failing the creation of an extents tracker: $ cat script case "$1" in get_size) echo 1m;; can_extents) ;;

On 3/28/19 11:18 AM, Richard W.M. Jones wrote: > This new pair of callbacks allows plugins to describe which extents in > the virtual disk are allocated, holes or zeroes. > --- > +++ b/server/extents.c > @@ -0,0 +1,210 @@ > +/* nbdkit > + * Copyright (C) 2019 Red Hat Inc. > + * All rights reserved. We started work on this phrase but it got waylaid by extents work; maybe

When offset + length == exts->end the code which is shortening the extents is subtracting 0 from length. And it is not optimized out. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- server/extents.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/extents.c b/server/extents.c index d3d1a15ab97c..c422491601f0 100644 --- a/server/extents.c +++

A devious plugin can cause an 8-fold increase in the reply size in relation to the request size, by alternating status every byte of the request. In the worst case, this can cause the client to reject our response and kill the connection because the response is too large. What's more, it consumes a lot of memory to track that many extents. Let's put an upper bound on the maximum number of

I'm not observing any noticeable performance differences, but I'm liking the diffstat. I can't push this patch until we release a new libnbd version with the _notify API addition, but am posting it now for playing with things. Eric Blake (2): nbd: Move transaction info from heap to stack nbd: Use nbdkit aio_*_notify variants plugins/nbd/nbd.c | 217