similar to: [nbdkit PATCH 0/3] .list_exports in nbd plugin

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

We're still seeing sporadic failures of 'nbdkit nbd tls=', and I'm still trying to come up with a root cause fix (it may involve smarter use of gnutls_bye() in libnbd). In the meantime, here's what we know: when the hang/failure happens, the 'nbdkit nbd tls=' client process is stuck in a poll() waiting to see EOF from the server, while the 'nbdkit example1'

With new-enough libnbd and our new dynamic-export mode, we can grab the export list from the server for replay to the client. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/nbd/nbdkit-nbd-plugin.pod | 7 ++ tests/Makefile.am | 2 + plugins/nbd/nbd.c | 53 ++++++++++ tests/test-nbd-dynamic-content.sh | 2 +- tests/test-nbd-dynamic-list.sh

libnbd-0.1.4-1 is now available in Fedora 29/30 updates testing. Diffs since v2 - rebase to master, bump from libnbd 0.1.2 to 0.1.3+, add tests to TLS usage which flushed out the need to turn relative pathnames into absolute, doc tweaks Now that the testsuite covers TLS and libnbd has been fixed to provide the things I found lacking when developing v2, I'm leaning towards pushing this on

Where qemu-img 4.0 used to say 'virtual size: 100M', the 4.1 release now says 'virtual size: 100 MiB'. Similarly, '5.0G' turned into '5 GiB'. But rather than worry about potential future changes to the human-readable output, we can just use --output=json (at which point we no longer even have to force qemu-img to the C locale). It might be slightly more robust to

This patch is a straightforward refactoring of libnbd?s nbdcopy program, and not very interesting. However I have plans for nbdcopy (see full todo below). I would like to use this utility for virt-v2v as a replacement for ?qemu-img convert?. qemu-img has caused us a series of problems: - change in zeroing behaviour caused a big performance regression - qemu-img reads extents up-front which

In v2: - use int instead of bool in the public header - split the tests from the code - don't overload test-layers; instead, add new tests - add a missing fflush exposed by the new tests - other minor cleanups Eric Blake (3): server: Add nbdkit_stdio_safe server: Sanitize stdin/out before running plugin code server: More tests of stdin/out handling docs/nbdkit-plugin.pod |

It may be useful to test whether the client's use of NBD_CMD_FLAG_NO_HOLE makes a difference; do this by adding a mode to --filter=nozero to force a non-trimming zero write. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/nozero/nbdkit-nozero-filter.pod | 19 +++++++---- filters/nozero/nozero.c | 45 ++++++++++++++++++++----- tests/test-nozero.sh

This causes a compile error on Solaris. Fix: Index: socket.c =================================================================== RCS file: /cvsroot/rsync/socket.c,v retrieving revision 1.73 diff -u -r1.73 socket.c --- socket.c 25 Jan 2002 02:13:05 -0000 1.73 +++ socket.c 11 Feb 2002 20:04:33 -0000 @@ -590,7 +590,7 @@ if ((listener = socket(PF_INET, SOCK_STREAM, 0)) == -1) goto failed;

Version 2.5.6 of rsync fail to compile on Tru64 Unix v5.1 with this error message: cc-wrapper -I. -I. -g -DHAVE_CONFIG_H -c socket.c -o socket.o cc: Error: socket.c, line 619: In this statement, "sin_len" is not a member of "sock2". (needmember) sock2.sin_len = sizeof(sock2); --------^ make: *** [socket.o] Error 1 error: compile failed. The problem

The last versions of rsync fail to compile on Tru64 Unix (alpha), because of a typo in configure.in. The problem is that the code in configure check for sockaddr.sa_len, while the code uses sockaddr.sin_len. This patch fixes the problem. Please include it in the next version of rsync. diff -ur src-2.6.0/configure.in src-2.6.0-local/configure.in --- src-2.6.0-local/configure.in

This is a rough-and-ready implementation of nbdinfo --map option, to display the allocation and other metadata of NBD exports. I only tested it lightly against nbdkit. It needs at least some tests. Command like these should work: nbdinfo --map nbd://localhost nbdinfo --map=qemu:dirty-bitmap nbd://localhost Rich.

Fixes bugs with JSON output in the previous patch, and adds some tests. Rich.

On Thu, Mar 26, 2020 at 02:34:41PM -0500, Eric Blake wrote: > We're still seeing sporadic failures of 'nbdkit nbd tls=', and I'm > still trying to come up with a root cause fix (it may involve smarter > use of gnutls_bye() in libnbd). In the meantime, here's what we know: > when the hang/failure happens, the 'nbdkit nbd tls=' client process is > stuck in

This rounds up the remaining bugs that I originally identified in: https://www.redhat.com/archives/libguestfs/2020-July/msg00153.html Eric Blake (4): api: Permit export list APIs when Connected info: Support --list with serializing servers info: Fix --json output when list size != 1 info: Permit --size --json generator/API.ml | 6 +++--- info/info-list-json.sh | 9 +++++++++

A rather trivial decoding; we may enhance it further if qemu extends things to give an integer depth alongside its tri-state encoding. --- I'll wait to push this to libnbd until the counterpart qemu patches land upstream, although it looks like I've got positive review. info/nbdinfo.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/info/nbdinfo.c b/info/nbdinfo.c index

If the server fails, nbdinfo can write partial output before the error message (albeit on different channels). Here is an example: $ nbdkit eval open='echo EIO fail >&2; exit 1' --run 'nbdinfo --json "$uri"' { "protocol": "newstyle-fixed", "TLS": false, nbdkit: eval[1]: error: /tmp/nbdkitii3pZW/open: fail nbdkit: eval[1]:

Otherwise, we get a failure due to Numerical result out of range. And for safety's sake, we are best capping our request to an aligned value, if the server insists on minimum alignment. Fixes: f3fd935c --- info/nbdinfo.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/info/nbdinfo.c b/info/nbdinfo.c index 1afdf98..2b22f51 100644 --- a/info/nbdinfo.c +++

On 9/29/20 6:23 AM, Richard W.M. Jones wrote: > This is not related to the current patch, but generally for libnbd: > > (1) Should we be testing interop separately for qemu-storage-daemon > (qsd), or is qsd basically qemu-nbd in a new wrapper so it's not worth > doing it? > > (2) I found a bug in the new nbdinfo behaviour: > > $ nbdkit -fv file dir=/scratch >

Necessary when writing a client that wants to avoid unnecessary EINVAL errors from sending unaligned requests. At some point, we may want to add synchronous convenience API wrappers that do request splitting or read-modify-write to obey server constraints while still appearing to the library client as accepting any possible request. But such a wrapper should only be synchronous and not copied to