search for: nbdkit_sourc

Patch 1 was previously posted here: https://www.redhat.com/archives/libguestfs/2020-June/msg00086.html to handle this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1848862 I was able to observe this bug and for me at least disabling readahead seems to cure it. Patches 2 and 3 are simplifications, removing a now-undocumented feature of virt-v2v-copy-to-local and thus simplifying greatly the

Part 2 fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1842440 Actually this fix on its own should be sufficient, but probably we want the nbdkit fixes too. Note this uses actual OCaml 4.05 features! ("let open" and the Unix.tcgetattr functions). I checked that both features are available on RHEL 7's OCaml. Rich.

I'm still testing this fix, so let's hold off the review for the moment. Also it may be better to specifically identify problematic servers rather than disabling this for every curl source. eg. I suspect that the problem is the Java server used by VCenter, so we might think about only disabling readahead for that single case. Rich.

...racter in the nbdkit-rate-filter command line. For example: virt-v2v .. --bandwidth 200M would invoke: nbdkit .. --filter rate rate==200M which causes a parse error. The --bandwidth-file option does not invoke a parse error but does not work, for similar reasons. Thanks: Ming Xie --- v2v/nbdkit_sources.ml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/v2v/nbdkit_sources.ml b/v2v/nbdkit_sources.ml index 979c37734d..402dfd0e55 100644 --- a/v2v/nbdkit_sources.ml +++ b/v2v/nbdkit_sources.ml @@ -118,11 +118,11 @@ let common_create ?bandwidth ?extra_debug ?extra_env plugin_...

...ase the maximum request size by changing the configuration on the VMware server, but it's not sufficient for VDDK with the readahead filter. As readahead is only an optimization, the simplest solution is to disable this filter when we're using nbdkit-vddk-plugin. Thanks: Ming Xie --- v2v/nbdkit_sources.ml | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/v2v/nbdkit_sources.ml b/v2v/nbdkit_sources.ml index 979c3773..e97583a5 100644 --- a/v2v/nbdkit_sources.ml +++ b/v2v/nbdkit_sources.ml @@ -97,9 +97,13 @@ let common_create ?bandwidth ?extra_debug ?extra_env plugin_name...

...ere VMware VCenter server's Tomcat HTTPS server stops responding to requests (or rather, responds only with 503 errors). The server later recovers and in fact because of the retry filter the conversion usually succeeds, but I found that we can avoid the problem by disabling readahead. --- v2v/nbdkit_sources.ml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/v2v/nbdkit_sources.ml b/v2v/nbdkit_sources.ml index f5e919116..7c177e358 100644 --- a/v2v/nbdkit_sources.ml +++ b/v2v/nbdkit_sources.ml @@ -99,12 +99,12 @@ let common_create ?bandwidth ?extra_debug ?extra_env password...

Trivial fix. We really need a regression test for all v2v inpus related to nbdkit. There is actually nothing at all at the moment. Of course if it was easy to test inputs over the network from a ‘make check’ rule then we'd be doing it already. I thought about adding something like a ‘-it file’ option which would use nbdkit-file-plugin for test only. However I'm rather cautious about

This is the simplest solution to this problem. There are two other possible fixes I considered: Increase the documented limit (see http://libguestfs.org/virt-v2v-input-vmware.1.html#vddk:-esxi-nfc-service-memory-limits). However at the moment we know the current limit works through extensive testing (without readahead), plus I have no idea nor any way to test if larger limits are supported by

...- 6 files changed, 80 insertions(+), 24 deletions(-) diff --git a/server/Makefile.am b/server/Makefile.am index 58b22341..8cfa0115 100644 --- a/server/Makefile.am +++ b/server/Makefile.am @@ -34,8 +34,12 @@ include $(top_srcdir)/common-rules.mk EXTRA_DIST = nbdkit.syms sbin_PROGRAMS = nbdkit +nbdkit_SOURCES = nbdkit.c +nbdkit_CFLAGS = $(WARNINGS_CFLAGS) +nbdkit_LDADD = libnbdkit.la -nbdkit_SOURCES = \ +lib_LTLIBRARIES = libnbdkit.la +libnbdkit_la_SOURCES = \ backend.c \ background.c \ captive.c \ @@ -70,10 +74,10 @@ nbdkit_SOURCES = \ $(top_srcdir)/include/nbdkit-filter.h \ $(NULL) if E...

...Tomcat HTTPS server stops responding to > requests (or rather, responds only with 503 errors). The server later > recovers and in fact because of the retry filter the conversion > usually succeeds, but I found that we can avoid the problem by > disabling readahead. > --- > v2v/nbdkit_sources.ml | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > Is it just a matter of readahead requesting too much data at once? Is it time to start revisiting the idea of moving the core of the 'blocksize' filter into the server itself, in order to make it easier for...

...har *str); diff --git a/server/Makefile.am b/server/Makefile.am index 9351fefc..b6728511 100644 --- a/server/Makefile.am +++ b/server/Makefile.am @@ -33,6 +33,7 @@ include $(top_srcdir)/common-rules.mk EXTRA_DIST = nbdkit.syms +pkglib_LTLIBRARIES = nbdkit-shim-dlopen.la sbin_PROGRAMS = nbdkit nbdkit_SOURCES = \ @@ -71,6 +72,17 @@ if ENABLE_LIBFUZZER nbdkit_SOURCES += fuzzer.c endif +nbdkit_shim_dlopen_la_SOURCES = shim.c +nbdkit_shim_dlopen_la_CPPFLAGS = \ + -I$(top_srcdir)/common/utils \ + $(NULL) +nbdkit_shim_dlopen_la_CFLAGS = $(WARNINGS_CFLAGS) +nbdkit_shim_dlopen_la_LIBADD = $(DL_LIBS) +nbdk...

The patches following do indeed allow you to compile nbdkit.exe, but it does not actually work yet. I'm posting this experimental series more as a work in progress and to get feedback. Note this does not require Windows itself to build or test. You can cross-compile it using mingw64-* packages on Fedora or Debian, and test it [spoiler alert: it fails] using Wine. Rich.

...= \ - utils.c \ + cleanup.c \ + cleanup.h \ + utils.c \ utils.h libutils_la_CPPFLAGS = \ -I$(top_srcdir)/include diff --git a/server/Makefile.am b/server/Makefile.am index 9e13c03..8aa8d3a 100644 --- a/server/Makefile.am +++ b/server/Makefile.am @@ -38,7 +38,6 @@ sbin_PROGRAMS = nbdkit nbdkit_SOURCES = \ background.c \ captive.c \ - cleanup.c \ connections.c \ crypto.c \ debug.c \ @@ -124,9 +123,11 @@ check_PROGRAMS = test-utils test_utils_SOURCES = \ test-utils.c \ utils.c \ - cleanup.c \ extents.c test_utils_CPPFLAGS = \ -I$(top_srcdir)/include \ - -I$(top_srcdir)/common...

...You probably want to look at what we do in virt-v2v, where we have almost exactly the same scenario and we use nbdkit to proxy the connection: https://github.com/libguestfs/virt-v2v/blob/master/v2v/nbdkit.ml https://github.com/libguestfs/virt-v2v/blob/f8e4e8643f105bc2abeb6b80b17a0ded3b636b7c/v2v/nbdkit_sources.ml#L276 Also at: http://libguestfs.org/nbdkit.1.html http://libguestfs.org/nbdkit-captive.1.html http://libguestfs.org/nbdkit-curl-plugin.1.html http://libguestfs.org/nbdkit-cache-filter.1.html Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my pro...

...24 ----------------- server/vfprintf.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 74 insertions(+), 30 deletions(-) diff --git a/server/Makefile.am b/server/Makefile.am index 9351fefc..4c789934 100644 --- a/server/Makefile.am +++ b/server/Makefile.am @@ -64,6 +64,7 @@ nbdkit_SOURCES = \ sockets.c \ threadlocal.c \ usergroup.c \ + vfprintf.c \ $(top_srcdir)/include/nbdkit-plugin.h \ $(top_srcdir)/include/nbdkit-filter.h \ $(NULL) diff --git a/server/internal.h b/server/internal.h index e39db8a8..e5c7f514 100644 --- a/server/internal.h +++ b/server/internal.h @@ -2...

...re +++ b/.gitignore @@ -47,6 +47,7 @@ Makefile.in /podwrapper.pl /src/nbdkit /src/nbdkit.pc +/src/protostrings.c /src/synopsis.c /src/test-utils /stamp-h1 diff --git a/src/Makefile.am b/src/Makefile.am index 65f9498..1563d74 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -49,6 +49,7 @@ nbdkit_SOURCES = \ options.h \ plugins.c \ protocol.h \ + protostrings.c \ sockets.c \ threadlocal.c \ utils.c \ @@ -77,11 +78,23 @@ nbdkit_LDFLAGS = \ $(PTHREAD_LIBS) \ $(DL_LDFLAGS) +# protostrings.c is generated from the protocol.h header file where it +# is used to map NBD protocol flags t...

With some crufty sed scripts we can generate functions that map from NBD protocol flags (eg. NBD_CMD_READ) to strings ("NBD_CMD_READ"). This works on GNU sed and with FreeBSD, also with GNU sed's --posix option, so I guess the sed code is POSIX-compatible. Rich.

...++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -40,6 +40,7 @@ nbdkit_SOURCES = \ crypto.c \ errors.c \ internal.h \ + locks.c \ main.c \ plugins.c \ protocol.h \ diff --git a/src/connections.c b/src/connections.c index 111a810..74bb8e4 100644 --- a/src/connections.c +++ b/src/connections.c @@ -211,7 +211,7 @@ _handle_single_connection (int sockin, int sockout)...

...++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -40,6 +40,7 @@ nbdkit_SOURCES = \ crypto.c \ errors.c \ internal.h \ + locks.c \ main.c \ plugins.c \ protocol.h \ diff --git a/src/connections.c b/src/connections.c index 111a810..74bb8e4 100644 --- a/src/connections.c +++ b/src/connections.c @@ -211,7 +211,7 @@ _handle_single_connection (int sockin, int sockout)...

v1 was here: https://www.redhat.com/archives/libguestfs/2018-November/msg00147.html v2: - Use stdbool for booleans. - Use __attribute__((format(printf))). - Don't abort on invalid options, exit with failure instead. - Preserve long/short option choices in the output. - Add '=' in long option output, ie. always use --longopt=arg. - Add "--" parameter