similar to: [nbdkit PATCH] data, memory: Optimize .zero > PAGE_SIZE

On Wed, Aug 14, 2019 at 09:10:15PM -0500, Eric Blake wrote: > When sparse_array_zero() is used for a range larger than a page, > there's no need to waste time in memset() or is_zero() - we already > know the page will be free()d. > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- > > Here's a fun one :) > > common/sparse/sparse.c | 7 +++++--

The NBD protocol is adding an extension to let servers advertise initialization state to the client: whether the image contains holes, and whether it is known to read as all zeroes. For memory-based plugins, it is fairly easy to advertise several cases: data and memory are usually sparse and detecting zero is easy (requires new functions to the sparse_array common code), although since the sparse

Should we use attribute((nonnull)) at all? There's a very interesting history of this in libvirt -- try looking at commit eefb881 plus the commits referencing eefb881 -- but it does seem to work for me using recent GCC and Clang. I only did a few functions because annotating them gets old quickly... Rich.

v1 was here: https://www.redhat.com/archives/libguestfs/2019-January/msg00008.html In v2 I have provided two patches: The first patch extends attribute((nonnull)) to most internal functions, but not to the external API. The second patch uses a macro so that attribute((format)) is only used in the public API on GCC or Clang. At least in theory these headers could be used by a C compiler which

These plugins are both based on the same sparse array structure which supports a simple implementation of extents. --- common/sparse/sparse.h | 7 +- common/sparse/sparse.c | 37 ++++++++++- plugins/data/data.c | 16 ++++- plugins/memory/memory.c | 16 ++++- README | 2 + tests/Makefile.am | 2 + tests/test-data-extents.sh | 131

On 3/28/19 11:18 AM, Richard W.M. Jones wrote: > These plugins are both based on the same sparse array structure which > supports a simple implementation of extents. > --- > common/sparse/sparse.h | 7 +- > common/sparse/sparse.c | 37 ++++++++++- > plugins/data/data.c | 16 ++++- > plugins/memory/memory.c | 16 ++++- > README |

I will be following up to this email with four separate threads each addressed to the appropriate single list, with proposed changes to: - the NBD protocol - qemu: both server and client - libnbd: client - nbdkit: server The feature in question adds a new optional NBD_INFO_ packet to the NBD_OPT_GO portion of handshake, adding up to 16 bits of information that the server can advertise to the

Annotate some function parameters with attribute((nonnull)). Only do this for internal headers where we are sure that we will be using sufficiently recent GCC or Clang. For the public header files (ie. include/nbdkit-*.h) it may be that people building out of tree plugins are using old GCC which had problems, or even other compilers that don't support this extension at all. Libvirt has an

These plugins are both based on the same sparse array structure which supports a simple implementation of extents. --- common/sparse/sparse.h | 7 ++++++- common/sparse/sparse.c | 29 ++++++++++++++++++++++++++++- plugins/data/data.c | 16 +++++++++++++++- plugins/memory/memory.c | 16 +++++++++++++++- 4 files changed, 64 insertions(+), 4 deletions(-) diff --git a/common/sparse/sparse.h

This is largely a simple refactoring, but it combines another definition of this function from virt-builder which had a slightly different prototype. --- builder/pxzcat-c.c | 20 +------------------- daemon/daemon.h | 18 ------------------ lib/guestfs-internal-all.h | 18 ++++++++++++++++++ 3 files changed, 19 insertions(+), 37 deletions(-) diff --git a/builder/pxzcat-c.c

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11]

On 2/17/20 9:13 AM, Max Reitz wrote: > Hi, > > It’s my understanding that without some is_zero infrastructure for QEMU, > it’s impossible to implement this flag in qemu’s NBD server. You're right that we may need some more infrastructure before being able to decide when to report this bit in all cases. But for raw files, that infrastructure already exists: does block_status at

On Wed, Apr 19, 2017 at 09:35:16AM -0500, Eric Blake wrote: > On 04/19/2017 09:17 AM, Richard W.M. Jones wrote: > > This is largely a simple refactoring, but it combines another > > definition of this function from virt-builder which had a slightly > > different prototype. > > --- > > > +/* Return true iff the buffer is all zero bytes. > > + * > > +

Fast testing whether the AIO buffer (or regions within it) contain all zeroes, which allows Python code to quickly do sparsification when copying. This includes the iszero.h header from nbdkit which is distributed under a compatible license. --- common/include/Makefile.am | 5 +-- common/include/iszero.h | 63 +++++++++++++++++++++++++++++++++++++ generator/generator | 17

v1 -> v2: The first patch is the same (the pure refactoring), but in the second patch I implement Eric Blake's suggested version. Rich.

Now that cleanup.h is in common code, we can use it in our plugins where it makes sense. Many uses of pthread_mutex_unlock() are not function-wide, and over small enough snippets of code as to be easier to read when left as-is; but when the scope is indeed function-wide or across multiple exit paths, it is nicer to use the macro for automatic unlock. Signed-off-by: Eric Blake

On Tue, Apr 23, 2019 at 02:46:10PM -0500, Eric Blake wrote: > On 3/28/19 11:18 AM, Richard W.M. Jones wrote: > > These plugins are both based on the same sparse array structure which > > supports a simple implementation of extents. > > --- > > common/sparse/sparse.h | 7 +- > > common/sparse/sparse.c | 37 ++++++++++- > > plugins/data/data.c

On 08/01/2018 06:10 AM, Richard W.M. Jones wrote: > This can truncate, extend, or round up/down to a multiple. > --- > common-rules.mk | 3 +- > configure.ac | 1 + > filters/offset/nbdkit-offset-filter.pod | 7 +- > filters/partition/nbdkit-partition-filter.pod | 1 + >

Just call close on the specified fd, if valid. --- daemon/daemon.h | 3 +++ daemon/guestfsd.c | 9 +++++++++ 2 files changed, 12 insertions(+) diff --git a/daemon/daemon.h b/daemon/daemon.h index fb74e91..0caad45 100644 --- a/daemon/daemon.h +++ b/daemon/daemon.h @@ -173,6 +173,7 @@ asprintf_nowarn (char **strp, const char *fmt, ...) extern void cleanup_free (void *ptr); extern void

This allows you to append one file to another: copy-file-to-file /input.txt /output.txt append:true will append the contents of /input.txt to /output.txt. --- daemon/copy.c | 38 +++++++++++++++++++++++++++++++------- generator/actions.ml | 29 +++++++++++++++++++++-------- 2 files changed, 52 insertions(+), 15 deletions(-) diff --git a/daemon/copy.c b/daemon/copy.c index