search for: sparse_array

...ervers advertise initialization state to the client: whether the image contains holes, and whether it is known to read as all zeroes. For memory-based plugins, it is fairly easy to advertise several cases: data and memory are usually sparse and detecting zero is easy (requires new functions to the sparse_array common code), although since the sparse array is reused between consecutive clients, a later client might get different answers than the first client. The null and full plugins are obviously zero. The zero plugin doesn't return any data, so it doesn't need changes. The info plugin is nev...

Should we use attribute((nonnull)) at all? There's a very interesting history of this in libvirt -- try looking at commit eefb881 plus the commits referencing eefb881 -- but it does seem to work for me using recent GCC and Clang. I only did a few functions because annotating them gets old quickly... Rich.

...44 --- a/common/sparse/sparse.h +++ b/common/sparse/sparse.h @@ -1,5 +1,5 @@ /* nbdkit - * Copyright (C) 2017-2018 Red Hat Inc. + * Copyright (C) 2017-2019 Red Hat Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -87,4 +87,9 @@ extern void sparse_array_zero (struct sparse_array *sa, uint32_t count, uint64_t offset) __attribute__((__nonnull__ (1))); +/* Return information about allocated pages and holes. */ +extern int sparse_array_extents (struct sparse_array *sa, + uint32_t cou...

...t __attribute__((__nonnull__ (1))) virtual_size (struct regions *regions) { if (regions->nr_regions == 0) diff --git a/common/sparse/sparse.h b/common/sparse/sparse.h index 3acb0af..818d804 100644 --- a/common/sparse/sparse.h +++ b/common/sparse/sparse.h @@ -64,12 +64,16 @@ extern void free_sparse_array (struct sparse_array *sa); /* Read bytes from the sparse array. * Note this can never return an error and never allocates. */ -extern void sparse_array_read (struct sparse_array *sa, void *buf, uint32_t count, uint64_t offset); +extern void sparse_array_read (struct sparse_array *sa, void *buf...

v1 was here: https://www.redhat.com/archives/libguestfs/2019-January/msg00008.html In v2 I have provided two patches: The first patch extends attribute((nonnull)) to most internal functions, but not to the external API. The second patch uses a macro so that attribute((format)) is only used in the public API on GCC or Clang. At least in theory these headers could be used by a C compiler which

..."iszero.h" #include "sparse.h" +#include "vector.h" /* Two level directory for the sparse array. * @@ -100,9 +101,10 @@ struct l1_entry { void **l2_dir; /* Pointer to L2 directory. */ }; +DEFINE_VECTOR_TYPE(l1_dir, struct l1_entry); + struct sparse_array { - struct l1_entry *l1_dir; /* L1 directory. */ - size_t l1_size; /* Number of entries in L1 directory. */ + l1_dir l1_dir; /* L1 directory. */ bool debug; }; @@ -123,9 +125,9 @@ free_sparse_array (struct sparse_array *sa) size_t i; if (sa) { -...

When sparse_array_zero() is used for a range larger than a page, there's no need to waste time in memset() or is_zero() - we already know the page will be free()d. Signed-off-by: Eric Blake <eblake@redhat.com> --- Here's a fun one :) common/sparse/sparse.c | 7 +++++-- 1 file changed, 5 insertions(...

On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > These plugins are both based on the same sparse array structure which > supports a simple implementation of extents. > --- > +int > +sparse_array_extents (struct sparse_array *sa, > + uint32_t count, uint64_t offset, > + struct nbdkit_extents *extents) > +{ > + uint32_t n, type; > + void *p; > + > + while (count > 0) { > + p = lookup (sa, offset, false, &n, NULL);...

...44 --- a/common/sparse/sparse.h +++ b/common/sparse/sparse.h @@ -1,5 +1,5 @@ /* nbdkit - * Copyright (C) 2017-2018 Red Hat Inc. + * Copyright (C) 2017-2019 Red Hat Inc. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -87,4 +87,9 @@ extern void sparse_array_zero (struct sparse_array *sa, uint32_t count, uint64_t offset) __attribute__((__nonnull__ (1))); +/* Return information about allocated pages and holes. */ +extern int sparse_array_extents (struct sparse_array *sa, + uint32_t cou...

--- plugins/vddk/vddk.c | 41 +++++++++++++++++++++++++---------------- TODO | 1 - 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c index 87c0d146..d1a3015f 100644 --- a/plugins/vddk/vddk.c +++ b/plugins/vddk/vddk.c @@ -51,6 +51,7 @@ #include "isaligned.h" #include "minmax.h" #include

...9-April/msg00192.html Thanks: Eric Blake. --- common/sparse/sparse.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/common/sparse/sparse.c b/common/sparse/sparse.c index b46bd31..cb44743 100644 --- a/common/sparse/sparse.c +++ b/common/sparse/sparse.c @@ -370,8 +370,6 @@ sparse_array_extents (struct sparse_array *sa, while (count > 0) { p = lookup (sa, offset, false, &n, NULL); - if (n > count) - n = count; /* Work out the type of this extent. */ if (p == NULL) @@ -388,6 +386,9 @@ sparse_array_extents (struct sparse_array *sa, if (nb...

...| 16 ++++- > plugins/memory/memory.c | 16 ++++- > README | 2 + > tests/Makefile.am | 2 + > tests/test-data-extents.sh | 131 +++++++++++++++++++++++++++++++++++++ > 7 files changed, 207 insertions(+), 4 deletions(-) > > +int > +sparse_array_extents (struct sparse_array *sa, > + uint32_t count, uint64_t offset, > + struct nbdkit_extents *extents) > +{ > + uint32_t n, type; > + void *p; > + > + while (count > 0) { > + p = lookup (sa, offset, false, &n, NULL);...

On 4/23/19 4:49 PM, Richard W.M. Jones wrote: >> >> ...to here, after the final nbdkit_add_extent, so that we can return a >> larger extent than the client's request. I remember when I originally >> asked, you declined due to odd interactions with REQ_ONE semantics, but >> since then, we changed how add_extent() works. Does it work now to defer >> the

On Sat, Mar 23, 2019 at 12:05:51PM -0500, Eric Blake wrote: > On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > > These plugins are both based on the same sparse array structure which > > supports a simple implementation of extents. > > --- > > > +int > > +sparse_array_extents (struct sparse_array *sa, > > + uint32_t count, uint64_t offset, > > + struct nbdkit_extents *extents) > > +{ > > + uint32_t n, type; > > + void *p; > > + > > + while (count > 0) { > > + p = l...

...ry/memory.c | 16 ++++- > > README | 2 + > > tests/Makefile.am | 2 + > > tests/test-data-extents.sh | 131 +++++++++++++++++++++++++++++++++++++ > > 7 files changed, 207 insertions(+), 4 deletions(-) > > > > +int > > +sparse_array_extents (struct sparse_array *sa, > > + uint32_t count, uint64_t offset, > > + struct nbdkit_extents *extents) > > +{ > > + uint32_t n, type; > > + void *p; > > + > > + while (count > 0) { > > + p = l...

On Wed, Aug 14, 2019 at 09:10:15PM -0500, Eric Blake wrote: > When sparse_array_zero() is used for a range larger than a page, > there's no need to waste time in memset() or is_zero() - we already > know the page will be free()d. > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- > > Here's a fun one :) > > common/sparse/sparse...

I will be following up to this email with four separate threads each addressed to the appropriate single list, with proposed changes to: - the NBD protocol - qemu: both server and client - libnbd: client - nbdkit: server The feature in question adds a new optional NBD_INFO_ packet to the NBD_OPT_GO portion of handshake, adding up to 16 bits of information that the server can advertise to the

...05:51PM -0500, Eric Blake wrote: > > On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > > > These plugins are both based on the same sparse array structure which > > > supports a simple implementation of extents. > > > --- > > > > > +int > > > +sparse_array_extents (struct sparse_array *sa, > > > + uint32_t count, uint64_t offset, > > > + struct nbdkit_extents *extents) > > > +{ > > > + uint32_t n, type; > > > + void *p; > > > + > > > + while (c...

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11]