search for: 5b7abc4

...d an appropriate error > (unless C<errno> is sufficient), then return C<-1>. > >+=head2 C<.extents> >+ >+ int zero (void *handle, uint32_t count, uint64_t offset, uint32_t flags, s/zero/extents/ [...] >diff --git a/server/filters.c b/server/filters.c >index 5b7abc4..d6cc00f 100644 >--- a/server/filters.c >+++ b/server/filters.c [...] >@@ -646,6 +680,31 @@ filter_zero (struct backend *b, struct connection *conn, > count, offset, flags, err); > } > >+static int >+filter_extents (struct backend *b, stru...

This tentative commit implements extents/can_extents, roughly as discussed in the previous thread here: https://www.redhat.com/archives/libguestfs/2019-March/msg00017.html I can't say that I'm a big fan of having the plugin allocate an extents array. There are no other plugin callbacks currently where we require the plugin to allocate complex data structures (or indeed do any allocation

...al = true; } + fl = backend->can_multi_conn (backend, conn); + if (fl == -1) + return -1; + if (fl) { + eflags |= NBD_FLAG_CAN_MULTI_CONN; + conn->can_multi_conn = true; + } + *flags = eflags; return 0; } diff --git a/server/filters.c b/server/filters.c index d02e7fb..5b7abc4 100644 --- a/server/filters.c +++ b/server/filters.c @@ -316,6 +316,13 @@ next_can_fua (void *nxdata) return b_conn->b->can_fua (b_conn->b, b_conn->conn); } +static int +next_can_multi_conn (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can_multi_c...

...extents) (struct backend *, struct connection *conn, uint32_t count, + uint64_t offset, uint32_t flags, + size_t *nr_extents, struct nbdkit_extent **extents, + int *err); }; /* plugins.c */ diff --git a/server/filters.c b/server/filters.c index 5b7abc4..d6cc00f 100644 --- a/server/filters.c +++ b/server/filters.c @@ -309,6 +309,13 @@ next_can_zero (void *nxdata) return b_conn->b->can_zero (b_conn->b, b_conn->conn); } +static int +next_can_extents (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can...

...truct nbdkit_extents_map *, + int (*fn) (uint64_t offset, uint64_t length, + uint32_t type, void *opaque), + void *opaque); +#endif diff --git a/server/filters.c b/server/filters.c index 5b7abc4..e284080 100644 --- a/server/filters.c +++ b/server/filters.c @@ -309,6 +309,13 @@ next_can_zero (void *nxdata) return b_conn->b->can_zero (b_conn->b, b_conn->conn); } +static int +next_can_extents (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can...

...set, uint64_t range_length, + int (*fn) (uint64_t offset, uint64_t length, + uint32_t type, void *opaque), + void *opaque); +#endif diff --git a/server/filters.c b/server/filters.c index 5b7abc4..e284080 100644 --- a/server/filters.c +++ b/server/filters.c @@ -309,6 +309,13 @@ next_can_zero (void *nxdata) return b_conn->b->can_zero (b_conn->b, b_conn->conn); } +static int +next_can_extents (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can...

...ts->nr_extents-1].length += length; + return 0; + } + else { + /* Add a new extent. */ + const struct nbdkit_extent e = + { .offset = offset, .length = length, .type = type }; + return append_extent (exts, &e); + } +} diff --git a/server/filters.c b/server/filters.c index 5b7abc4..5095188 100644 --- a/server/filters.c +++ b/server/filters.c @@ -309,6 +309,13 @@ next_can_zero (void *nxdata) return b_conn->b->can_zero (b_conn->b, b_conn->conn); } +static int +next_can_extents (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can...

...art. + */ + overlap = exts->start - offset; + length -= overlap; + offset += overlap; + } + + const struct nbdkit_extent e = + { .offset = offset, .length = length, .type = type }; + return append_extent (exts, &e); +} diff --git a/server/filters.c b/server/filters.c index 5b7abc4..5095188 100644 --- a/server/filters.c +++ b/server/filters.c @@ -309,6 +309,13 @@ next_can_zero (void *nxdata) return b_conn->b->can_zero (b_conn->b, b_conn->conn); } +static int +next_can_extents (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can...

I'm not sure which version we're up to now. Anyway I believe this addresses all the points that Eric raised in: https://www.redhat.com/archives/libguestfs/2019-March/msg00038.html https://www.redhat.com/archives/libguestfs/2019-March/msg00040.html In particular: - default state of extents_map is all allocated disk - support hole + non-zero - you can now iterate with bounds -

...ts->nr_extents-1].length += length; + return 0; + } + else { + /* Add a new extent. */ + const struct nbdkit_extent e = + { .offset = offset, .length = length, .type = type }; + return append_extent (exts, &e); + } +} diff --git a/server/filters.c b/server/filters.c index 5b7abc4..5095188 100644 --- a/server/filters.c +++ b/server/filters.c @@ -309,6 +309,13 @@ next_can_zero (void *nxdata) return b_conn->b->can_zero (b_conn->b, b_conn->conn); } +static int +next_can_extents (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can...

Second version based on nbdkit_extent* calls, as discussed here: https://www.redhat.com/archives/libguestfs/2019-March/msg00033.html Note in particular there is some subtlety about how filters would work in this implementation. See docs/nbdkit-filter.pod for the details. Rich.

First thing to say is that I need to do a *lot* more testing on this, so this is just an early peek. In particular, although it passed ‘make check && make check-valgrind’ I have *not* tested it against a multi-conn-aware client such as the Linux kernel >= 4.9. This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc as: "NBD_FLAG_CAN_MULTI_CONN: Indicates that

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11]

..._map: %zu: " + "adjacent extents with same type not coalesced\n", i); + fail = true; + } + } + } + + if (fail) { + dump_extents_map (map); + abort (); + } +} + +#endif /* IN_TEST_EXTENTS */ diff --git a/server/filters.c b/server/filters.c index 5b7abc4..e284080 100644 --- a/server/filters.c +++ b/server/filters.c @@ -309,6 +309,13 @@ next_can_zero (void *nxdata) return b_conn->b->can_zero (b_conn->b, b_conn->conn); } +static int +next_can_extents (void *nxdata) +{ + struct b_conn *b_conn = nxdata; + return b_conn->b->can...

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

I want to post this but mainly for discussion and early review. It's not safe for these patches to all go upstream yet (because not all filters have been checked/adjusted), but if any patches were to go upstream then probably 1 & 2 only are safe. File, VDDK, memory and data plugins all work, although I have only done minimal testing on them. The current tests, such as they are, all

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.