search for: backend_cach

...bably populate next_ops with backend_* functions rather than next_* wrapper functions: > @@ -410,8 +409,8 @@ static int > next_cache (void *nxdata, uint32_t count, uint64_t offset, > uint32_t flags, int *err) > { > - struct b_conn *b_conn = nxdata; > - return backend_cache (b_conn->b, b_conn->conn, count, offset, flags, err); > + struct b_h *b_h = nxdata; > + return backend_cache (b_h->b, count, offset, flags, err); > } > > static struct nbdkit_next_ops next_ops = { as in next_ops = { ... .cache = backend_cache, }; > stati...

The third patch is a large but mechanical change which gets rid of passing around struct connection * entirely within the server, preferring instead to reference the connection through thread-local storage. I hope this is a gateway to simplifying other parts of the code. Rich.

...extents, err); + struct backend *b_next = nxdata; + return backend_extents (b_next, count, offset, flags, extents, err); } static int next_cache (void *nxdata, uint32_t count, uint64_t offset, uint32_t flags, int *err) { - struct b_h *b_h = nxdata; - return backend_cache (b_h->b, count, offset, flags, err); + struct backend *b_next = nxdata; + return backend_cache (b_next, count, offset, flags, err); } static struct nbdkit_next_ops next_ops = { @@ -439,11 +408,9 @@ static int filter_prepare (struct backend *b, void *handle, int readonly) { struct bac...

...break; case NBD_CMD_TRIM: if (flags & NBD_CMD_FLAG_FUA) f |= NBDKIT_FLAG_FUA; - if (backend_trim (backend, count, offset, f, &err) == -1) + if (backend_trim (top, count, offset, f, &err) == -1) return err; break; case NBD_CMD_CACHE: - if (backend_cache (backend, count, offset, 0, &err) == -1) + if (backend_cache (top, count, offset, 0, &err) == -1) return err; break; @@ -273,14 +273,14 @@ handle_request (uint16_t cmd, uint16_t flags, uint64_t offset, uint32_t count, f |= NBDKIT_FLAG_FUA; if (flags & NBD_C...

...onn, + __attribute__((__nonnull__ (1, 5))); +extern int backend_extents (struct backend *b, uint32_t count, uint64_t offset, uint32_t flags, struct nbdkit_extents *extents, int *err) - __attribute__((__nonnull__ (1, 2, 6, 7))); -extern int backend_cache (struct backend *b, struct connection *conn, + __attribute__((__nonnull__ (1, 5, 6))); +extern int backend_cache (struct backend *b, uint32_t count, uint64_t offset, uint32_t flags, int *err) - __attribute__((__nonnull__ (1, 2, 6))); + __att...

v1 was here: https://www.redhat.com/archives/libguestfs/2020-February/msg00081.html v2 replaces struct connection *conn = GET_CONN; with GET_CONN; which sets conn implicitly and asserts that it is non-NULL. If we actually want to test if conn is non-NULL or behave differently, then you must use threadlocal_get_conn() instead, and some existing uses do that. Rich.

...nts *extents, int *err) -{ - struct backend *b_next = nxdata; - return backend_extents (b_next, count, offset, flags, extents, err); -} - -static int -next_cache (void *nxdata, uint32_t count, uint64_t offset, - uint32_t flags, int *err) -{ - struct backend *b_next = nxdata; - return backend_cache (b_next, count, offset, flags, err); -} - static struct nbdkit_next_ops next_ops = { - .reopen = next_reopen, - .get_size = next_get_size, - .can_write = next_can_write, - .can_flush = next_can_flush, - .is_rotational = next_is_rotational, - .can_trim = next_can_trim, - .can_zero = next_ca...

...nd *b, struct nbdkit_extents *extents, int *err) { GET_CONN; - struct b_conn_handle *h = &conn->handles[b->i]; + struct handle *h = get_handle (conn, b->i); int r; assert (h->handle && (h->state & HANDLE_CONNECTED)); @@ -632,7 +632,7 @@ backend_cache (struct backend *b, uint32_t flags, int *err) { GET_CONN; - struct b_conn_handle *h = &conn->handles[b->i]; + struct handle *h = get_handle (conn, b->i); int r; assert (h->handle && (h->state & HANDLE_CONNECTED)); diff --git a/server/con...

...++ b/server/internal.h @@ -153,6 +153,7 @@ extern int quit_fd; extern void set_up_quit_pipe (void); extern void close_quit_pipe (void); extern void handle_quit (int sig); +extern void do_nbdkit_shutdown (void); /* signals.c */ extern void set_up_signals (void); @@ -473,6 +474,7 @@ extern int backend_cache (struct backend *b, extern struct backend *plugin_register (size_t index, const char *filename, void *dl, struct nbdkit_plugin *(*plugin_init) (void)) __attribute__((__nonnull__ (2, 3, 4))); +extern void do_nbdkit_set_error (int err); /* filters.c */...

...p; NBDKIT_FLAG_REQ_ONE)); + if (invalid_range (b, conn, "extents", offset, count)) { + *err = EINVAL; + return -1; + } assert (h->can_extents >= 0); if (h->can_extents == 0) { /* By default it is safe assume that everything in the range is @@ -396,6 +486,10 @@ backend_cache (struct backend *b, struct connection *conn, debug ("%s: cache count=%" PRIu32 " offset=%" PRIu64, b->name, count, offset); + if (invalid_range (b, conn, "cache", offset, count)) { + *err = EINVAL; + return -1; + } assert (h->can_cache &...

It's easy to use the sh script to demonstrate that nbdkit is inefficiently calling into .get_size, .can_fua, and friends more than necessary. We've also commented on the list in the past that it would be nice to ensure that when filters call into next_ops, they are not violating constraints (as we've have to fix several bugs in the past where we did not have such checking to protect

...;b == b->next && nxdata->conn == conn); if (f->filter.cache) - return f->filter.cache (&next_ops, &nxdata, handle, + return f->filter.cache (&next_ops, nxdata, nxdata->handle, count, offset, flags, err); else return backend_cache (b->next, conn, count, offset, flags, err); -- 2.24.1

v1 was here: https://www.redhat.com/archives/libguestfs/2019-November/msg00003.html This version depends on: https://www.redhat.com/archives/libguestfs/2019-November/msg00004.html and this series: https://www.redhat.com/archives/libguestfs/2019-November/msg00009.html The delta has been reduced slightly because of changes made possible by cleaning up and fixing the quit path in nbdkit. It's

libFuzzer is Clang's fuzzer, and alternative to using AFL: https://llvm.org/docs/LibFuzzer.html I implemented an alternative method of fuzzing for libnbd earlier today and it's pretty simple: https://github.com/libguestfs/libnbd/commit/c19a6fbae9a21a7d4693418706c59e81ed256875 However it's considerably more difficult to use libFuzzer with non-library code -- in this case nbdkit.

I think this is my last round of patches for issues I identified with the retry filter. With this in place, it should be safe to interject another filter in between retry and the plugin. Eric Blake (5): retry: Don't call into closed plugin tests: Refactor test-retry-reopen-fail.sh tests: Enhance retry test to cover failed reopen server: Move prepare/finalize/close recursion to

The immediate reason for this patch is to reduce the amount of debugging in virt-v2v with using the virt-v2v -v option (because this implies running nbdkit in verbose mode too). Most of the messages are datapath ones about pread/pwrite requests, and in fact as we've added more filters on top of nbdkit these messages have got more and more verbose. However they are not particularly

I still don't have .prepare/.finalize working cleanly across reopen, but did find a nasty bug where a botched assertion means we failed to notice reads beyond EOF in both the xz and retry filter. Refactoring backend.c will make .finalize work easier. Eric Blake (5): xz: Avoid reading beyond EOF retry: Check size before transactions tests: Test retry when get_size values change

This is the previous 5/9 patch posted earlier today, split into reviewable chunks. This passes bisection with -x 'make && make check', but I didn't work very hard on the commit messages, so I refer you back to the original patch to explain how it works: https://www.redhat.com/archives/libguestfs/2020-March/msg00248.html Rich.

This creates libnbdkit.so as discussed in the following thread: https://www.redhat.com/archives/libguestfs/2020-March/thread.html#00203 test-delay-shutdown.sh fails for unclear reasons. This series starts by reverting "tests: Don't strand hung nbdkit processes" which is because several other tests fail randomly unless I revert this patch. I didn't investigate this yet so it