search for: nbdplug_open_handle

Another series on top of my exportname filter, marking off another todo bullet point. With this, you can now use the NBD plugin as a transparent passthrough of all export names served by the remote server in both directions (list advertisement server to client, and export name from client to server). Eric Blake (3): nbd: Implement .default_export, .export_description nbd: Add

...struct handle { /* These fields are read-only once initialized */ struct nbd_handle *nbd; int fd; /* Cache of nbd_aio_get_fd */ - int fds[2]; /* Pipe for kicking the reader thread */ bool readonly; pthread_t reader; @@ -105,6 +105,15 @@ static char *tls_psk; static struct handle *nbdplug_open_handle (int readonly); static void nbdplug_close_handle (struct handle *h); +/* Original signal mask, with SIGUSR1 unblocked */ +static sigset_t origmask; + +/* No-op signal handler for interrupting ppoll on SIGUSR1 */ +static void +nbdplug_sigusr1 (int sig) +{ +} + static void nbdplug_unload (void)...

...dplug_reader (void *handle) while (!nbd_aio_is_dead (h->nbd) && !nbd_aio_is_closed (h->nbd)) { struct pollfd fds[2] = { - [0].fd = h->fd, + [0].fd = nbd_aio_get_fd (h->nbd), [1].fd = h->fds[0], [1].events = POLLIN, }; @@ -469,7 +468,6 @@ nbdplug_open_handle (int readonly) #endif retry: - h->fd = -1; h->nbd = nbd_create (); if (!h->nbd) goto err; @@ -505,9 +503,6 @@ nbdplug_open_handle (int readonly) } goto err; } - h->fd = nbd_aio_get_fd (h->nbd); - if (h->fd == -1) - goto err; if (readonly)...

I fixed the deadlock - turned out to be an actual bug in the nbd plugin (see patch 8). I changed the command syntax so it's now: nbdkit nbd command=qemu arg=-f arg=qcow2 arg=/path/to/disk.qcow2 Nir wrote: 18:08 < nsoffer> rwmjones: regarding the nbd proxy patches, did you have specific flow that help us? 18:08 < nsoffer> rwmjones: or this is just a way to support qcow2 in the

...void) } nbd_close (nbd); } + return 0; +} - /* Create the shared connection. */ +/* Create the shared connection. Because this may create a background + * thread it must be done after we fork. + */ +static int +nbdplug_after_fork (void) +{ if (shared && (shared_handle = nbdplug_open_handle (false)) == NULL) return -1; return 0; @@ -858,6 +865,7 @@ static struct nbdkit_plugin plugin = { .config_complete = nbdplug_config_complete, .config_help = nbdplug_config_help, .magic_config_key = "uri", + .after_fork = nbdplug_after_fork, .dump_p...

...defined HAVE_MKOSTEMP || defined HAVE_PIPE2 # error "Unexpected: your system has incomplete atomic CLOEXEC support" # endif int f; diff --git a/plugins/nbd/nbd.c b/plugins/nbd/nbd.c index e8bc7798..95d910e7 100644 --- a/plugins/nbd/nbd.c +++ b/plugins/nbd/nbd.c @@ -431,11 +431,42 @@ nbdplug_open_handle (int readonly) nbdkit_error ("malloc: %m"); return NULL; } +#ifdef HAVE_PIPE2 if (pipe2 (h->fds, O_NONBLOCK)) { + nbdkit_error ("pipe2: %m"); + free (h); + return NULL; + } +#else + /* This plugin doesn't fork, so we don't care about CLOEXEC...

...d) printf ("libnbd_version=%s\n", nbd_get_version (nbd)); printf ("libnbd_tls=%d\n", nbd_supports_tls (nbd)); printf ("libnbd_uri=%d\n", nbd_supports_uri (nbd)); + printf ("libnbd_vsock=%d\n", USE_VSOCK); nbd_close (nbd); } @@ -484,6 +515,12 @@ nbdplug_open_handle (int readonly) r = nbd_connect_uri (h->nbd, uri); else if (sockname) r = nbd_connect_unix (h->nbd, sockname); + else if (raw_cid) +#if !USE_VSOCK + abort (); +#else + r = nbd_connect_vsock (h->nbd, cid, vport); +#endif else r = nbd_connect_tcp (h->nbd, hostna...

libnbd-0.1.2-1 is now available in Fedora 29/30 updates-testing, although it was not compiled against libxml2 so it lacks uri support (I ended up testing patch 4 with a self-built libnbd). Diffs since v1 - rebase to master, bump from libnbd 0.1 to 0.1.2, add URI support, better timing results Still not done - patch 5 needs associated tests Eric Blake (5): nbd: Check for libnbd nbd:

...d) printf ("libnbd_version=%s\n", nbd_get_version (nbd)); printf ("libnbd_tls=%d\n", nbd_supports_tls (nbd)); printf ("libnbd_uri=%d\n", nbd_supports_uri (nbd)); + printf ("libnbd_vsock=%d\n", USE_VSOCK); nbd_close (nbd); } @@ -545,6 +576,12 @@ nbdplug_open_handle (int readonly) r = nbd_connect_unix (h->nbd, sockname); else if (hostname) r = nbd_connect_tcp (h->nbd, hostname, port); + else if (raw_cid) +#if !USE_VSOCK + abort (); +#else + r = nbd_connect_vsock (h->nbd, cid, vport); +#endif else if (command.size > 0) r...

The first four patches are fairly routine clean up and can be reviewed/applied on their own. The fifth patch is problematic as described below. At the moment if you want to proxy through to qemu-nbd (eg. for handling qcow2 files) it's rather complicated and you end up having to manage the sockets and clean up yourself. However the library we use for the proxying supports a perfectly good

Patch 1 played with an early draft of Rich's Fedora 30 libnbd package: https://bugzilla.redhat.com/show_bug.cgi?id=1713767#c17 Note that comment 21 provides a newer package 0.1.1-1 with a different API; and that libnbd has more unreleased API changes in the pipeline (whether that will be called 0.2 or 0.1.2); so we'll have to tweak things based on what is actually available in distros.

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

...0); + err = nbd_aio_command_completed (h->nbd, trans->cookie); + assert (err != 0); + assert (err == 1 ? trans->err == 0 : trans->err == nbd_get_errno ()); err = trans->err; + } } if (sem_destroy (&trans->sem)) abort (); @@ -520,13 +481,8 @@ nbdplug_open_handle (int readonly) h->readonly = true; /* Spawn a dedicated reader thread */ - if ((errno = pthread_mutex_init (&h->trans_lock, NULL))) { - nbdkit_error ("failed to initialize transaction mutex: %m"); - goto err; - } if ((errno = pthread_create (&h->reader...

libnbd-0.1.4-1 is now available in Fedora 29/30 updates testing. Diffs since v2 - rebase to master, bump from libnbd 0.1.2 to 0.1.3+, add tests to TLS usage which flushed out the need to turn relative pathnames into absolute, doc tweaks Now that the testsuite covers TLS and libnbd has been fixed to provide the things I found lacking when developing v2, I'm leaning towards pushing this on

...%s\n", nbd_get_version (nbd)); > printf ("libnbd_tls=%d\n", nbd_supports_tls (nbd)); > printf ("libnbd_uri=%d\n", nbd_supports_uri (nbd)); > + printf ("libnbd_vsock=%d\n", USE_VSOCK); > nbd_close (nbd); > } > > @@ -545,6 +576,12 @@ nbdplug_open_handle (int readonly) > r = nbd_connect_unix (h->nbd, sockname); > else if (hostname) > r = nbd_connect_tcp (h->nbd, hostname, port); > + else if (raw_cid) > +#if !USE_VSOCK > + abort (); > +#else > + r = nbd_connect_vsock (h->nbd, cid, vport); > +#e...

...tf ("libnbd_uri=%d\n", nbd_supports_uri (nbd)); printf ("libnbd_vsock=%d\n", USE_VSOCK); +#if LIBNBD_HAVE_NBD_OPT_LIST + printf ("libnbd_dynamic_list=1\n"); +#else + printf ("libnbd_dynamic_list=0\n"); +#endif nbd_close (nbd); } @@ -685,6 +690,53 @@ nbdplug_open_handle (int readonly, const char *client_export) return NULL; } +#if LIBNBD_HAVE_NBD_OPT_LIST +static int +collect_one (void *opaque, const char *name, const char *desc) +{ + struct nbdkit_exports *exports = opaque; + + if (nbdkit_add_export (exports, name, desc) == -1) + nbdkit_debug ("Una...

...init (&trans->sem, 0, 0)) + assert (false); + trans->cb.callback = nbdplug_notify; + trans->cb.user_data = trans; +} + /* Register a cookie and kick the I/O thread. */ static void nbdplug_register (struct handle *h, struct transaction *trans, int64_t cookie) @@ -466,7 +468,7 @@ nbdplug_open_handle (int readonly) goto err; if (nbd_set_export_name (h->nbd, export) == -1) goto err; - if (nbd_add_meta_context (h->nbd, "base:allocation") == -1) + if (nbd_add_meta_context (h->nbd, LIBNBD_CONTEXT_BASE_ALLOCATION) == -1) goto err; if (nbd_set_tls (h->nbd,...

I'm not observing any noticeable performance differences, but I'm liking the diffstat. I can't push this patch until we release a new libnbd version with the _notify API addition, but am posting it now for playing with things. Eric Blake (2): nbd: Move transaction info from heap to stack nbd: Use nbdkit aio_*_notify variants plugins/nbd/nbd.c | 217

...f (setsockopt (fd, IPPROTO_TCP, TCP_NODELAY, &optval, - sizeof (int)) == -1) { - nbdkit_error ("cannot set TCP_NODELAY option: %m"); - close (fd); - return -1; - } - return fd; -} - /* Create the shared or per-connection handle. */ static struct handle * nbdplug_open_handle (int readonly) { struct handle *h; - struct old_handshake old; - uint64_t version; + int r; h = calloc (1, sizeof *h); if (h == NULL) { nbdkit_error ("malloc: %m"); return NULL; } + if (pipe (h->fds)) { + nbdkit_error ("pipe: %m"); + free (h)...

...f (setsockopt (fd, IPPROTO_TCP, TCP_NODELAY, &optval, - sizeof (int)) == -1) { - nbdkit_error ("cannot set TCP_NODELAY option: %m"); - close (fd); - return -1; - } - return fd; -} - /* Create the shared or per-connection handle. */ static struct handle * nbdplug_open_handle (int readonly) { struct handle *h; - struct old_handshake old; - uint64_t version; + int r; h = calloc (1, sizeof *h); if (h == NULL) { nbdkit_error ("malloc: %m"); return NULL; } + if (pipe (h->fds)) { + nbdkit_error ("pipe: %m"); + free (h)...