similar to: [PATCH nbdkit] New ondemand plugin.

On 8/14/20 12:20 PM, Richard W.M. Jones wrote: > This creates filesystems on demand. A client simply connects with a > desired export name and a new export is created. The export is > persistent (until deleted by the server admin), and clients may > disconnect and reconnect. In some respects this is similar to the > nbdkit-tmpdisk-plugin, or nbdkit-file-plugin with the dir=

This allows us to be much more flexible about what commands can be used. It also means we do not need to encode any special behaviour for type or label parameters. --- plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod | 115 ++++++++++---- tests/Makefile.am | 2 + plugins/tmpdisk/tmpdisk.c | 184 ++++++++++++++++------ plugins/tmpdisk/default-command.sh.in |

On Sat, Aug 15, 2020 at 03:41:39PM -0500, Eric Blake wrote: > On 8/14/20 12:20 PM, Richard W.M. Jones wrote: > >+Similar plugins include L<nbdkit-file-plugin(1)> which can serve a > >+predefined set of exports (clients cannot create more), > > Hmm - I wonder if it is worth a filter that runs a script any time > an .open fails. That script could send email to an

During the 1.21 development cycle we added support for listing exports. However at the time that 1.22 was released we did not feel the API was sufficiently finalized to commit to in a stable branch. Therefore this new API was removed before 1.22 was released, but the feature continues to be developed upstream and should appear in nbdkit 1.24. Note this also hides the nbdkit*export* functions so

If you're following nbdkit development upstream you'll have seen that we are still making changes to the .list_exports and related APIs. The current .list_exports API upstream is not how it will look finally. The latest set of proposals was here: https://www.redhat.com/archives/libguestfs/2020-August/thread.html#00330 At the same time I'd like to do an nbdkit 1.22 (stable) release.

I'm about to add an 'exportname' filter, and in the process, I noticed a few shortcomings in our API. Time to fix those before the 1.22 release locks our API in stone. Overloading .list_exports in order to determine a canonical export name at .open time is awkward; the two uses (answering NBD_OPT_LIST for a full list, vs. remapping a client's "" into a canonical name

This is a revision of my .default_export work, plus new work on .export_descriptions and a new exportname filter. I think it is now ready to check in. Things I'd still like in 1.22: - the file plugin should implement .list_exports (patch already posted, but it needs rebasing on this series) - the ext2 filter should override .list_exports when in exportname mode - the nbd plugin should be

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

I'm about to add an 'exportname' filter, and in the process, I noticed a few shortcomings in our API. Time to fix those before the 1.22 release locks our API in stone. First, .list_exports needs to know if it is pre- or post-TLS, as that may affect which names are exported. Next, overloading .list_exports to do both NBD_OPT_LIST and mapping "" to a canonical name is

More patches on the way for improving .list_exports signature and adding .export_description, but this is the promised code showing why nbdkit_string_intern is useful. Patch 4 is somewhat RFC: we could either add new API to take the boilerplate from: foo_config(const char *key, const char *value) { if (strcmp (key, "file") == 0) { CLEANUP_FREE char *tmp = nbdkit_realpath (value);

Add a new filter to make it easier to add exports to a plugin that does advertise them, to avoid advertising where a plugin's list might be an information leak, and to alter which export name is used in place of "". I would love to be able to have a strict mode enforcing that .open is called only with an export name that the plugin was willing to advertise, but for that, we'll

On Thu, Aug 06, 2020 at 09:23:46PM -0500, Eric Blake wrote: > + if (!filename == !directory) { A bit tricksy. In plugins/nbd/nbd.c I used: int c = !!sockname + !!hostname + !!uri + (command.size > 0) + (socket_fd >= 0) + !!raw_cid; /* Check the user passed exactly one connection parameter. */ if (c > 1) { nbdkit_error ("cannot mix Unix ‘socket’, TCP

On Thu, Aug 06, 2020 at 09:23:48PM -0500, Eric Blake wrote: > Add a new filter to make it easier to add exports to a plugin that > does advertise them, to avoid advertising where a plugin's list might does *not* advertise them(?) > be an information leak, and to alter which export name is used in > place of "". > > I would love to be able to have a strict mode

When using ext2file=exportname to pick the file to server from the client's export name, we do not want to leak the underlying plugin's export list. While touching this, take advantage of string lifetimes via nbdkit_strdup_intern and similar for less cleanup bookkeeping. Note that we don't actually implement a full .list_exports; doing that with NBD_OPT_LIST is likely prohibitive

On 8/15/20 4:13 PM, Richard W.M. Jones wrote: >>> +/* Because we rewind the exportsdir handle, we need a lock to protect >>> + * list_exports from being called in parallel. >>> + */ >>> +static pthread_mutex_t exports_lock = PTHREAD_MUTEX_INITIALIZER; >> >> An alternative is to diropen() each time .list_exports gets called. >> Either way

This parameter provided a name for the "default export" -- ie. the one and only export returned to the client by NBD_OPT_LIST. But nbdkit traditionally didn't care what export name the client requested. Since 1.16 plugins have been able to serve different content per export name (and return errors for unknown exports), but the -e option didn't reflect that and only created

Here's changes to the file plugin (which I'm happy with) and a new exportname filter (which is still at RFC stage; I need to finish implementing strict mode in .open, and add tests). I also discovered that we really want .list_exports and .open to know when they are used on plaintext vs. tls clients for --tls=on, and we may want to split out a new .default_export callback rather than

This allows us to be much more flexible about what commands can be used. It also means we do not need to encode any special behaviour for type or label parameters. --- plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod | 91 +++++++++----- plugins/tmpdisk/tmpdisk.c | 147 ++++++++++++++-------- plugins/tmpdisk/default-command.sh.in | 6 + 3 files changed, 164 insertions(+), 80

Add a new mode to the file plugin, using directory=DIR instead of [file=]FILE, to allow it to serve all regular/block files in a given directory, as well as advertising the names of those files it will be serving. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/file/nbdkit-file-plugin.pod | 28 +++++- tests/Makefile.am | 4 +- plugins/file/file.c

This can be used for creating temporary disks to thin clients, as a kind of "remote tmpfs". See also: https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html --- plugins/data/nbdkit-data-plugin.pod | 1 + plugins/file/nbdkit-file-plugin.pod | 1 + plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod | 7 +- plugins/memory/nbdkit-memory-plugin.pod |