similar to: [RFC nbdkit PATCH] protocol: Alter .list_exports, add .default_export

This is a revision of my .default_export work, plus new work on .export_descriptions and a new exportname filter. I think it is now ready to check in. Things I'd still like in 1.22: - the file plugin should implement .list_exports (patch already posted, but it needs rebasing on this series) - the ext2 filter should override .list_exports when in exportname mode - the nbd plugin should be

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

More patches on the way for improving .list_exports signature and adding .export_description, but this is the promised code showing why nbdkit_string_intern is useful. Patch 4 is somewhat RFC: we could either add new API to take the boilerplate from: foo_config(const char *key, const char *value) { if (strcmp (key, "file") == 0) { CLEANUP_FREE char *tmp = nbdkit_realpath (value);

I'm about to add an 'exportname' filter, and in the process, I noticed a few shortcomings in our API. Time to fix those before the 1.22 release locks our API in stone. Overloading .list_exports in order to determine a canonical export name at .open time is awkward; the two uses (answering NBD_OPT_LIST for a full list, vs. remapping a client's "" into a canonical name

On Wed, Aug 26, 2020 at 09:16:46PM -0500, Eric Blake wrote: > Rather than defaulting .list_exports to blindly returning "", it is > nicer to have it reflect the result of .default_export. Meanwhile, > language bindings will have a C callback for both .list_exports and > .default_export, even if the underlying script does not, which means > any logic in plugins.c for

I'm about to add an 'exportname' filter, and in the process, I noticed a few shortcomings in our API. Having .default_export makes it easy to answer NBD_INFO_NAME in response to a client request during NBD_OPT_GO, but answering NBD_INFO_DESCRIPTION is awkward - there's no guarantee the export name was given with a description in .list_exports. Note, however, that while we map

This picks up python and ocaml. Some of our languages are lacking a number of bindings (for example, lua and perl lack .extents, so I didn't have anything to copy from), and I felt less comfortable with golang and rust. But for python and ocaml, I was able to test a working implementation. Eric Blake (2): python: Implement .list_exports and friends ocaml: Implement .list_exports and

This takes Rich's API proposal and starts fleshing it out with enough meat that I was able to test 'nbdkit eval' advertising multiple exports with descriptions paired with 'qemu-nbd --list'. Eric Blake (3): server: Add exports list functions server: Prepare to use export list from plugin sh, eval: Add .list_exports support Richard W.M. Jones (1): server: Implement

Since v1: - patch 1: check size limits - patch 2: better handling of default export name canonicalization - patch 3: support filters as well as plugins - patch 4: new - patch 5: rewrite sh parser, fix testsuite to actually work and cover more cases (now that libnbd.git is fixed) Eric Blake (4): server: Add exports list functions server: Prepare to use export list from plugin log: Add

On Mon, Aug 24, 2020 at 05:02:56PM -0500, Eric Blake wrote: > On 8/24/20 7:52 AM, Eric Blake wrote: > >I'm about to add an 'exportname' filter, and in the process, I > >noticed a few shortcomings in our API. Time to fix those before > >the 1.22 release locks our API in stone. First, .list_exports > >needs to know if it is pre- or post-TLS, as that may

On Sat, Aug 15, 2020 at 03:41:39PM -0500, Eric Blake wrote: > On 8/14/20 12:20 PM, Richard W.M. Jones wrote: > >+Similar plugins include L<nbdkit-file-plugin(1)> which can serve a > >+predefined set of exports (clients cannot create more), > > Hmm - I wonder if it is worth a filter that runs a script any time > an .open fails. That script could send email to an

Rather than defaulting .list_exports to blindly returning "", it is nicer to have it reflect the result of .default_export. Meanwhile, language bindings will have a C callback for both .list_exports and .default_export, even if the underlying script does not, which means any logic in plugins.c for calling .default_export when .list_export is missing would have to be duplicated in each

If you're following nbdkit development upstream you'll have seen that we are still making changes to the .list_exports and related APIs. The current .list_exports API upstream is not how it will look finally. The latest set of proposals was here: https://www.redhat.com/archives/libguestfs/2020-August/thread.html#00330 At the same time I'd like to do an nbdkit 1.22 (stable) release.

Use the recently added nbdkit_string_intern to make this possible. Testsuite coverage is added in both tls-fallback as well as a cleanup to test-eval-exports.sh. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/eval/nbdkit-eval-plugin.pod | 2 ++ plugins/sh/nbdkit-sh-plugin.pod | 20 +++++++++-- plugins/sh/methods.h | 1 + plugins/eval/eval.c |

Fairly straightforward. I'd love for type export to be a bit more flexible to make description optional, but could not figure out how to decode that from the C side of things, so for now this just requires the caller to supply a description for all exports during .list_exports. Signed-off-by: Eric Blake <eblake at redhat.com> --- plugins/ocaml/NBDKit.mli | 9 ++++

Fairly straightforward. A user can supply { "name"; None } or { "name"; Some "description" } as desired. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/ocaml/NBDKit.mli | 9 ++++ plugins/ocaml/NBDKit.ml | 17 ++++++++ plugins/ocaml/example.ml | 33 +++++++++----- plugins/ocaml/ocaml.c | 92 +++++++++++++++++++++++++++++++++++++++- 4 files

Here's changes to the file plugin (which I'm happy with) and a new exportname filter (which is still at RFC stage; I need to finish implementing strict mode in .open, and add tests). I also discovered that we really want .list_exports and .open to know when they are used on plaintext vs. tls clients for --tls=on, and we may want to split out a new .default_export callback rather than

On 8/24/20 7:52 AM, Eric Blake wrote: > I'm about to add an 'exportname' filter, and in the process, I > noticed a few shortcomings in our API. Time to fix those before > the 1.22 release locks our API in stone. First, .list_exports > needs to know if it is pre- or post-TLS, as that may affect which > names are exported. Next, overloading .list_exports to do both >

On 7/31/20 5:22 PM, Eric Blake wrote: > From: "Richard W.M. Jones" <rjones@redhat.com> > > See also: > https://www.redhat.com/archives/libguestfs/2020-July/msg00090.html > Message-Id: <20200722124201.1823468-2-rjones@redhat.com> > > For now, this implementation is only for plugins; adding filter > support will come later (and probably requires

Now that we can differentiate content based on export name, we also need to be able to differentiate content for a --tls=on server, since TLS is optional according to whether the client has authenticated. For internal code and filters, this means adding a new parameter; the sh plugin can do likewise. For plugins, we can't add a parameter until the V3 protocol, so in the meantime, we add