similar to: [PATCH nbdkit 0/2] Generalize the tmpdisk plugin.

An evolution of v1 here: https://www.redhat.com/archives/libguestfs/2020-April/msg00035.html I want to generalize the tmpdisk plugin, particularly so you can use commands like ‘qemu-img create’ or even ‘virt-builder’. (Actually virt-builder really works - I tested it - but of course it causes a 30+ second delay when connecting to the server.) You can now use commands such as: nbdkit tmpdisk

v2 was here: https://www.redhat.com/archives/libguestfs/2020-April/msg00075.html In v3: - Add [VAR=VALUE ...] to manual. - Various minor improvements to the manual. - Work (at least, in theory - not tested) with block devices or symlinks. I didn't document this because it's hard to ensure these files or block devices would be cleaned up, so here be dragons. - Remove O_NOCTTY. -

v2 was here: https://www.redhat.com/archives/libguestfs/2020-March/msg00154.html v3: - Micro-optimize tmpdir. - Quote $disk in default command shell fragment. - Don't redirect mkfs output to /dev/null. Instead use exec </dev/null >/dev/null before the shell fragment. We may want to do this in other places where we run external shell scripts, or more generally for all

Unfinished (needs tests). This is my attempt to make a "remote tmpfs" plugin as outlined in this prior email: https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html Although it would be possible to construct something a bit like this using existing plugins and filters (perhaps with some new features in those filters) I think it may be nicer to have a dedicated plugin for

This can be used for creating temporary disks to thin clients, as a kind of "remote tmpfs". See also: https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html --- plugins/data/nbdkit-data-plugin.pod | 1 + plugins/file/nbdkit-file-plugin.pod | 1 + plugins/linuxdisk/nbdkit-linuxdisk-plugin.pod | 7 +- plugins/memory/nbdkit-memory-plugin.pod |

This allows us to be much more flexible about what commands can be used. It also means we do not need to encode any special behaviour for type or label parameters. XXX However one problem which is not addressed here: The $disk passed to the shell script fragment must be updated, not replaced. This is because currently the plugin has a file descriptor opened on this file. If it is deleted by

This allows us to be much more flexible about what commands can be used. It also means we do not need to encode any special behaviour for type or label parameters. --- plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod | 91 +++++++++----- plugins/tmpdisk/tmpdisk.c | 147 ++++++++++++++-------- plugins/tmpdisk/default-command.sh.in | 6 + 3 files changed, 164 insertions(+), 80

This allows us to be much more flexible about what commands can be used. It also means we do not need to encode any special behaviour for type or label parameters. --- plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod | 115 ++++++++++---- tests/Makefile.am | 2 + plugins/tmpdisk/tmpdisk.c | 184 ++++++++++++++++------ plugins/tmpdisk/default-command.sh.in |

On 3/17/20 3:53 AM, Richard W.M. Jones wrote: > This can be used for creating temporary disks to thin clients, as a > kind of "remote tmpfs". > > See also: > https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html > --- > +++ b/plugins/tmpdisk/nbdkit-tmpdisk-plugin.pod > @@ -0,0 +1,157 @@ > +=head1 NAME > + > +nbdkit-tmpdisk-plugin - create

Neutral refactoring. Makes the code a bit easier to handle and simplifies future commits. --- plugins/tmpdisk/Makefile.am | 17 ++++++++- plugins/tmpdisk/tmpdisk.c | 22 +++--------- plugins/tmpdisk/default-command.sh.in | 51 +++++++++++++++++++++++++++ .gitignore | 1 + 4 files changed, 72 insertions(+), 19 deletions(-) diff --git

This creates filesystems on demand. A client simply connects with a desired export name and a new export is created. The export is persistent (until deleted by the server admin), and clients may disconnect and reconnect. In some respects this is similar to the nbdkit-tmpdisk-plugin, or nbdkit-file-plugin with the dir= option. --- plugins/ondemand/nbdkit-ondemand-plugin.pod | 190 ++++++

For context see these threads: https://lists.gnu.org/archive/html/qemu-discuss/2020-06/threads.html#00053 https://lists.gnu.org/archive/html/qemu-block/2020-06/threads.html#01496 Rich.

This parameter provided a name for the "default export" -- ie. the one and only export returned to the client by NBD_OPT_LIST. But nbdkit traditionally didn't care what export name the client requested. Since 1.16 plugins have been able to serve different content per export name (and return errors for unknown exports), but the -e option didn't reflect that and only created

Since v1: - new patch to add uri_quote() - rebase on top of other recent patches needed while auditing shell_quote() - use uri_quote() instead of shell_quote() for producing $uri Eric Blake (2): common/utils: Add uri_quote and tests captive: Support $uri in --run docs/nbdkit-captive.pod | 8 ++- common/utils/utils.h | 1 + common/utils/test-quotes.c | 108

Since the v1 series (0/4, at [1]), I've applied patches 1 and 2, rewritten patch 3 [Forbid NUL in export and context names] into patch 4 here, patch 4 there turned into patch 6 here, and everything else here is new. [1]https://www.redhat.com/archives/libguestfs/2019-September/msg00180.html I don't know if there is a handy reusable function for checking whether a string contains valid

Needs some work still, see in particular the commit message for patch 3. Rich.

Evolution of this patch series: https://www.redhat.com/archives/libguestfs/2020-July/thread.html#00073 Instead of auth-script, this implements header-script and cookie-script. It can be used for similar purposes but the implementation is somewhat saner. Rich.

This is an RFC only, at the very least it lacks tests. This implements a rather complex new feature in nbdkit-curl-plugin allowing you to specify an external shell script that can be used to fetch an authorization token for services which requires a token or cookie for access, especially if that token must be renewed periodically. The motivation can be seen in the changes to the docs in patch 2.

For review only, this needs some clean up and more tests. My eyes are going cross-eyed looking at the calculate_offset_of_entry function, so time to take a break ... Rich.

The existing --run '$nbd' outputs an older form that differs between libguestfs and qemu, and which is not always a valid URI. For historical compatibility, we probably can't change that; but we can instead add a new '$uri' that outputs a valid URI. Note that the libguestfs '$nbd' TCP form is already a valid URI, but that libguestfs still needs to be taught about the