similar to: [PATCH nbdkit] Add ssh plugin using libssh.

--- plugins/curl/nbdkit-curl-plugin.pod | 22 +- plugins/ssh/nbdkit-ssh-plugin.pod | 150 ++++++++ configure.ac | 17 + plugins/ssh/ssh.c | 521 ++++++++++++++++++++++++++++ README | 4 + TODO | 10 +- plugins/ssh/Makefile.am | 69 ++++ 7 files changed, 779 insertions(+), 14

On Mon, Sep 23, 2019 at 12:05:11PM -0500, Eric Blake wrote: > > + int nbdkit_parse_long (const char *what, const char *str, long *r); > > + int nbdkit_parse_unsigned_long (const char *what, > > + const char *str, unsigned long *r); > > Do we really want to encourage the use of parse_long and > parse_unsigned_long? Those differ between

On 3/6/19 9:40 AM, Richard W.M. Jones wrote: > --- > plugins/curl/nbdkit-curl-plugin.pod | 22 +- > plugins/ssh/nbdkit-ssh-plugin.pod | 150 ++++++++ > configure.ac | 17 + > plugins/ssh/ssh.c | 521 ++++++++++++++++++++++++++++ > README | 4 + > TODO | 10 +- >

This is a retry filter implementation as outlined here: https://www.redhat.com/archives/libguestfs/2019-September/msg00167.html It is only lightly tested. One way to test it is to try an SSH copy (see the commit message for patch 2/2), and in the middle of the copy kill the per-connection sshd on the remote machine. You will see that the copy recovers after a few seconds. Add the nbdkit -v

This was a rather longer trip around the houses than I anticipated! The basic purpose of the patch series is to set $nbdkit_stdio_safe to "0" or "1" in sh and eval plugin scripts. To do that, I ended up adding a nicer way to manipulate environ lists, and to do that, I ended up adding a whole generic vector implementation which is applicable in a lot of different places.

Matt asked if xz should really be a filter rather than a plugin. The answer is yes, of course it should be! That's been something in the todo file for a while. The commit converts the xz plugin code into a filter (leaving the plugin around, but deprecating it). plugin: nbdkit xz file.xz filter: nbdkit --filter=xz file file.xz plugin: # can't be done filter: nbdkit

For review only, this needs some clean up and more tests. My eyes are going cross-eyed looking at the calculate_offset_of_entry function, so time to take a break ... Rich.

v1 was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00199.html v2: - Adds a fairly simple yet comprehensive test using sh plugin. - Rebase and retest. Patch 1 is a misc patch not really related to the series. Rich.

This is an RFC only, at the very least it lacks tests. This implements a rather complex new feature in nbdkit-curl-plugin allowing you to specify an external shell script that can be used to fetch an authorization token for services which requires a token or cookie for access, especially if that token must be renewed periodically. The motivation can be seen in the changes to the docs in patch 2.

v2 was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00221.html I think this is more like "the one". It handles reopen failing correctly, and there is a second test for that. I also ran my sshd tests locally and it worked in all scenarios I could think up (except of course sshd not being available at the start, but we want that to fail). Rich.

Evolution of this patch series: https://www.redhat.com/archives/libguestfs/2020-July/thread.html#00073 Instead of auth-script, this implements header-script and cookie-script. It can be used for similar purposes but the implementation is somewhat saner. Rich.

v2: - Fixes a number of bugs in corner cases. - Uses a 1M block size to fetch from the underlying plugin. This improves performance considerably. I also tested this much more thoroughly and can't find any more bugs. Rich.

v2 was here: https://www.redhat.com/archives/libguestfs/2020-March/msg00154.html v3: - Micro-optimize tmpdir. - Quote $disk in default command shell fragment. - Don't redirect mkfs output to /dev/null. Instead use exec </dev/null >/dev/null before the shell fragment. We may want to do this in other places where we run external shell scripts, or more generally for all

Unfinished (needs tests). This is my attempt to make a "remote tmpfs" plugin as outlined in this prior email: https://www.redhat.com/archives/libguestfs/2020-March/msg00134.html Although it would be possible to construct something a bit like this using existing plugins and filters (perhaps with some new features in those filters) I think it may be nicer to have a dedicated plugin for

v2 was here: https://www.redhat.com/archives/libguestfs/2020-October/msg00019.html v3: * defence -> defense * Use int64_t instead of int. This compiles on Windows. * Add GC wrappers to OCaml bindings. * New FreeBSD patch. * Removed "pid:" example from the ip filter manual, and added a warning beside the pid documentation. Rich.

This is just a simple update to: https://www.redhat.com/archives/libguestfs/2020-October/msg00015.html rebased on top of current nbdkit master because I pushed a few simple refactorings. Rich.

Here's changes to the file plugin (which I'm happy with) and a new exportname filter (which is still at RFC stage; I need to finish implementing strict mode in .open, and add tests). I also discovered that we really want .list_exports and .open to know when they are used on plaintext vs. tls clients for --tls=on, and we may want to split out a new .default_export callback rather than

I'm impressed that I was able to whip this out in just one day of hacking. Below, I'll include a diff between the plugin and the filter as of patch 1, if it aids review. Eric Blake (3): filters: Add ext2 filter ext2: Deprecate ext2 plugin ext2: Add mode for letting client exportname choose file from image TODO | 5 - configure.ac

Accepting only Unix sockets can be a bit limiting; let's be more flexible. Eric Blake (2): nbd: Refactor Unix socket connection nbd: Support TCP socket plugins/nbd/nbdkit-nbd-plugin.pod | 36 ++++-- plugins/nbd/nbd.c | 175 ++++++++++++++++++++++-------- TODO | 3 - 3 files changed, 161 insertions(+), 53 deletions(-) -- 2.20.1

On Mon, Sep 23, 2019 at 01:23:28PM -0500, Eric Blake wrote: > > Hopefully it will warn us if uid_t stops being int. (Also > > we're assuming pid_t == int). > > 64-bit mingw has had a long-standing bug that pid_t was declared as > 64-bit, even though getpid() returns a 32-bit 'int'; I wish they'd fix > their headers, but it violates the assumption of pid_t