search for: nbdkit_next_preconnect

Eric: Did you want to take this one any further? It might be one that we save for > 1.18: https://www.redhat.com/archives/libguestfs/2020-February/thread.html#00206 Another thing I've been thinking about for some time is splitting .config_complete into .config_complete + .get_ready (new name TBD). At the moment .config_complete is both the place where we finish processing config, and

.../nbdkit-filter.pod index 4105b8b7..a9dffb56 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -127,22 +127,24 @@ which is required. =head1 NEXT PLUGIN F<nbdkit-filter.h> defines some function types (C<nbdkit_next_config>, -C<nbdkit_next_config_complete>, C<nbdkit_next_preconnect>, -C<nbdkit_next_open>) and a structure called C<struct nbdkit_next_ops>. -These abstract the next plugin or filter in the chain. There is also -an opaque pointer C<nxdata> which must be passed along when calling -these functions. The value of C<nxdata> passed to C<....

...kend; +#else +typedef void backend; +#endif + /* Next ops. */ -typedef int nbdkit_next_config (void *nxdata, +typedef int nbdkit_next_config (backend *nxdata, const char *key, const char *value); -typedef int nbdkit_next_config_complete (void *nxdata); -typedef int nbdkit_next_preconnect (void *nxdata, int readonly); -typedef int nbdkit_next_open (void *nxdata, int readonly); +typedef int nbdkit_next_config_complete (backend *nxdata); +typedef int nbdkit_next_preconnect (backend *nxdata, int readonly); +typedef int nbdkit_next_open (backend *nxdata, int readonly); struct nbdkit_n...

I like this change. I think we were overloading the config_complete method before to do two different things (complete configuration; do any allocation/housekeeping necessary before we can start serving). The only questions in my mind are whether we want this before 1.18, and whether the name ("get_ready") is a good one. Rich.

...r.pod index 00f8e70d..510781e1 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -128,23 +128,23 @@ which is required. F<nbdkit-filter.h> defines some function types (C<nbdkit_next_config>, C<nbdkit_next_config_complete>, C<nbdkit_next_get_ready>, -C<nbdkit_next_preconnect>, C<nbdkit_next_open>) and a structure called -C<struct nbdkit_next_ops>. These abstract the next plugin or filter -in the chain. There is also an opaque pointer C<nxdata> which must be -passed along when calling these functions. The value of C<nxdata> -passed to C<...

...el) (void); - int (*get_ready) (nbdkit_next_get_ready *next, nbdkit_backend *nxdata); + int (*get_ready) (nbdkit_next_get_ready *next, nbdkit_backend *nxdata, + int thread_model); int (*after_fork) (nbdkit_next_after_fork *next, nbdkit_backend *nxdata); int (*preconnect) (nbdkit_next_preconnect *next, nbdkit_backend *nxdata, int readonly); diff --git a/server/filters.c b/server/filters.c index 90a9a948..0cfae344 100644 --- a/server/filters.c +++ b/server/filters.c @@ -183,10 +183,10 @@ filter_get_ready (struct backend *b) { struct backend_filter *f = container_of...

...connect stage (in particular before > + * any heavyweight NBD or TLS negotiations has been done). However we > + * count connections in the open/close calls since clients can drop > + * out between preconnect and open. > + */ Seems reasonable. > +static int > +limit_preconnect (nbdkit_next_preconnect *next, nbdkit_backend *nxdata, > + int readonly) > +{ > + if (next (nxdata, readonly) == -1) > + return -1; > + > + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lock); > + > + if (limit > 0 && connections >= limit) { > + too_many_clients_er...

This is a second version of the limit filter. v1 was posted here: https://www.redhat.com/archives/libguestfs/2020-March/msg00015.html I didn't bother to repost the other three patches because they are the same. The difference is this version of the filter takes security more seriously. It shouldn't be possible for malicious clients to connect more than limit=N times to the plugin now,

Since v1: - patch 1: check size limits - patch 2: better handling of default export name canonicalization - patch 3: support filters as well as plugins - patch 4: new - patch 5: rewrite sh parser, fix testsuite to actually work and cover more cases (now that libnbd.git is fixed) Eric Blake (4): server: Add exports list functions server: Prepare to use export list from plugin log: Add

.....cec12db7 100644 --- a/include/nbdkit-filter.h +++ b/include/nbdkit-filter.h @@ -65,13 +65,14 @@ typedef int nbdkit_next_config_complete (nbdkit_backend *nxdata); typedef int nbdkit_next_get_ready (nbdkit_backend *nxdata); typedef int nbdkit_next_after_fork (nbdkit_backend *nxdata); typedef int nbdkit_next_preconnect (nbdkit_backend *nxdata, int readonly); -typedef int nbdkit_next_open (nbdkit_backend *nxdata, int readonly); +typedef int nbdkit_next_open (nbdkit_backend *nxdata, + int readonly, const char *exportname); struct nbdkit_next_ops { /* Performs close + open on the u...

...*get_ready) (nbdkit_next_get_ready *next, nbdkit_backend *nxdata); > + int (*get_ready) (nbdkit_next_get_ready *next, nbdkit_backend *nxdata, > + int thread_model); > int (*after_fork) (nbdkit_next_after_fork *next, nbdkit_backend *nxdata); > int (*preconnect) (nbdkit_next_preconnect *next, nbdkit_backend *nxdata, > int readonly); > diff --git a/server/filters.c b/server/filters.c > index 90a9a948..0cfae344 100644 > --- a/server/filters.c > +++ b/server/filters.c > @@ -183,10 +183,10 @@ filter_get_ready (struct backend *b) > { >...

...d"); +} + +/* We limit connections in the preconnect stage (in particular before + * any heavyweight NBD or TLS negotiations has been done). However we + * count connections in the open/close calls since clients can drop + * out between preconnect and open. + */ +static int +limit_preconnect (nbdkit_next_preconnect *next, nbdkit_backend *nxdata, + int readonly) +{ + if (next (nxdata, readonly) == -1) + return -1; + + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lock); + + if (limit > 0 && connections >= limit) { + too_many_clients_error (); + return -1; + } + + return 0; +...

Patch 3 still needs tests added, but it is at least working from my simple command line tests. Eric Blake (3): server: Implement nbdkit_is_tls for use during .open server: Expose final thread_model to filter's .get_ready tlsdummy: New filter docs/nbdkit-filter.pod | 21 +- docs/nbdkit-plugin.pod | 34 ++- docs/nbdkit-tls.pod

...++++++++++++++++++--------------- 2 files changed, 96 insertions(+), 52 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index 55dfab1..5fed7ca 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -131,7 +131,12 @@ C<nbdkit_next_config_complete>, C<nbdkit_next_preconnect>, C<nbdkit_next_open>) and a structure called C<struct nbdkit_next_ops>. These abstract the next plugin or filter in the chain. There is also an opaque pointer C<nxdata> which must be passed along when calling -these functions. +these functions. The value of C<nxdata&gt...

...here's an implicit allow all for non-IP, non-Unix sockets, + * see the manual. + */ + if (family != AF_INET && family != AF_INET6 && family != AF_UNIX) return true; if (matches_rules_list ("ip: match source with allow", @@ -457,7 +514,7 @@ ip_preconnect (nbdkit_next_preconnect *next, void *nxdata, int readonly) /* Follow the rules. */ if (check_if_allowed ((struct sockaddr *) &addr) == false) { nbdkit_error ("client not permitted to connect " - "because of IP address restriction"); + "because of so...

...t a/include/nbdkit-filter.h b/include/nbdkit-filter.h index b4024ae5..2c5b36be 100644 --- a/include/nbdkit-filter.h +++ b/include/nbdkit-filter.h @@ -66,8 +66,10 @@ typedef int nbdkit_next_get_ready (nbdkit_backend *nxdata); typedef int nbdkit_next_after_fork (nbdkit_backend *nxdata); typedef int nbdkit_next_preconnect (nbdkit_backend *nxdata, int readonly); typedef int nbdkit_next_list_exports (nbdkit_backend *nxdata, int readonly, - int default_only, + int ignored, struct nbdkit_exports *exports); +...

This incomplete patch adds a new filter allowing more control over when nbdkit exits. You can now get nbdkit to exit gracefully on certain events, such as a file being created, a pipe held open by another process going away, or when another PID exits. There is also a script option to allow for completely custom events. It is untested at the moment, I'm posting it to get feedback on the

...e5..f059606b 100644 --- a/include/nbdkit-filter.h +++ b/include/nbdkit-filter.h @@ -65,6 +65,7 @@ typedef int nbdkit_next_config_complete (nbdkit_backend *nxdata); typedef int nbdkit_next_get_ready (nbdkit_backend *nxdata); typedef int nbdkit_next_after_fork (nbdkit_backend *nxdata); typedef int nbdkit_next_preconnect (nbdkit_backend *nxdata, int readonly); +struct nbdkit_exports; typedef int nbdkit_next_list_exports (nbdkit_backend *nxdata, int readonly, int default_only, struct nbdkit_exports *exports); @@ -130,6 +131,10 @@ NBDKIT_E...

...int is_tls) { /* Always pass readonly=1 to the underlying plugin. */ if (next (nxdata, 1, exportname) == -1) diff --git a/filters/limit/limit.c b/filters/limit/limit.c index 7c4477eb..fb862df7 100644 --- a/filters/limit/limit.c +++ b/filters/limit/limit.c @@ -91,7 +91,7 @@ limit_preconnect (nbdkit_next_preconnect *next, nbdkit_backend *nxdata, static void * limit_open (nbdkit_next_open *next, nbdkit_backend *nxdata, - int readonly, const char *exportname) + int readonly, const char *exportname, int is_tls) { if (next (nxdata, readonly, exportname) == -1) return NULL; diff -...

If you're following nbdkit development upstream you'll have seen that we are still making changes to the .list_exports and related APIs. The current .list_exports API upstream is not how it will look finally. The latest set of proposals was here: https://www.redhat.com/archives/libguestfs/2020-August/thread.html#00330 At the same time I'd like to do an nbdkit 1.22 (stable) release.