search for: serialize_connections

Rich pointed out that making thread_model dynamic even for plugins makes some sense, so here is the code for doing it for 'sh'. I'm less confident on how to do it for OCaml and Rust (not to mention that those allow the plugin to already compile in their model, rather than the language binding glue forcing a model). The other languages (lua, perl, python, ruby) still need to be

...ate mode 100755 tests/test-parallel-sh.sh > > diff --git a/plugins/sh/nbdkit-sh-plugin.pod b/plugins/sh/nbdkit-sh-plugin.pod > index 41b9e7c7..7c1781cd 100644 > --- a/plugins/sh/nbdkit-sh-plugin.pod > +++ b/plugins/sh/nbdkit-sh-plugin.pod > @@ -189,12 +189,11 @@ one of C<"serialize_connections">, C<"serialize_all_requests">, > C<"serialize_requests">, or C<"parallel">. > > This method is I<not> required; if omitted, then the plugin will be > -executed under the default sh thread model (currently > -C<&quot...

...tions(+), 12 deletions(-) create mode 100755 tests/test-parallel-sh.sh diff --git a/plugins/sh/nbdkit-sh-plugin.pod b/plugins/sh/nbdkit-sh-plugin.pod index 41b9e7c7..7c1781cd 100644 --- a/plugins/sh/nbdkit-sh-plugin.pod +++ b/plugins/sh/nbdkit-sh-plugin.pod @@ -189,12 +189,11 @@ one of C<"serialize_connections">, C<"serialize_all_requests">, C<"serialize_requests">, or C<"parallel">. This method is I<not> required; if omitted, then the plugin will be -executed under the default sh thread model (currently -C<"serialize_all_requests&q...

On 8/2/19 2:55 PM, Richard W.M. Jones wrote: >> +++ b/plugins/sh/nbdkit-sh-plugin.pod >> @@ -189,12 +189,11 @@ one of C<"serialize_connections">, C<"serialize_all_requests">, >> C<"serialize_requests">, or C<"parallel">. >> >> This method is I<not> required; if omitted, then the plugin will be >> -executed under the default sh thread model (currently &...

On Fri, Aug 07, 2020 at 05:00:52PM -0500, Eric Blake wrote: > The next patch wants to add a filter that will prevent DoS attacks > from a plaintext client; to be successful, the filter must guarantee > that nbdkit did not settle on SERIALIZE_CONNECTIONS. The easiest way > to solve this is to expose the final thread model to .get_ready, which > is after the point where .config_complete may have altered it, and > before any connections are permitted. > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- > docs/nbdkit-...

This is a major rewrite compared to my v1 series, where I've tried a lot harder to ensure that we still accommodate building on Haiku (although I have not actually yet fired up a Haiku VM to try it for myself). I also managed to make the sh plugin fully parallel, on capable platforms. See also my question on patch 10 on whether I've picked the best naming convention. Eric Blake (17):

.../10/20 8:01 AM, Richard W.M. Jones wrote: > On Fri, Aug 07, 2020 at 05:00:52PM -0500, Eric Blake wrote: >> The next patch wants to add a filter that will prevent DoS attacks >> from a plaintext client; to be successful, the filter must guarantee >> that nbdkit did not settle on SERIALIZE_CONNECTIONS. The easiest way >> to solve this is to expose the final thread model to .get_ready, which >> is after the point where .config_complete may have altered it, and >> before any connections are permitted. >> >> Signed-off-by: Eric Blake <eblake@redhat.com> >>...

...w for sure whether it is answering the question on a read-only connection vs. on a read-write connection, as the answer may differ between those two? Should we automatically set the bit for any plugin that has fully-serialized operation but does not provide the callback? That is, if a plugin uses SERIALIZE_CONNECTIONS, you can never have multiple clients at the same time anyways (so the flag is trivially ignorable - a client can try to set up a second connection for speed, but it won't help). A plugin that uses SERIALIZE_ALL_REQUESTS is also trivially supported - any flush request will complete (regardless...

...Richard W.M. Jones wrote: > >On Fri, Aug 07, 2020 at 05:00:52PM -0500, Eric Blake wrote: > >>The next patch wants to add a filter that will prevent DoS attacks > >>from a plaintext client; to be successful, the filter must guarantee > >>that nbdkit did not settle on SERIALIZE_CONNECTIONS. The easiest way > >>to solve this is to expose the final thread model to .get_ready, which > >>is after the point where .config_complete may have altered it, and > >>before any connections are permitted. > >> > >>Signed-off-by: Eric Blake <eblake@re...

The next patch wants to add a filter that will prevent DoS attacks from a plaintext client; to be successful, the filter must guarantee that nbdkit did not settle on SERIALIZE_CONNECTIONS. The easiest way to solve this is to expose the final thread model to .get_ready, which is after the point where .config_complete may have altered it, and before any connections are permitted. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-filter.pod | 9 ++++++++-...

...casecmp (s, "serialize-all-requests") == 0) + else if (ascii_strcasecmp (s, "serialize_all_requests") == 0 || + ascii_strcasecmp (s, "serialize-all-requests") == 0) r = NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS; - else if (strcasecmp (s, "serialize_connections") == 0 || - strcasecmp (s, "serialize-connections") == 0) + else if (ascii_strcasecmp (s, "serialize_connections") == 0 || + ascii_strcasecmp (s, "serialize-connections") == 0) r = NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS; e...

...on a read-write connection, as the answer may > differ between those two? Yes, I think this is a good idea. Will try that for v2. > Should we automatically set the bit for any plugin that has > fully-serialized operation but does not provide the callback? That is, > if a plugin uses SERIALIZE_CONNECTIONS, you can never have multiple > clients at the same time anyways (so the flag is trivially ignorable - a > client can try to set up a second connection for speed, but it won't > help). A plugin that uses SERIALIZE_ALL_REQUESTS is also trivially > supported - any flush request will c...

First thing to say is that I need to do a *lot* more testing on this, so this is just an early peek. In particular, although it passed ‘make check && make check-valgrind’ I have *not* tested it against a multi-conn-aware client such as the Linux kernel >= 4.9. This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc as: "NBD_FLAG_CAN_MULTI_CONN: Indicates that

...he command line, to reduce the chance that any work done by +C<.open> in later filters or the plugin can be exploited by plaintext +connections as a denial of service attack to starve further +authenticated connections. Note that this plugin will fail to load if +the plugin requests the C<SERIALIZE_CONNECTIONS> thread model, as a +plaintext client holding its connection open indefinitely would be +such a starvation. + +=head1 PARAMETERS + +=over 4 + +=item B<tlsreadme=>MESSAGE + +This optional parameter can be used to use C<MESSAGE> as the contents +of the dummy export exposed to plaintext...

Patch 3 still needs tests added, but it is at least working from my simple command line tests. Eric Blake (3): server: Implement nbdkit_is_tls for use during .open server: Expose final thread_model to filter's .get_ready tlsdummy: New filter docs/nbdkit-filter.pod | 21 +- docs/nbdkit-plugin.pod | 34 ++- docs/nbdkit-tls.pod

On Sat, Feb 22, 2020 at 05:11:01AM -0600, Eric Blake wrote: > On 2/22/20 4:37 AM, Richard W.M. Jones wrote: > >Another thing I've been thinking about for some time is splitting > >.config_complete into .config_complete + .get_ready (new name TBD). > >At the moment .config_complete is both the place where we finish > >processing config, and also the last chance we get

...t-plugin(3)>): + + nbdkit ./libmyplugin.so [args ...] + +=head2 Threads + +The first parameter of C<Plugin::new> is the thread model, which can +be one of the values in the table below. For more information on +thread models, see L<nbdkit-plugin(3)/THREADS>. + +=over 4 + +=item C<Serialize_Connections> + +=item C<Serialize_All_Requests> + +=item C<Serialize_Requests> + +=item C<Parallel> + +=back + +=head1 SEE ALSO + +L<nbdkit(1)>, +L<nbdkit-plugin(3)>, +L<cargo(1)>. + +=head1 AUTHORS + +Richard W.M. Jones + +=head1 COPYRIGHT + +Copyright (C) 2019 Red Hat I...

This adds very rough support for writing nbdkit plugins in Rust. This is not very idiomatic -- essentially we're handling the direct C calls from nbdkit in Rust. We have to use ‘unsafe’ in a few places because there's no way to tell the Rust code that nbdkit satisfies guarantees (eg. around thread safety, always returning leaked pointers back to the close function, always doing bounds

...c +++ b/src/locks.c @@ -45,7 +45,7 @@ static pthread_rwlock_t unload_prevention_lock = PTHREAD_RWLOCK_INITIALIZER; void lock_connection (void) { - int thread_model = plugin_thread_model (); + int thread_model = backend->thread_model (backend); if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS) { debug ("acquire connection lock"); @@ -56,7 +56,7 @@ lock_connection (void) void unlock_connection (void) { - int thread_model = plugin_thread_model (); + int thread_model = backend->thread_model (backend); if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTI...

...c +++ b/src/locks.c @@ -45,7 +45,7 @@ static pthread_rwlock_t unload_prevention_lock = PTHREAD_RWLOCK_INITIALIZER; void lock_connection (void) { - int thread_model = plugin_thread_model (); + int thread_model = backend->thread_model (backend); if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS) { debug ("acquire connection lock"); @@ -56,7 +56,7 @@ lock_connection (void) void unlock_connection (void) { - int thread_model = plugin_thread_model (); + int thread_model = backend->thread_model (backend); if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTI...