search for: plugin_cleanup

There are several race conditions where nbdkit exits (calling plugin_cleanup() which unloads the plugin), while the plugin is either in a callback or in plugin.close(). Avoid these by: (1) Taking a shared lock around all plugin callbacks. (2) Taking the same as an exclusive lock in plugin_cleanup. This delays plugin_cleanup until all callbacks have finished. (3) Removi...

This fixes the race conditions for me, using the test described here: https://www.redhat.com/archives/libguestfs/2017-September/msg00226.html Rich.

...ent is currently silent with regards to issuing >> transmission-phase transactions, nbdkit hangs for 5 seconds, then >> forcibly tears down the connection: > > I'm guessing it's because of commit 63f0eb0889c8f8a82ba06a02a8a92d695902baad > which I added to fix a race in plugin_cleanup(). See also: > https://www.redhat.com/archives/libguestfs/2017-September/msg00226.html > > [...] >> Why does current traffic from the client cause the plugin to be torn >> down faster? Does it matter? > > I believe because the main loop checks the !quit flag if there...

...068204b 100644 --- a/src/internal.h +++ b/src/internal.h @@ -144,17 +144,13 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou /* plugins.c */ extern void plugin_register (const char *_filename, void *_dl, struct nbdkit_plugin *(*plugin_init) (void)); extern void plugin_cleanup (void); +extern int plugin_thread_model (void); extern const char *plugin_name (void); extern void plugin_usage (void); extern const char *plugin_version (void); extern void plugin_dump_fields (void); extern void plugin_config (const char *key, const char *value); extern void plugin_config_co...

...068204b 100644 --- a/src/internal.h +++ b/src/internal.h @@ -144,17 +144,13 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou /* plugins.c */ extern void plugin_register (const char *_filename, void *_dl, struct nbdkit_plugin *(*plugin_init) (void)); extern void plugin_cleanup (void); +extern int plugin_thread_model (void); extern const char *plugin_name (void); extern void plugin_usage (void); extern const char *plugin_version (void); extern void plugin_dump_fields (void); extern void plugin_config (const char *key, const char *value); extern void plugin_config_co...

I'm observing a difference in timing on nbdkit shutdown that is dependent on client behavior, and I wonder if that's a bug, but I can't figure out where to patch it. When there is no connection present, sending SIGINT to nbdkit terminates immediately. If, on the other hand, there is a single client currently connected, the termination behavior on SIGINT depends on what the client has

...objects. I found the bug. plugin_lock_request() is used to lock client requests only, but it does _not_ lock any other calls into the plugin. Due to that, the following calls are neither locked against each other nor are they locked against client requests: plugin_register (const char *_filename, plugin_cleanup (void) plugin_config (const char *key, const char *value) plugin_config_complete (void) plugin_open (struct connection *conn, int readonly) plugin_close (struct connection *conn) plugin_get_size (struct connection *conn) plugin_can_write (struct connection *conn) plugin_can_flush (struct connection...

v1 -> v2: - Fixed everything mentioned in the review. Rich.

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

...s done: if > the client is currently silent with regards to issuing > transmission-phase transactions, nbdkit hangs for 5 seconds, then > forcibly tears down the connection: I'm guessing it's because of commit 63f0eb0889c8f8a82ba06a02a8a92d695902baad which I added to fix a race in plugin_cleanup(). See also: https://www.redhat.com/archives/libguestfs/2017-September/msg00226.html [...] > Why does current traffic from the client cause the plugin to be torn > down faster? Does it matter? I believe because the main loop checks the !quit flag if there is traffic on the connection. Th...

...ify (for example) that plugin callbacks could or could not be killed in the middle. Killing them in the middle of a callback could be bad for some plugins. Going back to the original problem, perhaps we could have fixed it instead by acquiring a lock around all plugin callbacks, and ensuring that plugin_cleanup also acquires the same lock. The lock would normally be uncontended so it shouldn't cause any performance issues. That would be a fairly simple revert of 63f0eb088 + alternate fix. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming an...

...((cleanup (cleanup_free))) @@ -142,28 +144,31 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou #define debug nbdkit_debug /* plugins.c */ -extern void plugin_register (const char *_filename, void *_dl, struct nbdkit_plugin *(*plugin_init) (void)); -extern void plugin_cleanup (void); -extern int plugin_thread_model (void); -extern const char *plugin_name (void); -extern void plugin_usage (void); -extern const char *plugin_version (void); -extern void plugin_dump_fields (void); -extern void plugin_config (const char *key, const char *value); -extern void plugin_config_co...

...((cleanup (cleanup_free))) @@ -142,28 +150,31 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou #define debug nbdkit_debug /* plugins.c */ -extern void plugin_register (const char *_filename, void *_dl, struct nbdkit_plugin *(*plugin_init) (void)); -extern void plugin_cleanup (void); -extern int plugin_thread_model (void); -extern const char *plugin_name (void); -extern void plugin_usage (void); -extern const char *plugin_version (void); -extern void plugin_dump_fields (void); -extern void plugin_config (const char *key, const char *value); -extern void plugin_config_co...

...((cleanup (cleanup_free))) @@ -142,28 +150,31 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou #define debug nbdkit_debug /* plugins.c */ -extern void plugin_register (const char *_filename, void *_dl, struct nbdkit_plugin *(*plugin_init) (void)); -extern void plugin_cleanup (void); -extern int plugin_thread_model (void); -extern const char *plugin_name (void); -extern void plugin_usage (void); -extern const char *plugin_version (void); -extern void plugin_dump_fields (void); -extern void plugin_config (const char *key, const char *value); -extern void plugin_config_co...

These patches should be ready to go in now; I will also post my work-in-progress for enabling full parallel handling that depends on these, but with that series, I was still getting crashes or hangs with test-socket-activation (I think I've nailed all the crashes I've seen, but the hang is rather insidious; see my other email

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

On 26.09.2016 14:29, Richard W.M. Jones wrote: > On Mon, Sep 26, 2016 at 02:18:02PM +0200, Carl-Daniel Hailfinger wrote: >> Hi, >> >> has anyone ever run "make check" from nbd against nbdkit with a python >> plugin? I usually get segfaults during such a run, and sometimes various >> other errors happen before the segfault, suggesting that some memory

This patch isn't complete (patch 6/6 isn't finished) so it's just for discussion, although it does compile and run. This introduces to nbdkit a concept of "filters" which can be placed in front of plugins to modify their behaviour. Some examples where you might use filters: * Serve a subset of the data, such as (offset, range) or a single partition from a disk image.

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through