search for: open_plugin_so

v1 was here: https://www.redhat.com/archives/libguestfs/2019-November/msg00003.html This version depends on: https://www.redhat.com/archives/libguestfs/2019-November/msg00004.html and this series: https://www.redhat.com/archives/libguestfs/2019-November/msg00009.html The delta has been reduced slightly because of changes made possible by cleaning up and fixing the quit path in nbdkit. It's

libFuzzer is Clang's fuzzer, and alternative to using AFL: https://llvm.org/docs/LibFuzzer.html I implemented an alternative method of fuzzing for libnbd earlier today and it's pretty simple: https://github.com/libguestfs/libnbd/commit/c19a6fbae9a21a7d4693418706c59e81ed256875 However it's considerably more difficult to use libFuzzer with non-library code -- in this case nbdkit.

...lter_init) (void)); /* locks.c */ extern void lock_connection (void); diff --git a/src/main.c b/src/main.c index 4790c46..38691c9 100644 --- a/src/main.c +++ b/src/main.c @@ -64,7 +64,8 @@ static int is_short_name (const char *); static char *make_random_fifo (void); -static struct backend *open_plugin_so (const char *filename, int short_name); +static struct backend *open_plugin_so (size_t i, const char *filename, int short_name); +static struct backend *open_filter_so (struct backend *next, size_t i, const char *filename, int short_name); static void start_serving (void); static void set_up_sign...

...main (int argc, char *argv[]) { @@ -178,6 +222,9 @@ main (int argc, char *argv[]) perror ("expecting stdin/stdout to be opened"); exit (EXIT_FAILURE); } +#ifdef WINDOWS_COMPAT + init_functions (); +#endif #if !ENABLE_LIBFUZZER threadlocal_init (); @@ -772,6 +819,7 @@ open_plugin_so (size_t i, const char *name, int short_name) bool free_filename = false; void *dl; struct nbdkit_plugin *(*plugin_init) (void); + void *(*functions_init) (struct nbdkit_functions *); char *error; if (short_name) { @@ -797,6 +845,18 @@ open_plugin_so (size_t i, const char *name, in...

...load prevention lock"); pthread_rwlock_unlock (&unload_prevention_lock); diff --git a/src/main.c b/src/main.c index 4eca859..b3e6bad 100644 --- a/src/main.c +++ b/src/main.c @@ -64,7 +64,7 @@ static int is_short_name (const char *); static char *make_random_fifo (void); -static void open_plugin_so (const char *filename, int short_name); +static struct backend *open_plugin_so (const char *filename, int short_name); static void start_serving (void); static void set_up_signals (void); static void run_command (void); @@ -103,6 +103,9 @@ volatile int quit; int quit_fd; static int write_quit_...

...load prevention lock"); pthread_rwlock_unlock (&unload_prevention_lock); diff --git a/src/main.c b/src/main.c index 4eca859..b3e6bad 100644 --- a/src/main.c +++ b/src/main.c @@ -64,7 +64,7 @@ static int is_short_name (const char *); static char *make_random_fifo (void); -static void open_plugin_so (const char *filename, int short_name); +static struct backend *open_plugin_so (const char *filename, int short_name); static void start_serving (void); static void set_up_signals (void); static void run_command (void); @@ -103,6 +103,9 @@ volatile int quit; int quit_fd; static int write_quit_...

...load prevention lock"); pthread_rwlock_unlock (&unload_prevention_lock); diff --git a/src/main.c b/src/main.c index 4eca859..b3e6bad 100644 --- a/src/main.c +++ b/src/main.c @@ -64,7 +64,7 @@ static int is_short_name (const char *); static char *make_random_fifo (void); -static void open_plugin_so (const char *filename, int short_name); +static struct backend *open_plugin_so (const char *filename, int short_name); static void start_serving (void); static void set_up_signals (void); static void run_command (void); @@ -103,6 +103,9 @@ volatile int quit; int quit_fd; static int write_quit_...

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

...x7928FEE: ??? ==14189== by 0x784662B: ??? ==14189== by 0x79329BA: ??? ==14189== by 0x78BFAC0: ??? ==14189== by 0x78C59C5: ??? ==14189== by 0x7841C4B: ??? ==14189== by 0x75FA074: ??? ==14189== by 0x40A512: plugin_register (plugins.c:749) ==14189== by 0x404436: open_plugin_so (main.c:740) ==14189== by 0x404436: main (main.c:565) We can improve this using the valgrind macro RUNNING_ON_VALGRIND to prevent calling dlclose(3). I have made this opt-in since we don't generally want to make production nbdkit binaries which rely on probing valgrind. Developers have...

v1 -> v2: - Fixed everything mentioned in the review. Rich.

...it) (void)); /* locks.c */ extern void lock_init_thread_model (void); diff --git a/src/main.c b/src/main.c index 90d464a..29332c4 100644 --- a/src/main.c +++ b/src/main.c @@ -64,7 +64,8 @@ static int is_short_name (const char *); static char *make_random_fifo (void); -static struct backend *open_plugin_so (const char *filename, int short_name); +static struct backend *open_plugin_so (size_t i, const char *filename, int short_name); +static struct backend *open_filter_so (struct backend *next, size_t i, const char *filename, int short_name); static void start_serving (void); static void set_up_sign...

...it) (void)); /* locks.c */ extern void lock_init_thread_model (void); diff --git a/src/main.c b/src/main.c index 90d464a..29332c4 100644 --- a/src/main.c +++ b/src/main.c @@ -64,7 +64,8 @@ static int is_short_name (const char *); static char *make_random_fifo (void); -static struct backend *open_plugin_so (const char *filename, int short_name); +static struct backend *open_plugin_so (size_t i, const char *filename, int short_name); +static struct backend *open_filter_so (struct backend *next, size_t i, const char *filename, int short_name); static void start_serving (void); static void set_up_sign...

This patch isn't complete (patch 6/6 isn't finished) so it's just for discussion, although it does compile and run. This introduces to nbdkit a concept of "filters" which can be placed in front of plugins to modify their behaviour. Some examples where you might use filters: * Serve a subset of the data, such as (offset, range) or a single partition from a disk image.

..."; /* --tls=require and oldstyle won't work. */ - if (tls == 2 && newstyle == 0) { + if (tls == 2 && !newstyle) { fprintf (stderr, "%s: cannot use oldstyle protocol (-o) and require TLS\n", program_name); @@ -727,7 +728,7 @@ open_plugin_so (size_t i, const char *name, int short_name) { struct backend *ret; char *filename = (char *) name; - int free_filename = 0; + bool free_filename = false; void *dl; struct nbdkit_plugin *(*plugin_init) (void); char *error; @@ -739,7 +740,7 @@ open_plugin_so (size_t i, const char *...

...el (backend); - debug ("release unload prevention lock"); pthread_rwlock_unlock (&unload_prevention_lock); diff --git a/src/main.c b/src/main.c index b3e6bad..90d464a 100644 --- a/src/main.c +++ b/src/main.c @@ -497,6 +497,7 @@ main (int argc, char *argv[]) } backend = open_plugin_so (filename, short_name); + lock_init_thread_model (); if (help) { usage (); -- 2.15.1

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

I pushed a few of the msys2 patches upstream. I changed the way that $(SHARED_LDFLAGS) works so it's more to my liking, and the others were pushed unchanged. Three patches remain which I'm posting on the mailing list for proper review. Rich.

...ugin --dump-plugin' or\n" - "'nbdkit /path/to/plugin.so --dump-plugin'\n", + "'nbdkit /path/to/plugin." SOEXT " --dump-plugin'\n", program_name); exit (EXIT_FAILURE); } @@ -819,7 +819,7 @@ open_plugin_so (size_t i, const char *name, int short_name) if (short_name) { /* Short names are rewritten relative to the plugindir. */ if (asprintf (&filename, - "%s/nbdkit-%s-plugin.so", plugindir, name) == -1) { + "%s/nbdkit-%s-plugin." SO...

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through

This is still tentative and needs a lot of work, but: - partition filter works, supporting MBR & GPT - prepare and finalize methods fixed - open method can now be changed (allowing readonly flag to be modified) - thread_model can be limited I believe I made most of the changes which were previously suggested in email. I think the only one I didn't was preventing inclusion of both