search for: nbdkit_thread_model_parallel

Displaying 20 results from an estimated 173 matches for "nbdkit_thread_model_parallel".

2019 May 17
0
[nbdkit PATCH 3/3] filters: Use only .thread_model, not THREAD_MODEL
...ertions(+), 68 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index 4033789..86894f1 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -6,8 +6,6 @@ nbdkit-filter - how to write nbdkit filters #include <nbdkit-filter.h> - #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL - static int myfilter_config (nbdkit_next_config *next, void *nxdata, const char *key, const char *value) @@ -107,24 +105,6 @@ should not expect to distribute filters separately from nbdkit. All filters should start by including this header file. -=head1 C<#define THRE...
2019 May 17
4
[nbdkit PATCH 0/3] Add noparallel filter
Being able to programmatically force nbdkit to be less parallel can be useful during testing. I was less sure about patch 3, but if you like it, I'm inclined to instead squash it into patch 1. This patch is written to apply after my NBD_CMD_CACHE work (since I touched the nocache filter); but can be rearranged if we think this series should go in first while that one undergoes any adjustments
2019 Jan 01
0
[PATCH nbdkit] plugins, filters: Define and use NBDKIT_HANDLE_NOT_NEEDED.
...@ -110,12 +110,7 @@ floppy_config_complete (void) static void * floppy_open (int readonly) { - /* We don't need a per-connection handle, so this just acts as a - * pointer to return. - */ - static int h; - - return &h; + return NBDKIT_HANDLE_NOT_NEEDED; } #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL diff --git a/plugins/full/full.c b/plugins/full/full.c index 3f5159f..cb18a48 100644 --- a/plugins/full/full.c +++ b/plugins/full/full.c @@ -83,9 +83,7 @@ full_config_complete (void) static void * full_open (int readonly) { - static int handle; - - return &handle; + return NBDKIT_HANDLE_N...
2018 Dec 31
1
Re: [PATCH nbdkit 5/9] cache: Allow this filter to serve requests in parallel.
On 12/28/18 12:45 PM, Richard W.M. Jones wrote: > Make the implicit lock explicit, and hold it around blk_* operations. > This allows us to relax the thread model for the filter to > NBDKIT_THREAD_MODEL_PARALLEL. > --- > filters/cache/blk.h | 7 ++++++ > filters/cache/cache.c | 57 +++++++++++++++++++++++++++++++------------ > 2 files changed, 49 insertions(+), 15 deletions(-) Nice. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.o...
2019 Nov 22
2
Re: [PATCH nbdkit v2 02/10] python: Add various constants to the API.
...IT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS); > + PyModule_AddIntConstant (m, "THREAD_MODEL_SERIALIZE_REQUESTS", > + NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS); > + PyModule_AddIntConstant (m, "THREAD_MODEL_PARALLEL", > + NBDKIT_THREAD_MODEL_PARALLEL); > + > + PyModule_AddIntConstant (m, "FLAG_MAY_TRIM", NBDKIT_FLAG_MAY_TRIM); > + PyModule_AddIntConstant (m, "FLAG_FUA", NBDKIT_FLAG_FUA); > + PyModule_AddIntConstant (m, "FLAG_REQ_ONE", NBDKIT_FLAG_REQ_ONE); > + PyModule_AddIntConstant (m,...
2017 Nov 17
0
[nbdkit PATCH 6/6] Add --threads option for supporting true parallel requests
...(+), 15 deletions(-) diff --git a/TODO b/TODO index 6c5bb5b..db7469b 100644 --- a/TODO +++ b/TODO @@ -12,10 +12,3 @@ * Glance and/or cinder plugins. * Performance - measure and improve it. - -* Implement true parallel request handling. Currently - NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS and - NBDKIT_THREAD_MODEL_PARALLEL are the same, because we handle - requests within each connection synchronously one at a time. We - could (and should) be able to handle them in parallel by having - another thread pool for requests. diff --git a/docs/nbdkit.pod b/docs/nbdkit.pod index e3043ba..4593391 100644 --- a/docs/nbdkit....
2019 Apr 23
0
[nbdkit PATCH 3/4] filters: Utilize ACQUIRE_LOCK_FOR_CURRENT_SCOPE
...5 ++++- 4 files changed, 17 insertions(+), 18 deletions(-) diff --git a/filters/log/log.c b/filters/log/log.c index 02a2522..513d390 100644 --- a/filters/log/log.c +++ b/filters/log/log.c @@ -45,6 +45,8 @@ #include <nbdkit-filter.h> +#include "cleanup.h" + #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL static uint64_t connections; @@ -114,12 +116,8 @@ struct handle { static uint64_t get_id (struct handle *h) { - uint64_t r; - - pthread_mutex_lock (&lock); - r = ++h->id; - pthread_mutex_unlock (&lock); - return r; + ACQUIRE_LOCK_FOR_CURRENT_SCOPE(&lock); + return ++h-&gt...
2019 Apr 24
1
Re: [nbdkit PATCH 4/4] filters: Check for mutex failures
...to squash in: diff --git i/filters/error/error.c w/filters/error/error.c index 22ebd1c..add7566 100644 --- i/filters/error/error.c +++ w/filters/error/error.c @@ -45,6 +45,7 @@ #include <nbdkit-filter.h> +#include "cleanup.h" #include "random.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL (Oddly enough, without that change, 'make' succeeds with status 0, merely warning about an implicit symbol declaration rather than failing the build, where it wasn't until 'make check' failing with .so failing to find all symbols that I noticed the problem. That's a bit dis...
2018 Dec 28
0
[PATCH nbdkit 5/9] cache: Allow this filter to serve requests in parallel.
Make the implicit lock explicit, and hold it around blk_* operations. This allows us to relax the thread model for the filter to NBDKIT_THREAD_MODEL_PARALLEL. --- filters/cache/blk.h | 7 ++++++ filters/cache/cache.c | 57 +++++++++++++++++++++++++++++++------------ 2 files changed, 49 insertions(+), 15 deletions(-) diff --git a/filters/cache/blk.h b/filters/cache/blk.h index 24bf6a1..ab9134e 100644 --- a/filters/cache/blk.h +++ b/filters/cache/blk...
2018 Aug 01
1
Re: [PATCH v2 nbdkit 5/6] Add truncate filter for truncating or extending the size of plugins.
...d, 459 insertions(+), 3 deletions(-) > > + > +=item * > + > +Round the size of the plugin up or down to the next multiple of C<N>. > +Use either C<round-up=N> or C<round-down=N>. Worth mentioning that N must be a power of 2. > + > +#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL > + > +/* These are the parameters. */ > +static int64_t truncate = -1; > +static unsigned round_up = 0, round_down = 0; > + > +/* The real size of the underlying plugin. */ > +static uint64_t real_size; > + > +/* The calculated size after applying the parameters. */ >...
2019 Mar 23
1
Re: [PATCH nbdkit 8/8] file: Implement extents.
...ected by this lock. */ > +static pthread_mutex_t lseek_lock = PTHREAD_MUTEX_INITIALIZER; > + > +/* to enable: -D file.zero=1 */ > +int file_debug_zero; > > static void > file_unload (void) > @@ -220,6 +227,21 @@ file_close (void *handle) > > #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL > > +/* For block devices, stat->st_size is not the true size. */ > +static int64_t > +block_device_size (int fd) > +{ > + off_t size; > + > + size = lseek (fd, 0, SEEK_END); Calling lseek without the lock? I'm not sure if you can guarantee that .size won't b...
2018 Sep 17
2
[PATCH nbdkit v2] common: Introduce round up, down; and divide round
Since we're using ({ .. }) gcc/clang extension, let's rewrite the rounding.h change too. Rich.
2019 Nov 22
1
Re: [PATCH nbdkit v2 02/10] python: Add various constants to the API.
...> > > + PyModule_AddIntConstant (m, "THREAD_MODEL_SERIALIZE_REQUESTS", > > > + NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS); > > > + PyModule_AddIntConstant (m, "THREAD_MODEL_PARALLEL", > > > + NBDKIT_THREAD_MODEL_PARALLEL); > > > + > > > + PyModule_AddIntConstant (m, "FLAG_MAY_TRIM", NBDKIT_FLAG_MAY_TRIM); > > > + PyModule_AddIntConstant (m, "FLAG_FUA", NBDKIT_FLAG_FUA); > > > + PyModule_AddIntConstant (m, "FLAG_REQ_ONE", NBDKIT_FLAG_REQ_O...
2018 Jan 17
0
[PATCH 1/9] plugins: Move locking to a new file.
...--- a/src/connections.c +++ b/src/connections.c @@ -211,7 +211,7 @@ _handle_single_connection (int sockin, int sockout) int nworkers = threads ? threads : DEFAULT_PARALLEL_REQUESTS; pthread_t *workers = NULL; - if (!plugin_is_parallel() || nworkers == 1) + if (plugin_thread_model () < NBDKIT_THREAD_MODEL_PARALLEL || nworkers == 1) nworkers = 0; conn = new_connection (sockin, sockout, nworkers); if (!conn) @@ -287,9 +287,9 @@ handle_single_connection (int sockin, int sockout) { int r; - plugin_lock_connection (); + lock_connection (); r = _handle_single_connection (sockin, sockout); -...
2018 Jan 16
0
[PATCH nbdkit 1/3] plugins: Move locking to a new file.
...--- a/src/connections.c +++ b/src/connections.c @@ -211,7 +211,7 @@ _handle_single_connection (int sockin, int sockout) int nworkers = threads ? threads : DEFAULT_PARALLEL_REQUESTS; pthread_t *workers = NULL; - if (!plugin_is_parallel() || nworkers == 1) + if (plugin_thread_model () < NBDKIT_THREAD_MODEL_PARALLEL || nworkers == 1) nworkers = 0; conn = new_connection (sockin, sockout, nworkers); if (!conn) @@ -287,9 +287,9 @@ handle_single_connection (int sockin, int sockout) { int r; - plugin_lock_connection (); + lock_connection (); r = _handle_single_connection (sockin, sockout); -...
2019 May 16
0
[nbdkit PATCH 2/2] cache, cow: Round size down
...ters/cache/cache.c b/filters/cache/cache.c index 360458f..e215cac 100644 --- a/filters/cache/cache.c +++ b/filters/cache/cache.c @@ -60,6 +60,7 @@ #include "reclaim.h" #include "isaligned.h" #include "minmax.h" +#include "rounding.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL @@ -188,10 +189,12 @@ cache_config_complete (nbdkit_next_config_complete *next, void *nxdata) return next (nxdata); } -/* Get the file size and ensure the cache is the correct size. */ +/* Get the file size; round it down to cache granularity before + * setting cache size. + */ static int64_...
2019 May 11
2
[nbdkit PATCH] cache: Reduce use of bounce-buffer
...ibution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are @@ -58,6 +58,7 @@ #include "cache.h" #include "blk.h" #include "reclaim.h" +#include "isaligned.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL @@ -233,11 +234,13 @@ cache_pread (struct nbdkit_next_ops *next_ops, void *nxdata, CLEANUP_FREE uint8_t *block = NULL; assert (!flags); - block = malloc (blksize); - if (block == NULL) { - *err = errno; - nbdkit_error ("malloc: %m"); - return -1; + if (!IS_ALIGNED (cou...
2017 Feb 20
1
Re: Fwd: nbdkit async
The concern is a client is blocked while processing a request. The nbdkit server design requires a thread per request being processed regardless of the number of connections or clients. We want to run 1000's of requests in parallel without needing a thread at nbdkit layer per request in flight. Our plugin layer is built around boost asio and a few threads in a worker pool running an io
2019 May 20
3
[nbdkit PATCH 0/2] More on .thread_model
Rich pointed out that making thread_model dynamic even for plugins makes some sense, so here is the code for doing it for 'sh'. I'm less confident on how to do it for OCaml and Rust (not to mention that those allow the plugin to already compile in their model, rather than the language binding glue forcing a model). The other languages (lua, perl, python, ruby) still need to be
2017 Nov 14
0
[PATCH 3/3] docs: Add a section about what happens to the plugin when nbdkit shuts down.
...always be called (eg. the server might be killed or segfault), so you should try to make the plugin as robust as possible by not requiring cleanup. +See also L</SHUTDOWN> below. =head2 C<.config> @@ -528,6 +529,16 @@ If none of the above thread models are suitable, then use C<NBDKIT_THREAD_MODEL_PARALLEL> and implement your own locking using C<pthread_mutex_t> etc. +=head1 SHUTDOWN + +When nbdkit receives certain signals it will shut down (see +L<nbdkit(1)/SIGNALS>). The server will wait for any currently running +plugin callbacks to finish and also call the C<.unload> call...