similar to: [PATCH nbdkit] server: Try hard to maintain invariant that fds 0, 1 and 2 are always open.

In preparation for parallel processing, we need to be sure that two threads belonging to the same connection cannot interleave their I/O except at message boundaries. Add a mutex around all reads and writes that must occur as a group (for now, there is no contention for either mutex). Signed-off-by: Eric Blake <eblake@redhat.com> --- src/connections.c | 27 +++++++++++++++++++++++++-- 1

Any time we reply to NBD_CMD_READ or NBD_CMD_BLOCK_STATUS, we end up calling conn->send() more than once. Now that we've disabled Nagle's algorithm, this implies that we try harder to send the small header immediately, rather than batching it with the rest of the payload, which causes more overhead in the amount of actual network traffic. For interfaces that support corking (gnutls, or

This structure was not freed along the non-error path, both resulting in a memory leak and providing an easy way for clients to blow up nbdkit servers if they enable TLS support. Ooops. Found by valgrind. --- src/crypto.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/crypto.c b/src/crypto.c index 948e79e..c2f9971 100644 --- a/src/crypto.c +++ b/src/crypto.c @@ -391,6 +391,10 @@

On Thu, Nov 16, 2017 at 09:26:54PM -0600, Eric Blake wrote: > In preparation for parallel processing, we need to be sure that > two threads belonging to the same connection cannot interleave > their I/O except at message boundaries. Add a mutex around > all reads and writes that must occur as a group (for now, there > is no contention for either mutex). > > Signed-off-by:

Mostly code motion. --- src/Makefile.am | 1 + src/connections.c | 14 +++---- src/internal.h | 14 ++++--- src/locks.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 ---

Mostly code motion. --- src/Makefile.am | 1 + src/connections.c | 14 +++---- src/internal.h | 14 ++++--- src/locks.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 ---

On 26.09.2016 19:20, Carl-Daniel Hailfinger wrote: > On 26.09.2016 14:29, Richard W.M. Jones wrote: >> On Mon, Sep 26, 2016 at 02:18:02PM +0200, Carl-Daniel Hailfinger wrote: >>> Hi, >>> >>> has anyone ever run "make check" from nbd against nbdkit with a python >>> plugin? I usually get segfaults during such a run, and sometimes various

This is a cheap way to find some use-after-free and uninitialized read problems when using glibc. This in fact reveals a race during filter shutdown (which this commit does not fix): Thread 2 (Thread 0x7f1caaa5ffc0 (LWP 7223)): #0 0x00007f1cab0a05f8 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0 #1 0x0000000000408842 in lock_unload () at locks.c:97 #2 0x00000000004066ff in

This returns the plugin name, which for plugins is the same as the ordinary .name method (but for filters will be different). --- src/connections.c | 4 ++-- src/internal.h | 1 + src/plugins.c | 1 + 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/src/connections.c b/src/connections.c index 921a5b2..e1ac543 100644 --- a/src/connections.c +++ b/src/connections.c @@ -221,7

On 3/18/19 11:59 AM, Richard W.M. Jones wrote: > This is a cheap way to find some use-after-free and uninitialized read > problems when using glibc. > > This in fact reveals a race during filter shutdown (which this > commit does not fix): > > Thread 2 (Thread 0x7f1caaa5ffc0 (LWP 7223)): > #0 0x00007f1cab0a05f8 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0

On Wed, Sep 27, 2017 at 11:54:48PM +0200, Hilko Bengen wrote: > Hi, > > when I tested building nbdkit 1.1.15 in a current Debian chroot, I ran > into the following test failure. A repeated build went fine through the > tests and so far I have not been able to reproduce it with the previous > version. > > The failing build was done using a clean Debian/sid, amd64 chroot

For GCC only, define unlikely() macro. Use it on error paths to move code out of the hot path. In the server only, use the debug() macro (don't call nbdkit_debug directly). This macro checks the verbose flag and moves the call to nbdkit_debug out of the hot path. --- server/connections.c | 11 ++++++----- server/internal.h | 17 ++++++++++++++++- server/plugins.c | 2 +-

Concerns: R 1.5.0 gui version, Windows (downloaded binary) and Linux (installed from sources). # Load the data from the attached file: kk<-read.table("__filename__", header=1) # attach the data: attach(kk) Snr<-factor(Snr) # fool around with a call to anova.glm(): anova.glm( aov( nFD~Type+size+Modality+Error(Snr/(Type+size+Modality)) ) ) # Error: object nFD not found # Well, I

Since commit 86fdb48c6a5362d66865493d9d2172166f99722e we have stored the connection object in thread-local storage. In this very large, but mostly mechanical change we stop passing the connection pointer around everywhere, and instead use the value stored in thread-local storage. This assumes a 1-1 mapping between the connection and the current thread which is true in *most* places.

It's finally time to implement one of the TODO items: we want to support a thread pool of parallel readers from the client, in order to allow multiple in-flight operations with potential out-of-order completion. We also need at least one plugin that supports parallel processing for testing the option; the file plugin fits the bill. Add and document a new command line option, -t/--threads=N,

For mainly historical reasons we tended to use int to store boolean values. However using bool is probably safer in some corner cases (eg. ?v == true? can fail badly if v is an int, but works for bool). bool was added in C99 so let's use it. --- server/internal.h | 16 +++++------ server/connections.c | 28 +++++++++--------- server/crypto.c | 4 +-- server/main.c | 67

[Dropping devel, adding libguestfs] This can be reproduced on x86-64 so I can reproduce it locally. It only appears to happen when the tests are run under rpmbuild, not when I run them as ‘make check’, but I'm unclear why this is. As Eric described earlier, the test runs two copies of nbdkit and a client, connected like this: qemu-img info ===> nbdkit nbd ===> nbdkit example1

File names on a Samba share with accented characters using the NFD form of UTF8 cannot be seen or used by Mac clients. While tying to find a solution, I came across this old thread : https://lists.samba.org/archive/samba/2014-December/187545.html ("[Samba] Mount unix samba 4 share to osx client without mangled file names") A reply in that thread suggested: > >/Do I have

This fixes the long-standing crash on close when nbdkit exits. I did try first to fix threads so we're using a proper thread pool, but that's difficult to implement. So this does the minimal change needed to fix the crash instead. There are still two segfaults that happen during running the test suite. One is deliberately caused (tests/test-captive.sh). The other appears to be an

We don't know how long a plugin's .get_size() will take, but we also documented that it shouldn't change per connection and therefore can be cached. It's not hard to see that we have to consult the size per connection (see commit b3a43ccd for a test that purposefully exposes different sizes to separate clients), nor to search the code to see we already cache it at the protocol