search for: pthread_rwlock_wrlock

This is a cheap way to find some use-after-free and uninitialized read problems when using glibc. This in fact reveals a race during filter shutdown (which this commit does not fix): Thread 2 (Thread 0x7f1caaa5ffc0 (LWP 7223)): #0 0x00007f1cab0a05f8 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0 #1 0x0000000000408842 in lock_unload () at locks.c:97 #2 0x00000000004066ff in filter_free (b=0x203c330) at filters.c:77 #3 0x000000000040a6f4 in main (argc=11, argv=0x7ffc1f4486e8) at main.c:649 Thread 1 (Thread 0x7f1caaa5e700 (LWP 7226)): #0 0x000000000040...

...thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS) - pthread_mutex_unlock (&all_requests_lock); + if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS && + pthread_mutex_unlock (&all_requests_lock)) + abort (); } void lock_unload (void) { - pthread_rwlock_wrlock (&unload_prevention_lock); + if (pthread_rwlock_wrlock (&unload_prevention_lock)) + abort (); } void unlock_unload (void) { - pthread_rwlock_unlock (&unload_prevention_lock); + if (pthread_rwlock_unlock (&unload_prevention_lock)) + abort (); } -- 2.20.1

..."); + pthread_mutex_unlock (connection_get_request_lock (conn)); + } + + if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS) { + debug ("release global request lock"); + pthread_mutex_unlock (&all_requests_lock); + } +} + +void +lock_unload (void) +{ + pthread_rwlock_wrlock (&unload_prevention_lock); +} + +void +unlock_unload (void) +{ + pthread_rwlock_unlock (&unload_prevention_lock); +} diff --git a/src/plugins.c b/src/plugins.c index 9b5d2d5..b7ab43d 100644 --- a/src/plugins.c +++ b/src/plugins.c @@ -46,10 +46,6 @@ #include "nbdkit-plugin.h" #i...

..."); + pthread_mutex_unlock (connection_get_request_lock (conn)); + } + + if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS) { + debug ("release global request lock"); + pthread_mutex_unlock (&all_requests_lock); + } +} + +void +lock_unload (void) +{ + pthread_rwlock_wrlock (&unload_prevention_lock); +} + +void +unlock_unload (void) +{ + pthread_rwlock_unlock (&unload_prevention_lock); +} diff --git a/src/plugins.c b/src/plugins.c index 9b5d2d5..b7ab43d 100644 --- a/src/plugins.c +++ b/src/plugins.c @@ -46,10 +46,6 @@ #include "nbdkit-plugin.h" #i...

...lock_cb (CURL *handle, curl_lock_data data, curl_lock_access access, + void *userptr) +{ + assert (data < ARRAY_SIZE (lockarray)); + + switch (access) { + case CURL_LOCK_ACCESS_SHARED: + pthread_rwlock_rdlock (&lockarray[data]); + break; + case CURL_LOCK_ACCESS_SINGLE: + pthread_rwlock_wrlock (&lockarray[data]); + break; + default: + /* CURL_LOCK_ACCESS_NONE is never used in the current libcurl code. */ + abort (); + } +} + +static void +unlock_cb (CURL *handle, curl_lock_data data, void *userptr) +{ + assert (data < ARRAY_SIZE (lockarray)); + + pthread_rwlock_unlock...

...: > This is a cheap way to find some use-after-free and uninitialized read > problems when using glibc. > > This in fact reveals a race during filter shutdown (which this > commit does not fix): > > Thread 2 (Thread 0x7f1caaa5ffc0 (LWP 7223)): > #0 0x00007f1cab0a05f8 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0 > #1 0x0000000000408842 in lock_unload () at locks.c:97 > #2 0x00000000004066ff in filter_free (b=0x203c330) at filters.c:77 Do we need some sort of pthread_join() in filter_free() to allow... > #3 0x000000000040a6f4 in main (argc=11, argv=0x7ffc1f448...

..._RWLOCK_INITIALIZER; /* Maximum read or write request that we will handle. */ #define MAX_REQUEST_SIZE (64 * 1024 * 1024) @@ -155,6 +156,11 @@ void plugin_cleanup (void) { if (dl) { + /* Acquiring this lock prevents any plugin callbacks from running + * simultaneously. + */ + pthread_rwlock_wrlock (&unload_prevention_lock); + debug ("%s: unload", filename); if (plugin.unload) plugin.unload (); @@ -163,6 +169,8 @@ plugin_cleanup (void) dl = NULL; free (filename); filename = NULL; + + pthread_rwlock_unlock (&unload_prevention_lock); } }...

I'm mainly posting this to the list as a back-up. It does work, it does _not_ improve performance in any noticable way. However I'm having lots of trouble getting HTTP/2 to work (with or without this patch) and that's stopping me from testing anything properly. Rich.

...ock_access access, > + void *userptr) > +{ > + assert (data < ARRAY_SIZE (lockarray)); > + > + switch (access) { > + case CURL_LOCK_ACCESS_SHARED: > + pthread_rwlock_rdlock (&lockarray[data]); > + break; > + case CURL_LOCK_ACCESS_SINGLE: > + pthread_rwlock_wrlock (&lockarray[data]); > + break; we could at least assert the return values > + default: > + /* CURL_LOCK_ACCESS_NONE is never used in the current libcurl code. */ > + abort (); > + } > +} > + > +static void > +unlock_cb (CURL *handle, curl_lock_data data,...

I do have a question about whether patch 2 is right, or whether I've exposed a bigger problem in the truncate (and possibly other) filter, but the rest seem fairly straightforward. Eric Blake (4): server: Check for pthread lock failures truncate: Factor out reading real_size under mutex plugins: Check for mutex failures filters: Check for mutex failures filters/cache/cache.c

This fixes the race conditions for me, using the test described here: https://www.redhat.com/archives/libguestfs/2017-September/msg00226.html Rich.

Reza Sheykhi <hajishey at msu.edu> writes: > I got the following answers: > > which perl > /usr/bin/perl > > which llvm-config > /usr/local/bin/llvm-config > > which llvm-as > /usr/local/bin/llvm-as > > /usr/bin/llvm-confing --version > bash: /usr/bin/llvm-confing: No such file or directory Uh, there is a typo on the command above, it should be

I got the following answers: which perl /usr/bin/perl which llvm-config /usr/local/bin/llvm-config which llvm-as /usr/local/bin/llvm-as /usr/bin/llvm-confing --version bash: /usr/bin/llvm-confing: No such file or directory /usr/local/bin/llvm-config --version 2.8 Quoting Óscar Fuentes <ofv at wanadoo.es>: > Reza Sheykhi <hajishey at msu.edu> writes: > >> Thank you

v1 -> v2: - Fixed everything mentioned in the review. Rich.

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

...e _ZNSt3__115__get_classnameEPKcb 0000000000000000 DF *UND* 0000000000000030 GLIBC_2.2.5 toupper 0000000000000000 DF *UND* 000000000000000f Base _ZNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED2Ev 0000000000000000 DF *UND* 00000000000000af GLIBC_2.2.5 pthread_rwlock_wrlock 0000000000000000 DF *UND* 000000000000000a GLIBC_2.2.5 pthread_self 0000000000000000 DF *UND* 000000000000001f Base _ZTv0_n24_NSt3__113basic_istreamIcNS_11char_traitsIcEEED0Ev 0000000000000000 DF *UND* 00000000000000e5 Base _ZNSt3__113basic_ostreamIcNS_11char_traitsI...

Hello, I update my building scripts to build LLVM 4.0 RC1 (with clang, lldb, libc++, libc++abi, lld) on CentOS 6 and I got a lot of undefined symbols during linking LLDB. I'm using clang-3.9 and this configuration: -DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=/usr/bin/clang -DCMAKE_CXX_COMPILER=/usr/bin/clang++

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

...elf13OutputSection10addSectionEPNS0_12InputSectionE res_thread_freeres 108 _ZN3lld3elf13OutputSection10addSectionEPNS0_12InputSectionE _ZdlPv 106 _ZN4llvm12PassRegistryD2Ev _ZdlPv 105 _ZN4llvm3sys11RWMutexImpl14writer_releaseEv __pthread_rwlock_unlock 105 _ZN4llvm3sys11RWMutexImpl14writer_acquireEv pthread_rwlock_wrlock 104 _ZN4llvm12PassRegistryD2Ev __cfree 104 __pthread_once_slow __once_proxy 104 __pthread_once_slow _pthread_cleanup_pop 104 __pthread_once_slow _pthread_cleanup_push 104 __pthread_once __pthread_once_slow 104 _ZNK4llvm12DenseMapBaseINS_8DenseMapINS_9StringRefENS_6detail13DenseSetEmptyENS_12DenseMa...

A rebased version of the lld patch is attached. Cheers, Rafael On 31 July 2017 at 15:11, Rafael Avila de Espindola <rafael.espindola at gmail.com> wrote: > Tobias Edler von Koch <tobias at codeaurora.org> writes: > >> Hi Rafael, >> >> On 07/31/2017 04:20 PM, Rafael Avila de Espindola via llvm-dev wrote: >>> However, do we need to start with