similar to: [PATCH nbdkit 0/3] server: Fix crash on close.

v1 was discussed here: https://www.redhat.com/archives/libguestfs/2019-October/thread.html#00100 v2: - Bind to VMADDR_CID_ANY (instead of HOST) and update the documentation accordingly. - Don't bother with SOCK_CLOEXEC fallback path that can never be used. Rich.

This is a series of patches to libnbd and nbdkit adding AF_VSOCK support. On the host side it allows you to start an nbdkit instance which listens on a virtio-vsock socket: $ ./nbdkit -fv --vsock memory 1G ... nbdkit: debug: bound to vsock 2:10809 On the guest side you can then use libnbd to connect to the server: $ ./run nbdsh -c 'h.connect_vsock(2, 10809)' -c

[Dropping devel, adding libguestfs] This can be reproduced on x86-64 so I can reproduce it locally. It only appears to happen when the tests are run under rpmbuild, not when I run them as ‘make check’, but I'm unclear why this is. As Eric described earlier, the test runs two copies of nbdkit and a client, connected like this: qemu-img info ===> nbdkit nbd ===> nbdkit example1

v1 was here: https://www.redhat.com/archives/libguestfs/2019-January/msg00008.html In v2 I have provided two patches: The first patch extends attribute((nonnull)) to most internal functions, but not to the external API. The second patch uses a macro so that attribute((format)) is only used in the public API on GCC or Clang. At least in theory these headers could be used by a C compiler which

I've posted some of these patches or ideas before; but now I'm confident enough with the series that it should be ready to push; at any rate, I can now run test-socket-activation in a tight loop without triggering any crashes or hangs. With this in place, I'm going back to work on making the nbd forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use

I'm not someone who thinks VLAs are automatically bad and unlike Linux kernel code they can sometimes be used safely in userspace. However for an internet exposed server there is an argument that they might cause some kind of exploitable situation especially if the code is compiled without other stack hardening features. Also in highly multithreaded code with limited stack sizes (as nbdkit

This is an incomplete port to Windows. Currently the server compiles and starts up successfully, but goes into an infinite loop when you connect to it. Nevertheless I think the approach is ready for feedback. This being Windows the changes go quite deep. Rich.

This is my second attempt at this. The first version (also not working) was here: https://www.redhat.com/archives/libguestfs/2019-October/msg00062.html In part 1/2 I introduce a new .ready_to_serve plugin method which is called after forking and just before accepting any client connection. The idea would be that plugins could start background threads here. However this doesn't work well in

On 10/18/19 10:38 AM, Richard W.M. Jones wrote: > On platforms which support it (only Linux currently) nbdkit can act as > a vsock server. Guests running on the host see a raw NBD socket which > it can connect to by opening an AF_VSOCK connection. (Although only > libnbd supports this). > > The current limitations are: > > * nbdkit can only act as a host (cid ==

On platforms which support it (only Linux currently) nbdkit can act as a vsock server. Guests running on the host see a raw NBD socket which it can connect to by opening an AF_VSOCK connection. (Although only libnbd supports this). The current limitations are: * nbdkit can only act as a host (cid == VMADDR_CID_HOST == 2). * There is no access control. Any guest which has vsock enabled can

This is a cheap way to find some use-after-free and uninitialized read problems when using glibc. This in fact reveals a race during filter shutdown (which this commit does not fix): Thread 2 (Thread 0x7f1caaa5ffc0 (LWP 7223)): #0 0x00007f1cab0a05f8 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0 #1 0x0000000000408842 in lock_unload () at locks.c:97 #2 0x00000000004066ff in

Also available here: https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw This is the port to Windows using native Windows APIs (not MSYS or Cygwin). This patch series is at the point where it basically now works. I can run the server with the memory plugin, and access it remotely using guestfish, creating filesystems and so on without any apparent problems. Nevertheless there are many

This method can be used for plugins to get control after the server has forked and changed user but before it accepts a connection. This is very late and the only real use for this is for a plugin to create background threads for its own use. --- docs/nbdkit-filter.pod | 20 +++++++++++++++----- docs/nbdkit-plugin.pod | 27 ++++++++++++++++++++++++++- include/nbdkit-filter.h | 2 ++

Also available here: https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw-nolib After a lot of work I have made the port to Windows work without using a separate library. Instead, on Windows only, we build an "import library" (library of stubs) which resolves references to nbdkit_* functions in the main program and fixes up the plugin, basically the first technique outlined in

On 26.09.2016 14:29, Richard W.M. Jones wrote: > On Mon, Sep 26, 2016 at 02:18:02PM +0200, Carl-Daniel Hailfinger wrote: >> Hi, >> >> has anyone ever run "make check" from nbd against nbdkit with a python >> plugin? I usually get segfaults during such a run, and sometimes various >> other errors happen before the segfault, suggesting that some memory

Having implemented a GC for HLVM, I am now turning my attention to implementing a GC that supports parallelism. To do this, I would like to use atomic instructions as well as mutexes. What is the status of LLVM's atomic instrinsics (e.g. CAS)? Is anyone using them in real projects? I realised that an obvious test would be to compile some simple example programs with llvm-g++ instead of

v1 was here: https://www.redhat.com/archives/libguestfs/2020-February/msg00081.html v2 replaces struct connection *conn = GET_CONN; with GET_CONN; which sets conn implicitly and asserts that it is non-NULL. If we actually want to test if conn is non-NULL or behave differently, then you must use threadlocal_get_conn() instead, and some existing uses do that. Rich.

I want to make my nbd forwarding plugin fully parallel - but to do that, I first need to make nbdkit itself fully parallel ;) With this series, I was finally able to demonstrate out-of-order responses when using qemu-io (which is great at sending back-to-back requests prior to waiting for responses) coupled with the nbd file plugin (which has a great feature of rdelay and wdelay, to make it

The third patch is a large but mechanical change which gets rid of passing around struct connection * entirely within the server, preferring instead to reference the connection through thread-local storage. I hope this is a gateway to simplifying other parts of the code. Rich.

On 9/5/19 6:28 AM, Richard W.M. Jones wrote: > I'm not someone who thinks VLAs are automatically bad and unlike Linux > kernel code they can sometimes be used safely in userspace. However > for an internet exposed server there is an argument that they might > cause some kind of exploitable situation especially if the code is > compiled without other stack hardening features.