similar to: [PATCH libnbd 0/2] Two patches to make libnbd work on FreeBSD.

With these patches, a majority of tests pass. The notable things which are still broken: - Because FreeBSD links /home -> /usr/home, $(pwd) gives a different result from realpath(2). Therefore some tests which implicitly rely on (eg) a plugin which calls nbdkit_realpath internally and then checking that path against $(pwd) fail. - Shebangs (#!) don't seem to work the same way

On Thu, Jul 04, 2019 at 09:45:26AM +0200, Martin Kletzander wrote: > On Wed, Jul 03, 2019 at 05:17:42PM +0100, Richard W.M. Jones wrote: > >Two simple patches which make libnbd compile on FreeBSD. > > > >Are we OK to copy common/include/byte-swapping.h from nbdkit? There > >is no license issue that I know of. Should we put it in lib/ or > >create a common/

Appears to have a measurable benefit, see 3/3 for test results. Rich.

I'm not sure if this is worth pursuing. On paper, it makes sense (if we know we have multiple commands batched to send over the wire, AND those commands are short in length, we might as well use MSG_MORE), but the measurement numbers with it applied might just be in the noise. Eric Blake (2): examples: Enhance access patterns of threaded-reads-and-writes states: Another use for MSG_MORE

This time around, the numbers are indeed looking better than in v1; and I like the interface better. Eric Blake (2): server: Prefer send() over write() server: Group related transmission send()s server/internal.h | 7 +++- server/connections.c | 51 +++++++++++++++++++++++++--- server/crypto.c | 11 ++++--

On Thu, Jul 04, 2019 at 10:25:03AM +0200, Martin Kletzander wrote: > On Thu, Jul 04, 2019 at 09:14:13AM +0100, Richard W.M. Jones wrote: > >On Thu, Jul 04, 2019 at 09:45:26AM +0200, Martin Kletzander wrote: > >>On Wed, Jul 03, 2019 at 05:17:42PM +0100, Richard W.M. Jones wrote: > >>>Two simple patches which make libnbd compile on FreeBSD. > >>> >

Slightly RFC, as I need more time to investigate why Unix sockets appeared to degrade with this patch. But as TCP sockets (over loopback to localhost) and TLS sessions (regardless of underlying Unix or TCP) both showed improvements, this looks like a worthwhile series. Eric Blake (2): server: Add support for corking server: Cork around grouped transmission send()s server/internal.h | 3

Necessary when writing a client that wants to avoid unnecessary EINVAL errors from sending unaligned requests. At some point, we may want to add synchronous convenience API wrappers that do request splitting or read-modify-write to obey server constraints while still appearing to the library client as accepting any possible request. But such a wrapper should only be synchronous and not copied to

[Adding libguestfs mailing list] I reproduced it locally - the difference was installing "gcc-go". I only had golang-bin installed previously. With gcc-go installed: ../run go install libguestfs.org/libnbd write of Go pointer 0xc000016060 to non-Go memory 0x7f5fe8297390 fatal error: Go pointer stored into non-Go memory runtime stack: runtime_mstart

If we compare the "common" subdirectory between nbdkit @ 6b4178d0fdfe ("ci: Temporarily disable perl in MacOS", 2023-02-27) and libnbd @ d05cd8f384a7 ("Version 1.15.11.", 2023-02-28), we find differences. We can categorize these differences along two (orthogonal) axes: - Intentional or unintentional. Intentional differences are for example when one of the libnbd

Since we disabled Nagle's algorithm we may send very small packets over the wire in some situations where we are calling send(2) from states that are responsible for small parts of the protocol. By setting the MSG_MORE flag we can indicate to the kernel that more data will follow (usually) immediately and so it can append the data to the same outgoing packet. Although there is some

There's an obvious bug in this patch in that it doesn't reset the h->wflags field in all cases. Updated patch below. I rechecked the performance measurements and they are the same after the updated patch. Rich. >From 15a687b50acecebcfd3dc6222d93e6df984b83c6 Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" <rjones@redhat.com> Date: Sat, 8 Jun 2019 19:12:22 +0100

Patch 1 played with an early draft of Rich's Fedora 30 libnbd package: https://bugzilla.redhat.com/show_bug.cgi?id=1713767#c17 Note that comment 21 provides a newer package 0.1.1-1 with a different API; and that libnbd has more unreleased API changes in the pipeline (whether that will be called 0.2 or 0.1.2); so we'll have to tweak things based on what is actually available in distros.

libnbd-0.1.2-1 is now available in Fedora 29/30 updates-testing, although it was not compiled against libxml2 so it lacks uri support (I ended up testing patch 4 with a self-built libnbd). Diffs since v1 - rebase to master, bump from libnbd 0.1 to 0.1.2, add URI support, better timing results Still not done - patch 5 needs associated tests Eric Blake (5): nbd: Check for libnbd nbd:

libnbd-0.1.4-1 is now available in Fedora 29/30 updates testing. Diffs since v2 - rebase to master, bump from libnbd 0.1.2 to 0.1.3+, add tests to TLS usage which flushed out the need to turn relative pathnames into absolute, doc tweaks Now that the testsuite covers TLS and libnbd has been fixed to provide the things I found lacking when developing v2, I'm leaning towards pushing this on

On Tue, May 30, 2023 at 04:06:32PM +0200, Laszlo Ersek wrote: > On 5/25/23 15:00, Eric Blake wrote: > > Support sending 64-bit requests if extended headers were negotiated. > > This includes setting NBD_CMD_FLAG_PAYLOAD_LEN any time we send an > > extended NBD_CMD_WRITE; this is such a fundamental part of the > > protocol that for now it is easier to silently ignore

Version 1 was here: https://www.redhat.com/archives/libguestfs/2020-April/thread.html#00160 Version 2 side-steps the objections to the first patch by using a well-formed alternate nbdkit.pc file and running ordinary pkg-config against it, so any parsing of --cflags etc will be done by pkg-config. The first patch is essentially the same idea as:

This adds coalescing of adjacent extents of the same type, as mentioned by Eric Blake in the commit message here: https://github.com/libguestfs/libnbd/commit/46072f6611f80245846a445766da071e457b00cd The patch series is rather long because it detours through adding the <vector.h> library from nbdkit into libnbd and replacing ad hoc uses of realloc, char ** etc in various places. Rich.

Following up to cf1a3045, if we know that we have more requests to transmit (because the user was queueing up requests while we were busy elsewhere), and our current request is short (a non-write, or a write with a small payload), then our current command can be batched with the next command. The numbers here were not as dramatic and may be merely in the noise; over three runs of: $ time

This might not be the final solution, but it certainly seems to solve a deadlock for me that I could trigger by using 'nbdkit --filter=noparallel memory 512k' and calling nbd_aio_pread for a request larger than 256k (enough for the Linux kernel to block the server until libnbd read()s), immediately followed by nbd_aio_pwrite for a request larger than 256k (enough to block libnbd until the