search for: send_mor

...as a side effect, it leads to more network overhead when we send a reply split across more than one write(). Take advantage of various means for grouping related writes (Linux' MSG_MORE for sockets, gnutls' corking for TLS) to send a larger packet, and adjust callers to pass in our internal SEND_MORE flag as a hint when to use the improvements. I tested with appropriate combinations from: $ nbdkit {-p 10810,-U -} \ {--tls=require --tls-verify-peer --tls-psk=./keys.psk,} memory size=64m \ --run './aio-parallel-load{-tls,} {$unixsocket,nbd://localhost:10810}' with the following IO...

In the recent commit 3842a080 to add SEND_MORE support, I blindly implemented the tls code as: if (SEND_MORE) { cork send } else { send uncork } because it showed improvements for my test case of aio-parallel-load from libnbd. But that test sticks to 64k I/O requests. But with further investigation, I've learned that even t...

This time around, the numbers are indeed looking better than in v1; and I like the interface better. Eric Blake (2): server: Prefer send() over write() server: Group related transmission send()s server/internal.h | 7 +++- server/connections.c | 51 +++++++++++++++++++++++++--- server/crypto.c | 11 ++++--

...c.c | 2 +- server/test-public.c | 2 +- 13 files changed, 574 insertions(+), 555 deletions(-) diff --git a/server/internal.h b/server/internal.h index 1e7b4cf0..aed5a7da 100644 --- a/server/internal.h +++ b/server/internal.h @@ -154,15 +154,12 @@ enum { SEND_MORE = 1, /* Hint to use MSG_MORE/corking to group send()s */ }; -typedef int (*connection_recv_function) (struct connection *, - void *buf, size_t len) - __attribute__((__nonnull__ (1, 2))); -typedef int (*connection_send_function) (struct connection *, -...

v1 was here: https://www.redhat.com/archives/libguestfs/2020-February/msg00081.html v2 replaces struct connection *conn = GET_CONN; with GET_CONN; which sets conn implicitly and asserts that it is non-NULL. If we actually want to test if conn is non-NULL or behave differently, then you must use threadlocal_get_conn() instead, and some existing uses do that. Rich.

The third patch is a large but mechanical change which gets rid of passing around struct connection * entirely within the server, preferring instead to reference the connection through thread-local storage. I hope this is a gateway to simplifying other parts of the code. Rich.

...he uncork operation because more data would be > appended. nbdkit and libnbd do this with the MSG_MORE flag (plaintext) and TLS corking (tls); when building up a message to the other side, a flag is set any time we know we are likely to send more data very shortly. nbdkit wraps it under a flag SEND_MORE, which applies to both plaintext: https://gitlab.com/nbdkit/nbdkit/-/blob/master/server/connections.c#L415 and to TLS connections: https://gitlab.com/nbdkit/nbdkit/-/blob/master/server/crypto.c#L396 while libnbd uses MSG_MORE a bit more directly for the same purpose for plaintext, but isn't (...

...fication, are permitted provided that the following conditions are @@ -400,7 +400,10 @@ raw_send_socket (const void *vbuf, size_t len, int flags) ssize_t r; int f = 0; - assert (sock >= 0); + if (sock < 0) { + errno = EBADF; + return -1; + } #ifdef MSG_MORE if (flags & SEND_MORE) f |= MSG_MORE; -- 2.39.2

...ixed_new_option_reply.option = htobe32 (option); + fixed_new_option_reply.reply = htobe32 (reply); + fixed_new_option_reply.replylen = htobe32 (sizeof info + len); + name.info = htobe16 (info); + + if (conn->send (&fixed_new_option_reply, + sizeof fixed_new_option_reply, SEND_MORE) == -1 || + conn->send (&name, sizeof name, SEND_MORE) == -1 || + conn->send (str, len, 0) == -1) { + nbdkit_error ("write: %s: %m", name_of_nbd_opt (option)); + return -1; + } + + return 0; +} + static int send_newstyle_option_reply_meta_context (uint32_t o...

This takes Rich's API proposal and starts fleshing it out with enough meat that I was able to test 'nbdkit eval' advertising multiple exports with descriptions paired with 'qemu-nbd --list'. Eric Blake (3): server: Add exports list functions server: Prepare to use export list from plugin sh, eval: Add .list_exports support Richard W.M. Jones (1): server: Implement

...tocol.c @@ -367,7 +367,7 @@ send_simple_reply (struct connection *conn, uint32_t error) { ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&conn->write_lock); - struct simple_reply reply; + struct nbd_simple_reply reply; int r; int f = (cmd == NBD_CMD_READ && !error) ? SEND_MORE : 0; @@ -404,8 +404,8 @@ send_structured_reply_read (struct connection *conn, * that yet we acquire the lock for the whole function. */ ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&conn->write_lock); - struct structured_reply reply; - struct structured_reply_offset_data offset_data; + s...

Since v1: - patch 1: check size limits - patch 2: better handling of default export name canonicalization - patch 3: support filters as well as plugins - patch 4: new - patch 5: rewrite sh parser, fix testsuite to actually work and cover more cases (now that libnbd.git is fixed) Eric Blake (4): server: Add exports list functions server: Prepare to use export list from plugin log: Add

We should have only one NBD protocol file. Let's make nbdkit's version the canonical one, and use it in libnbd. Rich.

More patches on the way for improving .list_exports signature and adding .export_description, but this is the promised code showing why nbdkit_string_intern is useful. Patch 4 is somewhat RFC: we could either add new API to take the boilerplate from: foo_config(const char *key, const char *value) { if (strcmp (key, "file") == 0) { CLEANUP_FREE char *tmp = nbdkit_realpath (value);

This is a revision of my .default_export work, plus new work on .export_descriptions and a new exportname filter. I think it is now ready to check in. Things I'd still like in 1.22: - the file plugin should implement .list_exports (patch already posted, but it needs rebasing on this series) - the ext2 filter should override .list_exports when in exportname mode - the nbd plugin should be

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

I will be following up to this email with four separate threads each addressed to the appropriate single list, with proposed changes to: - the NBD protocol - qemu: both server and client - libnbd: client - nbdkit: server The feature in question adds a new optional NBD_INFO_ packet to the NBD_OPT_GO portion of handshake, adding up to 16 bits of information that the server can advertise to the