search for: crypto_sess

...onnection *conn, const void *vbuf, size_t len) return 0; } +/* Change the cork status to batch a group of send calls, and either succeed + * completely (returns 0) or fail (returns -1). + */ +static int +crypto_cork (struct connection *conn, bool cork) +{ + gnutls_session_t session = conn->crypto_session; + + assert (session != NULL); + + if (cork) + gnutls_record_cork (session); + else if (gnutls_record_uncork (session, GNUTLS_RECORD_WAIT) < 0) + return -1; + + return 0; +} + /* There's no place in the NBD protocol to send back errors from * close, so this function ignores e...

These are a couple of patches in preparation for the Block Status implementation. While the patches (especially the second one) are very large they are really just elementary code motion. Rich.

...connections.c b/src/connections.c index dada9aa..dd43a9a 100644 --- a/src/connections.c +++ b/src/connections.c @@ -62,6 +62,8 @@ /* Connection structure. */ struct connection { pthread_mutex_t request_lock; + pthread_mutex_t read_lock; + pthread_mutex_t write_lock; void *handle; void *crypto_session; @@ -206,6 +208,8 @@ new_connection (int sockin, int sockout) conn->sockin = sockin; conn->sockout = sockout; pthread_mutex_init (&conn->request_lock, NULL); + pthread_mutex_init (&conn->read_lock, NULL); + pthread_mutex_init (&conn->write_lock, NULL);...

I want to make my nbd forwarding plugin fully parallel - but to do that, I first need to make nbdkit itself fully parallel ;) With this series, I was finally able to demonstrate out-of-order responses when using qemu-io (which is great at sending back-to-back requests prior to waiting for responses) coupled with the nbd file plugin (which has a great feature of rdelay and wdelay, to make it

Slightly RFC, as I need more time to investigate why Unix sockets appeared to degrade with this patch. But as TCP sockets (over loopback to localhost) and TLS sessions (regardless of underlying Unix or TCP) both showed improvements, this looks like a worthwhile series. Eric Blake (2): server: Add support for corking server: Cork around grouped transmission send()s server/internal.h | 3

...d43a9a 100644 > --- a/src/connections.c > +++ b/src/connections.c > @@ -62,6 +62,8 @@ > /* Connection structure. */ > struct connection { > pthread_mutex_t request_lock; > + pthread_mutex_t read_lock; > + pthread_mutex_t write_lock; > void *handle; > void *crypto_session; > > @@ -206,6 +208,8 @@ new_connection (int sockin, int sockout) > conn->sockin = sockin; > conn->sockout = sockout; > pthread_mutex_init (&conn->request_lock, NULL); > + pthread_mutex_init (&conn->read_lock, NULL); > + pthread_mutex_init (&a...

...c/connections.c index e1ac543..51a9772 100644 --- a/src/connections.c +++ b/src/connections.c @@ -69,10 +69,12 @@ struct connection { pthread_mutex_t write_lock; pthread_mutex_t status_lock; int status; /* 1 for more I/O with client, 0 for shutdown, -1 on error */ - void *handle; void *crypto_session; int nworkers; + void **handles; + size_t nr_handles; + uint64_t exportsize; int readonly; int can_flush; @@ -100,16 +102,37 @@ static void raw_close (struct connection *); /* Accessors for public fields in the connection structure. * Everything else is private to this file....

...ruct connection *conn) gnutls_bye (session, GNUTLS_SHUT_RDWR); if (sockin >= 0) - close (sockin); + close_or_nullify_fd (sockin); if (sockout >= 0 && sockin != sockout) - close (sockout); + close_or_nullify_fd (sockout); gnutls_deinit (session); conn->crypto_session = NULL; diff --git a/server/main.c b/server/main.c index 65025a6..65bef30 100644 --- a/server/main.c +++ b/server/main.c @@ -60,6 +60,7 @@ static struct backend *open_filter_so (struct backend *next, size_t i, const cha static void start_serving (void); static void write_pidfile (void); stati...

...values */ uint64_t exportsize; int can_write; @@ -195,7 +203,7 @@ struct b_conn_handle { }; static inline void -reset_b_conn_handle (struct b_conn_handle *h) +reset_handle (struct handle *h) { h->handle = NULL; h->state = 0; @@ -222,7 +230,7 @@ struct connection { void *crypto_session; int nworkers; - struct b_conn_handle *handles; + struct handle *handles; /* One per plugin and filter. */ size_t nr_handles; char exportname[NBD_MAX_STRING + 1]; @@ -239,6 +247,12 @@ struct connection { connection_close_function close; }; +static inline struct handle...

...c/connections.c index 7a4c7bb..e225b5c 100644 --- a/src/connections.c +++ b/src/connections.c @@ -69,10 +69,12 @@ struct connection { pthread_mutex_t write_lock; pthread_mutex_t status_lock; int status; /* 1 for more I/O with client, 0 for shutdown, -1 on error */ - void *handle; void *crypto_session; int nworkers; + void **handles; + size_t nr_handles; + uint64_t exportsize; int readonly; int can_flush; @@ -100,16 +102,37 @@ static void raw_close (struct connection *); /* Accessors for public fields in the connection structure. * Everything else is private to this file....

I've posted some of these patches or ideas before; but now I'm confident enough with the series that it should be ready to push; at any rate, I can now run test-socket-activation in a tight loop without triggering any crashes or hangs. With this in place, I'm going back to work on making the nbd forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use

This time around, the numbers are indeed looking better than in v1; and I like the interface better. Eric Blake (2): server: Prefer send() over write() server: Group related transmission send()s server/internal.h | 7 +++- server/connections.c | 51 +++++++++++++++++++++++++--- server/crypto.c | 11 ++++--

We noticed while writing various libnbd tests that when the delay filter is in use, there are scenarios where we had to resort to SIGKILL to get rid of nbdkit, because it was non-responsive to SIGINT. I'm still trying to figure out the best way to add testsuite coverage of this, but already proved to myself that it works from the command line, under two scenarios that both used to cause long

...,8 +312,9 @@ crypto_free (void) * (returns > 0), read an EOF (returns 0), or fail (returns -1). */ static int -crypto_recv (struct connection *conn, void *vbuf, size_t len) +crypto_recv (void *vbuf, size_t len) { + struct connection *conn = GET_CONN; gnutls_session_t session = conn->crypto_session; char *buf = vbuf; ssize_t r; @@ -355,8 +356,9 @@ crypto_recv (struct connection *conn, void *vbuf, size_t len) * (returns 0) or fail (returns -1). flags is ignored for now. */ static int -crypto_send (struct connection *conn, const void *vbuf, size_t len, int flags) +crypto_send (con...

v1 was here: https://www.redhat.com/archives/libguestfs/2020-February/msg00081.html v2 replaces struct connection *conn = GET_CONN; with GET_CONN; which sets conn implicitly and asserts that it is non-NULL. If we actually want to test if conn is non-NULL or behave differently, then you must use threadlocal_get_conn() instead, and some existing uses do that. Rich.

The third patch is a large but mechanical change which gets rid of passing around struct connection * entirely within the server, preferring instead to reference the connection through thread-local storage. I hope this is a gateway to simplifying other parts of the code. Rich.

It's confusing to use the same terminology for a single backend as for the linked list of backends. In particular it's often not clear if we're calling the next backend or the whole chain of backends. --- server/internal.h | 14 ++++++++++-- server/connections.c | 20 ++++++++--------- server/locks.c | 2 +- server/main.c

It's easy to use the sh script to demonstrate that nbdkit is inefficiently calling into .get_size, .can_fua, and friends more than necessary. We've also commented on the list in the past that it would be nice to ensure that when filters call into next_ops, they are not violating constraints (as we've have to fix several bugs in the past where we did not have such checking to protect

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

Also available here: https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw This is the port to Windows using native Windows APIs (not MSYS or Cygwin). This patch series is at the point where it basically now works. I can run the server with the memory plugin, and access it remotely using guestfish, creating filesystems and so on without any apparent problems. Nevertheless there are many