search for: nbd_flag_can_multi_conn

...say is that I need to do a *lot* more testing on this, > so this is just an early peek. In particular, although it passed > ‘make check && make check-valgrind’ I have *not* tested it against a > multi-conn-aware client such as the Linux kernel >= 4.9. > > This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc > as: > > "NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates > entirely without cache, or that the cache it uses is shared among > all connections to the given device. In particular, if this flag is > present, then the effects...

It is safe to return NBD_FLAG_CAN_MULTI_CONN from almost all readonly plugins. However only do it where the plugin thread model is NBDKIT_THREAD_MODEL_PARALLEL and the plugin could plausibly be used in a high performance situation. --- plugins/floppy/floppy.c | 8 ++++++++ plugins/iso/iso.c | 8 ++++++++ plugins/pattern/pattern.c...

On Sat, Jan 05, 2019 at 07:42:56AM +0000, Richard W.M. Jones wrote: > I've yet to do testing, but it seems to be necessary if we're going to > do scaling with the Linux client. Some preliminary numbers on this. I've enabled multi-conn for both the file and memory plugins, and I'm using fio with 8 threads to test this against the Linux kernel client. Without multi-conn:

Here are some more interesting numbers, concentrating on the memory plugin and RAM disks. All are done using 8 threads and multi-conn, on a single unloaded machine with 16 cores, using a Unix domain socket. (1) The memory plugin using the sparse array, as implemented upstream in 1.9.8: read: IOPS=103k, BW=401MiB/s (420MB/s)(46.0GiB/120002msec) write: IOPS=103k, BW=401MiB/s

On Sat, Jan 05, 2019 at 07:42:56AM +0000, Richard W.M. Jones wrote: > > Should the callback include a bool readonly parameter that lets the > > filter/plugin know for sure whether it is answering the question on a > > read-only connection vs. on a read-write connection, as the answer may > > differ between those two? > > Yes, I think this is a good idea. Will try

This allows multiple connections from a single client, and should be safe assuming flush/FUA has been implemented correctly in the previous commit. --- plugins/file/file.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/plugins/file/file.c b/plugins/file/file.c index dcff0ee..628f8fb 100644 --- a/plugins/file/file.c +++ b/plugins/file/file.c @@ -250,6 +250,13 @@ file_get_size (void

First thing to say is that I need to do a *lot* more testing on this, so this is just an early peek. In particular, although it passed ‘make check && make check-valgrind’ I have *not* tested it against a multi-conn-aware client such as the Linux kernel >= 4.9. This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc as: "NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates entirely without cache, or that the cache it uses is shared among all connections to the given device. In particular, if this flag is present, then the effects of NBD_CMD_FLUSH and NBD_CMD_...

>From the protocol document: "NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates entirely without cache, or that the cache it uses is shared among all connections to the given device. In particular, if this flag is present, then the effects of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA MUST be visible across all connections when the server se...

...a *lot* more testing on this, > > so this is just an early peek. In particular, although it passed > > ‘make check && make check-valgrind’ I have *not* tested it against a > > multi-conn-aware client such as the Linux kernel >= 4.9. > > > > This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc > > as: > > > > "NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates > > entirely without cache, or that the cache it uses is shared among > > all connections to the given device. In particular, if this flag is > &g...

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11] data, memory: Return NBD_FLAG_CAN_MULTI_CONN. - Writeable plugins where it should be safe to set the flag always. [11/11] memory, data: Use fine-...

...(void *handle) { struct handle *h = handle; - return h->flags & NBD_FLAG_SEND_WRITE_ZEROES; + return !!(h->flags & NBD_FLAG_SEND_WRITE_ZEROES); } static int @@ -1215,7 +1215,7 @@ nbd_can_multi_conn (void *handle) { struct handle *h = handle; - return h->flags & NBD_FLAG_CAN_MULTI_CONN; + return !!(h->flags & NBD_FLAG_CAN_MULTI_CONN); } static int -- 2.25.1

...same data can be served more quickly later. Is it worth mentioning that this is fine for a client that is expected to be the only writing client of a given server; but that it can result in stale data if the server can modify the data via other means? (In particular, since we don't implement NBD_FLAG_CAN_MULTI_CONN, we already admit that two clients trying to write in parallel are not guaranteed to see consistent read results after a flush - while caching read data only makes that more apparent) > +++ b/tests/test-cache-on-read.sh What you have here looks okay, but I have a possible idea for an additional...

...ronous calls and expects when the > plugin functions return that it can then send the nbd reply to the socket. > > It's parallel thread model is also not implemented as of yet I think this bit can be fixed fairly easily. One change which is especially easy to make is to send back the NBD_FLAG_CAN_MULTI_CONN flag (under control of the plugin). Anyway this doesn't solve your problem ... > but the > current design still mandates a worker thread per parallel op in progress > due to the synchronous design of the plugin calls. And the synchronous / 1 thread per client design of the server....

...> - return h->flags & NBD_FLAG_SEND_WRITE_ZEROES; > + return !!(h->flags & NBD_FLAG_SEND_WRITE_ZEROES); > } > > static int > @@ -1215,7 +1215,7 @@ nbd_can_multi_conn (void *handle) > { > struct handle *h = handle; > > - return h->flags & NBD_FLAG_CAN_MULTI_CONN; > + return !!(h->flags & NBD_FLAG_CAN_MULTI_CONN); > } > > static int While I don't like the idea of modifying plugins to fix broken nbdkit behaviour, you make a good case for this being safer, so: ACK Rich. -- Richard Jones, Virtualization Group, Red Hat http://peo...

----- Forwarded message ----- Date: Sat, 18 Feb 2017 22:21:19 -0500 Subject: nbdkit async Hello, Hope this is the right person to contact regarding nbdkit design. I have a high latency massively parallel device that I am currently implementing as an nbdkit plugin in c++ and have run into some design limitations due to the synchronous callback interface nbdkit requires. Nbdkit is currently

...ommon/protocol/protocol.h @@ -94,6 +94,7 @@ extern const char *name_of_nbd_flag (int); #define NBD_FLAG_ROTATIONAL (1 << 4) #define NBD_FLAG_SEND_TRIM (1 << 5) #define NBD_FLAG_SEND_WRITE_ZEROES (1 << 6) +#define NBD_FLAG_SEND_DF (1 << 7) #define NBD_FLAG_CAN_MULTI_CONN (1 << 8) /* NBD options (new style handshake only). */ @@ -217,6 +218,7 @@ extern const char *name_of_nbd_cmd (int); extern const char *name_of_nbd_cmd_flag (int); #define NBD_CMD_FLAG_FUA (1<<0) #define NBD_CMD_FLAG_NO_HOLE (1<<1) +#define NBD_CMD_FLAG_DF (1&l...

...b/flags.c +++ b/lib/flags.c @@ -42,6 +42,16 @@ nbd_internal_set_size_and_flags (struct nbd_handle *h, return -1; } + /* It is unsafe to connect to a server with multi-conn set unless + * the server says it is safe to do so. + */ + if (h->multi_conn > 1 && (eflags & NBD_FLAG_CAN_MULTI_CONN) == 0) { + set_error (EINVAL, "handshake: multi-conn is set on this handle, " + "but the server does not advertize multi-conn support " + "so disconnecting because it is not safe to continue"); + return -1; + } + h->exportsize =...

Per recent discussion here: https://www.redhat.com/archives/libguestfs/2019-May/thread.html#00175

...emented by qemu-nbd (the server) for readonly connections. Note that multi-conn and --shared are (a bit) different. Multi-conn is a flag which the server can send to clients to indicate not only that multiple connections are possible, but also that they are come with certain guarantees: bit 8, NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates entirely without cache, or that the cache it uses is shared among all connections to the given device. In particular, if this flag is present, then the effects of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA MUST be visible across all connections when the server se...

...or Tcl. * Filters can be stacked in front of plugins to transform the output. Clone source: https://github.com/libguestfs/nbdkit Download tarball: http://download.libguestfs.org/nbdkit/1.10-stable/ (other downloads: http://download.libguestfs.org/nbdkit/ ) New in this release: - Implemented NBD_FLAG_CAN_MULTI_CONN which greatly improves performance for clients that support it. - Implemented NBD_OPT_INFO (Eric Blake). - nbdkit-data-plugin and nbdkit-memory-plugin: By adding fine-grained locking, these plugins now use the fully parallel thread model. - nbdkit-cache-filter: The size of the cache can now...