similar to: LIBNBD SECURITY: Assertion failure with unexpected block status

Right now, libnbd has refused to issue a command not advertised as supported by a server, including unknown flags, mainly because the NBD protocol does not guarantee what the server will do, and libnbd would rather stay in sync with the server than drop the connection. However, for integration purposes, it can be handy to coerce libnbd into sending something to see how the server will react

Right now, libnbd has refused to issue a command not advertised as supported by a server, mainly because the NBD protocol does not guarantee what the server will do, and libnbd would rather stay in sync with the server than drop the connection. However, for integration purposes, it can be handy to coerce libnbd into sending something to see how the server will react (whether it be an extension

The NBD protocol states that a 0-length request is undefined; we were inconsistent in that we let it through for read, write, and cache, but blocked it for trim, zero, and block_status. The NBD protocol also has documented rules on handling access beyond EOF, but we are currently wasting traffic to the server when we can give the same answer ourselves. Exposing this as a strictness knob gives

The NBD protocol states that a 0-length request is undefined; we were inconsistent in that we let it through for read, write, and cache, but blocked it for trim, zero, and block_status. The NBD protocol also has documented rules on handling access beyond EOF, but we are currently wasting traffic to the server when we can give the same answer ourselves. Exposing these as two more strictness knobs

The next strict knob: allow the user to pass unknown flags across the wire (this is different than passing a known flag at the wrong time). It is interesting to note that NBD only permits 16 bits of flags, but we have a signature that takes uint32_t; if we wanted, we could pack libnbd-specific flags in the upper bits that the NBD protocol would never see. --- generator/API.ml | 46

Qemu was able to demonstrate that knowing whether a zero operation is fast is useful when copying from one image to another: there is a choice between bulk pre-zeroing and then revisiting the data sections (fewer transactions, but depends on the zeroing to be fast), vs. visiting every portion of the disk only once (more transactions, but no time lost to duplicated I/O due to slow zeroes). As

On Fri, Sep 11, 2020 at 04:49:55PM -0500, Eric Blake wrote: > The next strict knob: allow the user to pass unknown flags across the > wire (this is different than passing a known flag at the wrong time). > > It is interesting to note that NBD only permits 16 bits of flags, but > we have a signature that takes uint32_t; if we wanted, we could pack > libnbd-specific flags in the

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

When structured replies are negotiated, the server may advertise support for the DF flag (the server promises to return at most one data/hole chunk, or to fail with NBD_EOVERFLOW if the chunk would be too large). As both nbdkit and qemu-nbd support this flag (the former only trivially, but the latter by not compressing holes over the wire), it is worth exposing to clients, if only for testing

Pipeline #932580084 has failed! Project: libnbd ( https://gitlab.com/nbdkit/libnbd ) Branch: master ( https://gitlab.com/nbdkit/libnbd/-/commits/master ) Commit: 653f9c21 ( https://gitlab.com/nbdkit/libnbd/-/commit/653f9c211da6943ab00a1fa665f0b8f0d21333d9 ) Commit Message: api: Fix block status assertion under set_stric... Commit Author: Eric Blake ( https://gitlab.com/ebblake ) Pipeline

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

On Sat, Jul 15, 2023 at 08:49:51PM -0500, Eric Blake wrote: > A compliant server should not send NBD_REPLY_TYPE_BLOCK_STATUS unless > we successfully negotiated a meta context. And our default strictness > settings refuse to let us send NBD_CMD_BLOCK_STATUS unless we > negotiated a meta context. But when you mix non-default settings > (using nbd_set_strict to disable

Right now, we require the user to supply potential metacontext names in advance, without knowing what the server actually supports until after the connection or nbd_opt_info call is complete. But the NBD protocol also supports a client being able to query what contexts a server supports, including where a client asks with a prefix and the server replies back with all answers that match the

On 5/29/19 10:13 PM, Eric Blake wrote: > Benchmark-wise, using the same setup as in commit e897ed70, I see > an order-of-magnitude slowdown: > > Pre-patch, the runs averaged 1.266s, 1.30E+08 bits/s > Post-patch, the runs averaged 11.663s, 1.41E+07 bits/s > > This will need further profiling to determine how much is nbdkit's > fault, and how much is libnbd's. I

On Sun, Jul 16, 2023 at 04:39:18PM +0000, Tage Johansson wrote: > > @@ -194,7 +198,10 @@ calls. The cookie is unique (per libnbd handle) and E<ge> 1. > > > > You may register a function which is called when the command > > completes, see L</Completion callbacks> below. In this case we have > > -specified a null completion callback. > >

I'm pleased to announce the release of libnbd 1.4.0, a high performance Network Block Device (NBD) client library. Key features of libnbd: * Synchronous API for ease of use. * Asynchronous API for writing non-blocking, multithreaded clients. You can mix both APIs freely. * High performance. * Minimal dependencies for the basic library. * Well-documented, stable API. * Bindings

A compliant server should not send NBD_REPLY_TYPE_BLOCK_STATUS unless we successfully negotiated a meta context. And our default strictness settings refuse to let us send NBD_CMD_BLOCK_STATUS unless we negotiated a meta context. But when you mix non-default settings (using nbd_set_strict to disable STRICT_COMMANDS) to send a block status without having negotiated it, coupled with a non-compliant

We have been inconsistent on how much we reject client-side without even consulting the server, vs. how much we depend on the server to detect failure (even if our request can be deemed undefined per NBD protocol). I'd like to change it so that by default, we reject as much as we can client-side for less traffic, but where the user can also change things on the fly for server-side integration

On 7/14/2023 4:13 PM, Eric Blake wrote: > On Fri, Jul 14, 2023 at 09:13:42AM +0200, Laszlo Ersek wrote: >> On 7/13/23 21:29, Eric Blake wrote: >>> The documentation has claimed since commit 6f4dcdab that any >>> completion callback will be called exactly once; but this is not >>> consistent with the code: if nbd_aio_* itself returns an error, then >>>

Go 1.17 or newer is required to use unsafe.Slice(), which in turn allows us to write a simpler conversion from a C array to a Go object during callbacks. To check if this makes sense, look at https://repology.org/project/go/versions compared to our list in ci/manifest.yml, at the time I made this commit: Alpine 3.15: 1.17.10 AlmaLinux 8: 1.19.10 CentOS Stream 8: 1.20.4 Debian 10: 1.11.6 Debian