search for: write_zeroes

...onse to NBD_CMD_BLOCK_STATUS. > > OTOH, we wouldn’t need such a flag for the implementation, because we > could just send a 64-bit discard/make_zero over the whole block device > length to the NBD server, and then the server internally does the right > thing(TM). AFAIU discard and write_zeroes currently have only 32 bit > length fields, but there were plans for adding support for 64 bit > versions anyway. From my naïve outsider perspective, doing that doesn’t > seem a more complicated protocol addition than adding some way to tell > whether an NBD export is zero. Adding 64-...

The upstream protocol recently promoted NBD_CMD_WRITE_ZEROES from experimental to a documented extension. Exposing support for this allows plugin writers to create sparse files when driven by a client that knows how to use the extension; meanwhile, even if a plugin does not support this extension, the server benefits from less network traffic from the client...

...read-only export (so failing with EPERM is wrong) [meanwhile, if we don't advertise flush because plugin_can_flush() fails, then rejecting with EINVAL is still okay] - returning EPERM (aka EROFS) for read-only exports should probably take precedence over anything else - out-of-bounds WRITE and WRITE_ZEROES should fail with ENOSPC, not EIO - out-of-bounds READ and TRIM should fail with EINVAL, not EIO We also had dead code: valid_range() and validate_request() cannot return -1. Make these functions return bool instead. And finally, fix a comment typo. Signed-off-by: Eric Blake <eblake@redhat.co...

...order to alter what the server advertises to the guest during the handshake phase; at which point the guest is out-of-spec if it tries to use the command that the plugin did not advertise. But while those other commands have no real impact to the amount of wire traffic, we want to always advertise WRITE_ZEROES support to the client (because it is so much more efficient to pass a WRITE_ZEROES command over the wire than a regular WRITE with a payload of zeroes), even when the plugin doesn't support it. Since having a can_zero callback does not affect what we advertise, the only other reason to impleme...

I still need to add testsuite coverage. Perhaps it might be easier if I create a new '--filter=log logfile=foo' filter that produces a log of which commands a client sent, then compare the log using a known client that uses write_zeroes (qemu-io works well) both with and without --filter=nozero to prove that the change in advertisement changes the commands sent over the wire (that would also test the log filter at the same time...). I'll wait for a review on this series, particularly since patch 2 intentionally breaks filter...

On 8/28/19 4:57 AM, Vladimir Sementsov-Ogievskiy wrote: >> Hence, it is desirable to have a way for clients to specify that a >> particular write zero request is being attempted for a fast wipe, and >> get an immediate failure if the zero request would otherwise take the >> same time as a write. Conversely, if the client is not performing a >> pre-initialization

...the server advertises to the guest during the handshake phase; at > which point the guest is out-of-spec if it tries to use the command that > the plugin did not advertise. But while those other commands have no > real impact to the amount of wire traffic, we want to always advertise > WRITE_ZEROES support to the client (because it is so much more efficient > to pass a WRITE_ZEROES command over the wire than a regular WRITE with a > payload of zeroes), even when the plugin doesn't support it. Since > having a can_zero callback does not affect what we advertise, the only > oth...

While our plugin code always advertises WRITE_ZEROES on writable connections (because we can emulate .zero by calling .pwrite), and FUA support when .flush exists (because we can emulate the persistence by calling .flush), it is conceivable that a filter may want to explicitly avoid advertising particular bits. (More to the point, I plan on writing...

A client cannot attempt TRIM on a read-only connection (we already filter it in connections.c with EROFS); as such, it makes no sense to query the plugin's .can_trim nor to advertise TRIM support to the client on such a connection, similar to how we do not advertise WRITE_ZEROES. FUA only has defined meanings on write operations, but the NBD protocol allows us to advertise it on reads and to ignore clients that send it on read (older qemu did so), so we can continue to advertise it regardless of readonly state. Likewise, while flush is unlikely to make a difference on a...

On 8/30/19 1:00 PM, Vladimir Sementsov-Ogievskiy wrote: > 23.08.2019 17:37, Eric Blake wrote: >> When creating a read-only image, we are still advertising support for >> TRIM and WRITE_ZEROES to the client, even though the client should not >> be issuing those commands. But seeing this requires looking across >> multiple functions: >> >> @@ -458,10 +458,13 @@ static int nbd_negotiate_handle_export_name(NBDClient *client, >> return -EINVAL; >&...

...implementations do the right thing by themselves” moot. OTOH, we wouldn’t need such a flag for the implementation, because we could just send a 64-bit discard/make_zero over the whole block device length to the NBD server, and then the server internally does the right thing(TM). AFAIU discard and write_zeroes currently have only 32 bit length fields, but there were plans for adding support for 64 bit versions anyway. From my naïve outsider perspective, doing that doesn’t seem a more complicated protocol addition than adding some way to tell whether an NBD export is zero. So I’m still wondering whether...

...23, 2018 at 10:10:12PM -0600, Eric Blake wrote: > I still need to add testsuite coverage. Perhaps it might be easier > if I create a new '--filter=log logfile=foo' filter that produces > a log of which commands a client sent, then compare the log using > a known client that uses write_zeroes (qemu-io works well) both > with and without --filter=nozero to prove that the change in > advertisement changes the commands sent over the wire (that would > also test the log filter at the same time...). > > I'll wait for a review on this series, particularly since patch 2 &gt...

On 3/22/19 11:42 AM, Eric Blake wrote: > > Hence, it is desirable to have a way for clients to specify that a > particular write zero request is being attempted for a fast wipe, and > get an immediate failure if the zero request would otherwise take the > same time as a write. Conversely, if the client is not performing a > pre-initialization pass, it is still more efficient

Begin the language binding followups to my new .zero callback, since Rich was indeed correct that we want them. I'm more familiar with python and perl (at least to the point that I was able to modify the appropriate example files and prove to myself that the bindings worked), so I've started with those. I'm less familiar with ruby and ocaml, so I've left those for tomorrow (it

...gt; -------- > - > -It should be possible to layer filters over plugins to do things like: > +Suggestions for filters > +----------------------- > > * adding artificial delays (see wdelay/rdelay options in the file > plugin) How about a filter that intentionally disables WRITE_ZEROES and/or FUA support, if for no other reason than for testing sane fallbacks and comparing speed differences? > +++ b/docs/nbdkit-filter.pod > @@ -0,0 +1,501 @@ > +=encoding utf8 > + > +=head1 NAME > + > +nbdkit-filter - How to write nbdkit filters > + > +=head1 SYNOPSIS...

On Wed, Feb 12, 2020 at 06:09:11AM -0600, Eric Blake wrote: > On 2/12/20 1:27 AM, Wouter Verhelst wrote: > >Hi, > > > >On Mon, Feb 10, 2020 at 10:52:55PM +0000, Richard W.M. Jones wrote: > >>But anyway ... could a flag indicating that the whole image is sparse > >>be useful, either as well as NBD_INIT_SPARSE or instead of it? You > >>could use it to

On Fri, Jan 19, 2018 at 10:22:27AM -0600, Eric Blake wrote: > > +Suggestions for filters > > +----------------------- > > > > * adding artificial delays (see wdelay/rdelay options in the file > > plugin) > > How about a filter that intentionally disables WRITE_ZEROES and/or FUA > support, if for no other reason than for testing sane fallbacks and > comparing speed differences? If you have ideas for the TODO file just go ahead and commit them, no need for review. > > +=head1 SYNOPSIS > > > +To see example filters, take a look at the sourc...

While it may be counterintuitive at first, the introduction of NBD_CMD_WRITE_ZEROES and NBD_CMD_BLOCK_STATUS has caused a performance regression in qemu [1], when copying a sparse file. When the destination file must contain the same contents as the source, but it is not known in advance whether the destination started life with all zero content, then there are cases where it is f...

...dle) +{ + int r; + + if (zeromode == NOTRIM) { + r = next_ops->can_zero (nxdata); + if (r == -1) + return -1; + if (!r) { + nbdkit_error ("zeromode 'notrim' requires plugin zero support"); + return -1; + } + } + return 0; +} /* Advertise desired WRITE_ZEROES mode. */ static int nozero_can_zero (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) { - return emulate; + return zeromode != NONE; } static int @@ -81,11 +105,15 @@ nozero_zero (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle, uint32_t count, uint64...

....pread or from .pread to > no-op. Be thinking about that as you review the rest of the series. I'm worried that we end up in the same situation that we do with .zero: because we emulate it, we end up with a slow zero support, whereas clients [well, Nir S. mainly] actually want NBD_FLAG_SEND_WRITE_ZEROES == *fast* zeroing is available. In particular in the NBD_CMD_CACHE case: for some plugins emulating this behind the scenes with pread makes some sense. eg. It works for nbdkit-file-plugin because it will prefetch the data into Linux's page cache. However if a plugin makes a network access, e...