search for: zeromode

...-) diff --git a/filters/nozero/nbdkit-nozero-filter.pod b/filters/nozero/nbdkit-nozero-filter.pod index 7e06570..8e694bb 100644 --- a/filters/nozero/nbdkit-nozero-filter.pod +++ b/filters/nozero/nbdkit-nozero-filter.pod @@ -18,14 +18,19 @@ testing client or server fallbacks. =over 4 -=item B<zeromode=none|emulate> +=item B<zeromode=none|emulate|notrim> Optional, controls which mode the filter will use. Mode B<none> -(default) means that zero support is not advertised to the client; -mode B<emulate> means that zero support is emulated by the filter -using the plugin's...

...a new NBD_FLAG was okay; and this thread solves the second bullet of having reference implementations. Here's what I did for testing full-path interoperability: nbdkit memory -> qemu-nbd -> nbdkit nbd -> nbdsh $ nbdkit -p 10810 --filter=nozero --filter=delay memory 1m delay-write=3 zeromode=emulate $ qemu-nbd -p 10811 -f raw nbd://localhost:10810 $ nbdkit -p 10812 nbd nbd://localhost:10811 $ time nbdsh --connect nbd://localhost:10812 -c 'buf = h.zero(512, 0)' # takes more than 3 seconds, but succeeds $ time nbdsh --connect nbd://localhost:10812 -c 'buf = h.zero(512, 0, nbd...

This is a subset of the last half of the larger 9-patch series. The uncontroversial first half of that series is pushed, but here, I tried to reduce the size of the patches by splitting out some of the more complex changes, so that the rest of the changes remaining in the series are more mechanical. In turn, it forced me to write timing tests, which let me spot another spot where we are wasting

...rmitted provided that the following conditions are @@ -102,12 +102,22 @@ nozero_zero (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle, uint32_t count, uint64_t offs, uint32_t flags, int *err) { + int writeflags = 0; + bool need_flush = false; + assert (zeromode != NONE); flags &= ~NBDKIT_FLAG_MAY_TRIM; if (zeromode == NOTRIM) return next_ops->zero (nxdata, count, offs, flags, err); + if (flags & NBDKIT_FLAG_FUA) { + if (next_ops->can_fua (nxdata) == NBDKIT_FUA_EMULATE) + need_flush = true; + else + writeflags =...

...disk.img + nbdkit --filter=nozero file disk.img Serve the file F<disk.img>, allowing the client to take advantage of less network traffic via C<NBD_CMD_WRITE_ZEROES>, but still forcing the data to be written explicitly rather than punching any holes: - nbdkit --filter=nozero file zeromode=emulate file=disk.img + nbdkit --filter=nozero file zeromode=emulate disk.img =head1 SEE ALSO diff --git a/filters/offset/nbdkit-offset-filter.pod b/filters/offset/nbdkit-offset-filter.pod index 61496d2..a232e46 100644 --- a/filters/offset/nbdkit-offset-filter.pod +++ b/filters/offset/nbdkit-o...

...disk.img + nbdkit --filter=nozero file disk.img Serve the file F<disk.img>, allowing the client to take advantage of less network traffic via C<NBD_CMD_WRITE_ZEROES>, but still forcing the data to be written explicitly rather than punching any holes: - nbdkit --filter=nozero file zeromode=emulate file=disk.img + nbdkit --filter=nozero file zeromode=emulate disk.img =head1 SEE ALSO diff --git a/filters/offset/nbdkit-offset-filter.pod b/filters/offset/nbdkit-offset-filter.pod index 61496d2..a232e46 100644 --- a/filters/offset/nbdkit-offset-filter.pod +++ b/filters/offset/nbdkit-o...

...things to prime the .can_write cache according to .open. The same script from the previous patch now fails a lot faster under -r (since nozero is not a filter that changes readonly=false to true, the plugin now fails .can_zero quickly): $ /bin/time -f %e ./nbdkit -U - -r --filter=nozero sh script zeromode=notrim \ --run 'qemu-nbd --list -k $unixsocket >/dev/null' nbdkit: sh[1]: error: zeromode 'notrim' requires plugin zero support qemu-nbd: Failed to read initial magic: Unexpected end-of-file before all bytes were read Command exited with non-zero status 1 0.06 Signed-off-by: E...

...ents:" cat nozero4.log || : + echo "Log 5a file contents:" + cat nozero5a.log || : + echo "Log 5b file contents:" + cat nozero5b.log || : rm -f $files exit $status @@ -88,6 +97,8 @@ trap cleanup INT QUIT TERM EXIT ERR # 2: log before filter with zeromode=none (default), to ensure no ZERO request # 3: log before filter with zeromode=emulate, to ensure ZERO from client # 4: log after filter with zeromode=emulate, to ensure no ZERO to plugin +# 5a/b: both sides of nbd plugin: even though server side does not advertise +# ZERO, the client side still...

Another couple things I noticed that are worth improving, but aren't strictly related to implementing fast zero support. Eric Blake (2): server: Assert sane error responses nozero: More efficient FUA handling filters/nozero/nozero.c | 17 +++++++++++-- server/filters.c | 56 +++++++++++++++++++++++++++++++++-------- server/protocol.c | 32 +++++++++++++++++------ 3 files

Remove the need to use file= (and in future other) parameters for many plugins. eg. Using the file plugin becomes: nbdkit file disk.img Rich.

v1 was here: https://www.redhat.com/archives/libguestfs/2018-September/msg00024.html v2: - As discussed in the patch review, tighten up the characters permitted in keys. - Update documentation to note that relative paths can be made safe by prefixing with ./ and absolute paths do not need any extra steps. - I pushed patch 1/6 from the v1 series since it was just a trivial

v1: https://www.redhat.com/archives/libguestfs/2018-September/msg00024.html v2: https://www.redhat.com/archives/libguestfs/2018-September/msg00034.html v3: - Fixed is_config_key zero length test. - Fixed is_config_key so it uses strspn and is not O(n^2). - Changed >= 1.7 to >= 1.8 in the documentation. Rich.

...behavior for such code to happen). diff --git i/filters/nozero/nozero.c w/filters/nozero/nozero.c index 16ec710b..21863707 100644 --- i/filters/nozero/nozero.c +++ w/filters/nozero/nozero.c @@ -45,7 +45,6 @@ #define MAX_WRITE (64 * 1024 * 1024) -static const char buffer[MAX_WRITE]; static enum ZeroMode { NONE, EMULATE, @@ -111,6 +110,8 @@ nozero_zero (struct nbdkit_next_ops *next_ops, void *nxdata, while (count) { uint32_t size = MIN (count, MAX_WRITE); + static const char buffer[MAX_WRITE]; /* Always contains zeroes */ + if (next_ops->pwrite (nxdata, buffer, size, offs,...

...nbd destination. Then I use a series of nbdkit > filters to force the destination to behave in various manners: > log logfile=>(sed ...|uniq -c) (track how many normal/fast zero > requests the client makes) > nozero $params (fine-tune how zero requests behave - the parameters > zeromode and fastzeromode are the real drivers of my various tests) > blocksize maxdata=256k (allows large zero requests, but forces large > writes into smaller chunks, to magnify the effects of write delays and > allow testing to provide obvious results with a smaller image) > delay delay-wri...

Hi all, It seems GCC 9 does not put read-only zero-initialized variables into the BSS section anymore; instead it is put into RODATA. (See the thread at [0], especially [1]) In filter/nozero a 64M large, static, zeroed, read-only array is declared. The new behavior of GCC puts this array as-is into the binary inflating the size by a factor of around 10000. (Clang 8 and older GCCs work

...test-nozero.sh @@ -31,6 +31,7 @@ # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF # SUCH DAMAGE. +source functions.sh set -e files="nozero1.img nozero1.log nozero1.sock nozero1.pid @@ -101,40 +102,24 @@ trap cleanup INT QUIT TERM EXIT ERR # 4: log after filter with zeromode=emulate, to ensure no ZERO to plugin # 5a/b: both sides of nbd plugin: even though server side does not advertise # ZERO, the client side still exposes it, and just skips calling nbd's .zero -nbdkit -P nozero1.pid -U nozero1.sock --filter=log \ +start_nbdkit -P nozero1.pid -U nozero1.sock --f...

...st "$pid5b" && kill $pid5b - # For easier debugging, dump the final log files before removing them. echo "Log 1 file contents:" cat nozero1.log || : echo "Log 2 file contents:" @@ -96,41 +88,19 @@ cleanup_fn cleanup # 4: log after filter with zeromode=emulate, to ensure no ZERO to plugin # 5a/b: both sides of nbd plugin: even though server side does not advertise # ZERO, the client side still exposes it, and just skips calling nbd's .zero -nbdkit -P nozero1.pid -U nozero1.sock --filter=log \ +start_nbdkit -P nozero1.pid -U nozero1.sock --f...

Combine much common code into tests/functions.sh. Patch 1: Preparation for patch 3. Patch 2: Fix a long-standing bug in how man pages links are generated. Patch 3: Common code for iterating a test function over every plugin. Patch 4: Common code for starting nbdkit in a test and waiting for the PID file to appear. This is the largest and most complex of the patches but is

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3]

v1 was here: https://www.redhat.com/archives/libguestfs/2018-September/msg00057.html v2: - Fix tab vs spaces in configure.ac. - To generate list of plugins, use printf instead of xargs. - Use 'source ./functions.sh' instead of 'source functions'. - functions.sh: Consistent quoting in foreach_plugin function. - functions.sh: Change the contract of start_nbdkit so it