search for: nozero

...-by: Eric Blake <eblake@redhat.com> --- TODO | 2 +- configure.ac | 3 +- docs/nbdkit-filter.pod | 9 ++++ docs/nbdkit.pod | 1 + filters/Makefile.am | 1 + filters/nozero/Makefile.am | 62 ++++++++++++++++++++++++ filters/nozero/nbdkit-nozero-filter.pod | 84 +++++++++++++++++++++++++++++++++ filters/nozero/nozero.c | 68 ++++++++++++++++++++++++++ 8 files changed, 228 insertions(+), 2 deletions(-) create mode 100644 filters/nozero/Make...

It may be useful to test whether the client's use of NBD_CMD_FLAG_NO_HOLE makes a difference; do this by adding a mode to --filter=nozero to force a non-trimming zero write. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/nozero/nbdkit-nozero-filter.pod | 19 +++++++---- filters/nozero/nozero.c | 45 ++++++++++++++++++++----- tests/test-nozero.sh | 27 +++++++++++++-- 3 files chang...

Hi all, It seems GCC 9 does not put read-only zero-initialized variables into the BSS section anymore; instead it is put into RODATA. (See the thread at [0], especially [1]) In filter/nozero a 64M large, static, zeroed, read-only array is declared. The new behavior of GCC puts this array as-is into the binary inflating the size by a factor of around 10000. (Clang 8 and older GCCs work "correctly") It would be possible to remove the "const" qualifier or use some co...

On Fri, 2019-07-05T08:04-0500, Eric Blake wrote: > On 7/4/19 1:38 PM, Thomas Weißschuh wrote: > > In filter/nozero a 64M large, static, zeroed, read-only array is declared. > > The new behavior of GCC puts this array as-is into the binary inflating the > > size by a factor of around 10000. > > Odd that you are only seeing this for the nozero filter, since we also > have the same sort of bu...

...dd testsuite coverage. Perhaps it might be easier > if I create a new '--filter=log logfile=foo' filter that produces > a log of which commands a client sent, then compare the log using > a known client that uses write_zeroes (qemu-io works well) both > with and without --filter=nozero to prove that the change in > advertisement changes the commands sent over the wire (that would > also test the log filter at the same time...). > > I'll wait for a review on this series, particularly since patch 2 > intentionally breaks filter ABI (more ABI/API breaks on the wa...

The nozero filter can split a large request with FUA into several smaller requests; optimize whether the FUA flag is passed on to the next layer based on whether FUA is emulated with flush (only the last write needs it) or is natively supported (every write needs it). Missed in commit df0cc21d. Signed-off-by...

I still need to add testsuite coverage. Perhaps it might be easier if I create a new '--filter=log logfile=foo' filter that produces a log of which commands a client sent, then compare the log using a known client that uses write_zeroes (qemu-io works well) both with and without --filter=nozero to prove that the change in advertisement changes the commands sent over the wire (that would also test the log filter at the same time...). I'll wait for a review on this series, particularly since patch 2 intentionally breaks filter ABI (more ABI/API breaks on the way when I resubmit my FUA...

On 7/4/19 1:38 PM, Thomas Weißschuh wrote: > Hi all, > > It seems GCC 9 does not put read-only zero-initialized variables into the BSS > section anymore; instead it is put into RODATA. > (See the thread at [0], especially [1]) > Thanks for the pointers. > In filter/nozero a 64M large, static, zeroed, read-only array is declared. > The new behavior of GCC puts this array as-is into the binary inflating the > size by a factor of around 10000. Odd that you are only seeing this for the nozero filter, since we also have the same sort of buffer in server/plugins.c....

...his extension by quite a few years, so we have to be able to fall back to a plain macro. --- configure.ac | 20 ++++++++++- common/include/minmax.h | 63 +++++++++++++++++++++++++++++++++++ filters/blocksize/blocksize.c | 3 +- filters/cache/lru.c | 3 +- filters/nozero/nozero.c | 3 +- plugins/pattern/pattern.c | 3 +- common/include/Makefile.am | 1 + filters/blocksize/Makefile.am | 3 +- filters/nozero/Makefile.am | 3 +- 9 files changed, 93 insertions(+), 9 deletions(-) diff --git a/configure.ac b/configure.ac index 606e632..ee1ff0d 10064...

...his extension by quite a few years, so we have to be able to fall back to a plain macro. --- configure.ac | 20 ++++++++++- common/include/minmax.h | 63 +++++++++++++++++++++++++++++++++++ filters/blocksize/blocksize.c | 3 +- filters/cache/lru.c | 3 +- filters/nozero/nozero.c | 3 +- plugins/pattern/pattern.c | 3 +- common/include/Makefile.am | 1 + filters/blocksize/Makefile.am | 3 +- filters/nozero/Makefile.am | 3 +- 9 files changed, 93 insertions(+), 9 deletions(-) diff --git a/configure.ac b/configure.ac index 606e632..ee1ff0d 10064...

Another couple things I noticed that are worth improving, but aren't strictly related to implementing fast zero support. Eric Blake (2): server: Assert sane error responses nozero: More efficient FUA handling filters/nozero/nozero.c | 17 +++++++++++-- server/filters.c | 56 +++++++++++++++++++++++++++++++++-------- server/protocol.c | 32 +++++++++++++++++------ 3 files changed, 84 insertions(+), 21 deletions(-) -- 2.20.1

Similar to the existing fua, nozero and noextents filters, add a filter to make it easy to override the basic caching functionality, in part to facilitate timing tests of whether a plugin's cache implementation is worthwhile. A worthwhile test to add to the testsuite would connect the log filter both before and after the nocache...

On 7/5/19 8:21 AM, Thomas Weißschuh wrote: > On Fri, 2019-07-05T08:04-0500, Eric Blake wrote: >> On 7/4/19 1:38 PM, Thomas Weißschuh wrote: >>> In filter/nozero a 64M large, static, zeroed, read-only array is declared. >>> The new behavior of GCC puts this array as-is into the binary inflating the >>> size by a factor of around 10000. >> >> Odd that you are only seeing this for the nozero filter, since we also >> have th...

...docs/nbdkit-filter.pod | 45 ++++++---- filters/fua/nbdkit-fua-filter.pod | 1 + filters/nocache/nbdkit-nocache-filter.pod | 1 + filters/noextents/nbdkit-noextents-filter.pod | 1 + .../noparallel/nbdkit-noparallel-filter.pod | 66 ++++++++++++++ filters/nozero/nbdkit-nozero-filter.pod | 1 + configure.ac | 2 + include/nbdkit-filter.h | 7 +- filters/cow/cow.c | 2 - filters/delay/delay.c | 2 - filters/error/error.c...

...- filters/cow/nbdkit-cow-filter.pod | 4 ++-- filters/delay/nbdkit-delay-filter.pod | 2 +- filters/error/nbdkit-error-filter.pod | 6 +++--- filters/fua/nbdkit-fua-filter.pod | 8 ++++---- filters/log/nbdkit-log-filter.pod | 4 ++-- filters/nozero/nbdkit-nozero-filter.pod | 4 ++-- filters/offset/nbdkit-offset-filter.pod | 4 ++-- filters/partition/nbdkit-partition-filter.pod | 2 +- plugins/file/file.c | 1 + plugins/file/nbdkit-file-plugin.pod | 5 ++++- tests/test-blocksize.sh...

...- filters/cow/nbdkit-cow-filter.pod | 4 ++-- filters/delay/nbdkit-delay-filter.pod | 2 +- filters/error/nbdkit-error-filter.pod | 6 +++--- filters/fua/nbdkit-fua-filter.pod | 8 ++++---- filters/log/nbdkit-log-filter.pod | 4 ++-- filters/nozero/nbdkit-nozero-filter.pod | 4 ++-- filters/offset/nbdkit-offset-filter.pod | 4 ++-- filters/partition/nbdkit-partition-filter.pod | 2 +- plugins/file/file.c | 1 + plugins/file/nbdkit-file-plugin.pod | 7 ++++++- tests/test-blocksize.sh...

On Fri, 2019-07-05T08:48-0500, Eric Blake wrote: > On 7/5/19 8:21 AM, Thomas Weißschuh wrote: > > On Fri, 2019-07-05T08:04-0500, Eric Blake wrote: > >> On 7/4/19 1:38 PM, Thomas Weißschuh wrote: > >>> In filter/nozero a 64M large, static, zeroed, read-only array is declared. > >>> The new behavior of GCC puts this array as-is into the binary inflating the > >>> size by a factor of around 10000. > >> > >> Odd that you are only seeing this for the nozero filter, since we a...

...-- filters/delay/delay.c | 2 -- filters/error/error.c | 2 -- filters/fua/fua.c | 2 -- filters/log/log.c | 2 -- filters/nocache/nocache.c | 2 -- filters/noextents/noextents.c | 2 -- filters/noparallel/noparallel.c | 2 -- filters/nozero/nozero.c | 2 -- filters/offset/offset.c | 2 -- filters/partition/partition.c | 2 -- filters/rate/rate.c | 2 -- filters/readahead/readahead.c | 2 -- filters/stats/stats.c | 2 -- filters/truncate/truncate.c | 2 -- filters/xz/xz.c...

On 7/5/19 9:52 AM, Thomas Weißschuh wrote: >> Good to know. So the fix is indeed removing 'const', rather than >> reducing scope (although I might as well reduce scope while at it, for >> consistency). > > Does this not mean, the 64M are zeroed on every loop iteration in every call to > these functions? No. static variables are guaranteed by the C standard to

...t use the .zero callback when it won't work. At the same time, we still want to advertise zero support to the client, even if we won't be calling .zero, since handling NBD_CMD_WRITE_ZEROES allows for less network traffic; if it is ever truly necessary to avoid advertising to the guest, the nozero filter can accomplish that task. So, this patch exposes a .can_zero callback for plugins, which can usually be omitted, but which can be used to short- circuit calls to .zero, with the nbd plugin as the first user; while plugins.c continues to report true (well, the same result as .can_write) to t...