search for: minblock

...100755 tests/test-blocksize.sh diff --git a/TODO b/TODO index a691ff3..d02671d 100644 --- a/TODO +++ b/TODO @@ -86,11 +86,6 @@ Suggestions for filters unneeded intermediate flushing; hence, where this filter is placed in the stack may have a performance impact. -* blocksize filter: setting minblock performs read-modify-write of - requests that are too small or unaligned for the plugin; setting - maxdata breaks up too-large read/write; setting maxlen breaks up - too-large trim/zero - nbdkit-cache-filter needs considerable work: * allow the user to limit the total size of the cache diff -...

...100755 tests/test-blocksize.sh diff --git a/TODO b/TODO index a691ff3..d02671d 100644 --- a/TODO +++ b/TODO @@ -86,11 +86,6 @@ Suggestions for filters unneeded intermediate flushing; hence, where this filter is placed in the stack may have a performance impact. -* blocksize filter: setting minblock performs read-modify-write of - requests that are too small or unaligned for the plugin; setting - maxdata breaks up too-large read/write; setting maxlen breaks up - too-large trim/zero - nbdkit-cache-filter needs considerable work: * allow the user to limit the total size of the cache diff -...

Based on Rich's review of my v1 that touched only cache.c, I have now tried to bring all three filters with alignment rounding in line with one another. There is definitely room for future improvements once we teach nbdkit to let filters and plugins advertise block sizes, but I'm hoping to get NBD_CMD_CACHE implemented first. Eric Blake (2): blocksize: Process requests in linear order

It is easy to demonstrate that our blocksize filter has a bug: if the minblock= setting is a higher granularity than the underlying plugin actually supports, and the client is trying to collect block status for the entire disk, a mid-block transition in the plugin can result in the filter rounding a request so small that it no longer makes progress, causing the client to see:...

Since v1: add the blocksize filter, add testsuite coverage of the log filter, several fixes to the log filter based on what adding tests revealed I'm still working on FUA flag support patches on top of this; the patches should all be committed in the same release, as we want to minimize the number of releases that cause a filter ABI/API bump Eric Blake (3): backend: Rework internal/filter

When using a filter that rounds up to alignment boundaries, the tail bytes of the plugin are difficult to access correctly. Rather than duplicating lots of code already in the truncate filter, it's easier to just make the other filters default to rounding down and add doc links on how to round up instead. Eric Blake (2): blocksize: Lift restriction against 0-size file cache, cow: Round

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3]

...maximum length for any single transaction without data (trim, zero +or extents). If omitted, this defaults to 0xffffffff rounded down to C<minsize> alignment (that is, the inherent 32-bit limit of the NBD protocol). This need not be a power of two, but must be an integer multiple of C<minblock>, and should be at least as large as diff --git a/filters/blocksize/blocksize.c b/filters/blocksize/blocksize.c index 9e0e713..55b2d59 100644 --- a/filters/blocksize/blocksize.c +++ b/filters/blocksize/blocksize.c @@ -45,6 +45,7 @@ #include <nbdkit-filter.h> #include "minmax.h&quo...

...cksize.c b/filters/blocksize/blocksize.c > index 34f58c9..9e0e713 100644 > --- a/filters/blocksize/blocksize.c > +++ b/filters/blocksize/blocksize.c > @@ -164,6 +164,17 @@ blocksize_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, > return size == -1 ? size : size & ~(minblock - 1); > } > > +static int > +blocksize_can_multi_conn (struct nbdkit_next_ops *next_ops, void *nxdata, > + void *handle) > +{ > + /* Although we are serializing all requests anyway so this likely > + * doesn't make a difference, return fals...

...t a/filters/blocksize/blocksize.c b/filters/blocksize/blocksize.c index 34f58c9..9e0e713 100644 --- a/filters/blocksize/blocksize.c +++ b/filters/blocksize/blocksize.c @@ -164,6 +164,17 @@ blocksize_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, return size == -1 ? size : size & ~(minblock - 1); } +static int +blocksize_can_multi_conn (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle) +{ + /* Although we are serializing all requests anyway so this likely + * doesn't make a difference, return false because the bounce buffer + * is not...

...--filter=log file logfile=blocksize1.log blocksize1.img > -nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ > +start_nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ > --filter=log file logfile=blocksize2.log blocksize2.img \ > minblock=1024 maxdata=512k maxlen=1M If the first process starts but the second fails, then you have not set pid1=, and leak the first process. This is a regression from the old code, which managed to capture pid1 before triggering the trap to cleanup(), and thus killed the successful nbdkit. Several...

...er=log file logfile=blocksize1.log blocksize1.img > >-nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ > >+start_nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ > > --filter=log file logfile=blocksize2.log blocksize2.img \ > > minblock=1024 maxdata=512k maxlen=1M > > If the first process starts but the second fails, then you have not > set pid1=, and leak the first process. This is a regression from > the old code, which managed to capture pid1 before triggering the > trap to cleanup(), and thus killed the succes...

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11]

...e.c +++ b/filters/blocksize/blocksize.c @@ -138,8 +138,6 @@ blocksize_config_complete (nbdkit_next_config_complete *next, void *nxdata) "maxdata=<SIZE> Maximum size for read/write (default 64M).\n" \ "maxlen=<SIZE> Maximum size for trim/zero (default 4G-minblock)." -/* TODO: Should we have a .prepare to cache per-connection FUA mode? */ - /* Round size down to avoid issues at end of file. */ static int64_t blocksize_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, diff --git a/filters/cache/cache.c b/filters/cache/cache.c index b5dbccd2.....

Combine much common code into tests/functions.sh. Patch 1: Preparation for patch 3. Patch 2: Fix a long-standing bug in how man pages links are generated. Patch 3: Common code for iterating a test function over every plugin. Patch 4: Common code for starting nbdkit in a test and waiting for the PID file to appear. This is the largest and most complex of the patches but is

I'm still working my way through the filters before this series will be complete, but this is enough of a start to at least get some feedback on the idea of implementing another NBD protocol extension. Eric Blake (9): server: Internal hooks for implementing NBD_CMD_CACHE plugins: Add .cache callback file, split: Implement .cache with posix_fadvise nbd: Implement NBD_CMD_CACHE

....log file=blocksize1.img + --filter=log file logfile=blocksize1.log blocksize1.img nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ - --filter=log file logfile=blocksize2.log file=blocksize2.img \ + --filter=log file logfile=blocksize2.log blocksize2.img \ minblock=1024 maxdata=512k maxlen=1M # We may have to wait a short time for the pid files to appear. diff --git a/tests/test-cache.sh b/tests/test-cache.sh index 6f058b8..215bab1 100755 --- a/tests/test-cache.sh +++ b/tests/test-cache.sh @@ -41,7 +41,7 @@ rm -f $files truncate -s 1G cache.img # Run n...

....log file=blocksize1.img + --filter=log file logfile=blocksize1.log blocksize1.img nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ - --filter=log file logfile=blocksize2.log file=blocksize2.img \ + --filter=log file logfile=blocksize2.log blocksize2.img \ minblock=1024 maxdata=512k maxlen=1M # We may have to wait a short time for the pid files to appear. diff --git a/tests/test-cache.sh b/tests/test-cache.sh index 6f058b8..215bab1 100755 --- a/tests/test-cache.sh +++ b/tests/test-cache.sh @@ -41,7 +41,7 @@ rm -f $files truncate -s 1G cache.img # Run n...

...id -U blocksize1.sock \ --filter=log file logfile=blocksize1.log blocksize1.img -nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ +start_nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \ --filter=log file logfile=blocksize2.log blocksize2.img \ minblock=1024 maxdata=512k maxlen=1M - -# We may have to wait a short time for the pid files to appear. -for i in `seq 1 10`; do - if test -f blocksize1.pid && test -f blocksize2.pid; then - break - fi - sleep 1 -done - -pid1="$(cat blocksize1.pid)" || : -pid2="$(cat b...

First thing to say is that I need to do a *lot* more testing on this, so this is just an early peek. In particular, although it passed ‘make check && make check-valgrind’ I have *not* tested it against a multi-conn-aware client such as the Linux kernel >= 4.9. This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc as: "NBD_FLAG_CAN_MULTI_CONN: Indicates that