similar to: [nbdkit PATCH] file: Add trim support

Upgrade this plugin to version 2 of the API. The main change is implementing FUA support. --- plugins/file/file.c | 62 ++++++++++++++++++++++++++++++--------------- 1 file changed, 42 insertions(+), 20 deletions(-) diff --git a/plugins/file/file.c b/plugins/file/file.c index 2a8c9b5..dcff0ee 100644 --- a/plugins/file/file.c +++ b/plugins/file/file.c @@ -52,6 +52,8 @@ #include

First thing to say is that I need to do a *lot* more testing on this, so this is just an early peek. In particular, although it passed ‘make check && make check-valgrind’ I have *not* tested it against a multi-conn-aware client such as the Linux kernel >= 4.9. This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc as: "NBD_FLAG_CAN_MULTI_CONN: Indicates that

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11]

On 08/02/2018 02:05 PM, Nir Soffer wrote: > When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or > block device on kernel < 4.9, we used to call fallocate() for every > zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When > trimming, we used to try unsupported fallocate() on every call. > > Change file handle to remember if punching holes or

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - can_trim changed

This is the second version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v1: - Split to smaller patches - Skip linux only includes on other systems - Skip code using BLKZEROOUT if the macro is not defined - Try BLKZEROOUT only if the offset and count are aligned to device

Writing "file-like" plugins is hard because you have to implement your own .zero, .trim, .extents, etc, and that is very complicated. However implementations of these functions already exist in the file plugin. By factoring out the file plugin into a separate "fileops" mini-library we can reuse these implementations in other plugins. This refactoring commit creates a new

This is the third version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v2: - Revert file_can_trim change, since it is too late to change the value after negotiation. Changing the capability dinamically may be useful internally, but it should be done via other means. -

There's a lot of code in nbdkit-file-plugin which it would be nice to reuse elsewhere. One possible approach (as outlined here) is simply to move the file callbacks (like file.pread, file.pwrite, file.zero etc) to a new mini-library. They can then be consumed by other plugins fairly easily by doing: static void * foo_open (int readonly) { struct fileops *fops; int fd, flags; /*

Needs some work still, see in particular the commit message for patch 3. Rich.

On Fri, Jul 27, 2018 at 04:23:10PM -0500, Eric Blake wrote: > On 07/27/2018 04:03 PM, Nir Soffer wrote: > >When using block device on RHEL 7.5, file plugin fails to zero with this > >error (copied from strace): > > > >[pid 39551] fallocate(8, FALLOC_FL_ZERO_RANGE, 1536, 64000) = -1 ENODEV (No such device) > > > >This is expected error according to the manual:

When using block device on RHEL 7.5, file plugin fails to zero with this error (copied from strace): [pid 39551] fallocate(8, FALLOC_FL_ZERO_RANGE, 1536, 64000) = -1 ENODEV (No such device) This is expected error according to the manual: ENODEV fd does not refer to a regular file or a directory. (If fd is a pipe or FIFO, a different error results.) Treat this error as EOPNOSUPP. Tested only

Fix issues Eric found in the original patch: https://www.redhat.com/archives/libguestfs/2018-July/msg00072.html - When handling ENODEV, the caller is expecting EOPNOTSUPP to trigger fallback. - ENODEV should be ignored in file_trim. Tested only on Fedora 28 and RHEL 7.5. --- plugins/file/file.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff

Fix issues Eric found in the original patch: https://www.redhat.com/archives/libguestfs/2018-July/msg00072.html - When handling ENODEV, the caller is expecting ENOTSUPP to trigger fallback. - ENODEV should be ignored in file_trim. Tested only on Fedora 28. --- plugins/file/file.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git

On Thu, Aug 2, 2018 at 10:30 PM Eric Blake <eblake@redhat.com> wrote: > On 08/02/2018 02:05 PM, Nir Soffer wrote: > > When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or > > block device on kernel < 4.9, we used to call fallocate() for every > > zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When > > trimming, we used to try

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - zero changed to:

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - zero changed to:

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - zero changed to:

This version addresses some of the comments on v2. Changes since v2: - file_zero: Add missing space in function call - is_aligned: Assert that align is indeed a power of 2 - Spelling in commit message Not changed: - Eric commented that spacing was off: https://www.redhat.com/archives/libguestfs/2018-August/msg00113.html but I could not find anything wrong. - Eric asked if ioctl.h will cause

On 07/27/2018 04:03 PM, Nir Soffer wrote: > When using block device on RHEL 7.5, file plugin fails to zero with this > error (copied from strace): > > [pid 39551] fallocate(8, FALLOC_FL_ZERO_RANGE, 1536, 64000) = -1 ENODEV (No such device) > > This is expected error according to the manual: > > ENODEV fd does not refer to a regular file or a directory. (If fd is a >