similar to: [PATCH 1/2] file: Add missing include for FALLOC_FL_*

On RHEL 7.5 we need to include <linux/falloc.h> for FALLOC_FL_* macros. Without the macros, fallocate is never used and we fall back to manual zeroing. Here are examples runs with this change with a local file on ext4: $ export SOCK=/tmp/nbd.sock $ export FILE=/var/tmp/nbd.img $ export BLOCK=/dev/loop2 $ src/nbdkit -f plugins/file/.libs/nbdkit-file-plugin.so file=$FILE -U $SOCK $ time

If we may not trim, we tried ZERO_RANGE, but this is not well supported yet, for example it is not available on NFS 4.2. ZERO_RANGE and PUNCH_HOLE are supported now on block devices, but not on RHRL 7, so we fallback to slow manual zeroing there. Change the logic to support block devices on RHEL 7, and file systems that do not support ZERO_RANGE. The new logic: - If we may trim, try PUNCH_HOLE -

On 08/02/2018 02:05 PM, Nir Soffer wrote: > fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with > modern kernel, but when it is not, we fall back to manual zeroing. > > Check if the underlying file is a block device when opening the file, > and fall back to ioctl(BLKZEROOUT) for aligned zero requests for a > block device. > > +++ b/plugins/file/file.c >

This is the second version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v1: - Split to smaller patches - Skip linux only includes on other systems - Skip code using BLKZEROOUT if the macro is not defined - Try BLKZEROOUT only if the offset and count are aligned to device

On 07/30/2018 01:04 PM, Nir Soffer wrote: > On RHEL 7.5 we need to include <linux/falloc.h> for FALLOC_FL_* macros. Rather, on any Linux system that pre-dates glibc 2.18, where the flags were finally supported directly in <fcntl.h> > Without the macros, fallocate is never used and we fall back to manual > zeroing. > > +++ b/plugins/file/file.c > @@ -42,6 +42,10

On 07/29/2018 06:35 AM, Nir Soffer wrote: > On RHEL 7.5 we need to include <linux/falloc.h> for FALLOC_FL_* macros. > Without the macros, fallocate is never used and we fall back to manual > zeroing. > > @@ -41,6 +41,7 @@ > #include <sys/types.h> > #include <sys/stat.h> > #include <errno.h> > +#include <linux/falloc.h> /* For

This is the third version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v2: - Revert file_can_trim change, since it is too late to change the value after negotiation. Changing the capability dinamically may be useful internally, but it should be done via other means. -

If we may not trim, we tried ZERO_RANGE, but this is not well supported yet, for example it is not available on NFS 4.2. ZERO_RANGE and PUNCH_HOLE are supported now on block devices, but not on RHRL 7, so we fallback to slow manual zeroing there. Change the logic to support block devices on RHEL 7, and file systems that do not support ZERO_RANGE. The new logic: - If we may trim, try PUNCH_HOLE -

This version addresses comments on v3. Changes since v3: - Finally got spacing right (Eric) - Reorder includes (Richard) - Return 0 or -1 instead of r (Richard) - Add common/include/isaligned.h to Makefile.am (Richard) v3 was here: https://www.redhat.com/archives/libguestfs/2018-August/msg00177.html Nir Soffer (4): file: Avoid unsupported fallocate() calls file: Support zero without

This version addresses some of the comments on v2. Changes since v2: - file_zero: Add missing space in function call - is_aligned: Assert that align is indeed a power of 2 - Spelling in commit message Not changed: - Eric commented that spacing was off: https://www.redhat.com/archives/libguestfs/2018-August/msg00113.html but I could not find anything wrong. - Eric asked if ioctl.h will cause

There's a lot of code in nbdkit-file-plugin which it would be nice to reuse elsewhere. One possible approach (as outlined here) is simply to move the file callbacks (like file.pread, file.pwrite, file.zero etc) to a new mini-library. They can then be consumed by other plugins fairly easily by doing: static void * foo_open (int readonly) { struct fileops *fops; int fd, flags; /*

On Thu, Aug 2, 2018 at 10:39 PM Eric Blake <eblake@redhat.com> wrote: > On 08/02/2018 02:05 PM, Nir Soffer wrote: > > fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with > > modern kernel, but when it is not, we fall back to manual zeroing. > > > > Check if the underlying file is a block device when opening the file, > > and fall back to

On 07/29/2018 07:04 AM, Nir Soffer wrote: > If we may not trim, we tried ZERO_RANGE, but this is not well supported > yet, for example it is not available on NFS 4.2. ZERO_RANGE and > PUNCH_HOLE are supported now on block devices, but not on RHRL 7, so we > fallback to slow manual zeroing there. > > Change the logic to support block devices on RHEL 7, and file systems > that

While it may be counterintuitive at first, the introduction of NBD_CMD_WRITE_ZEROES and NBD_CMD_BLOCK_STATUS has caused a performance regression in qemu [1], when copying a sparse file. When the destination file must contain the same contents as the source, but it is not known in advance whether the destination started life with all zero content, then there are cases where it is faster to request

Fix issues Eric found in the original patch: https://www.redhat.com/archives/libguestfs/2018-July/msg00072.html - When handling ENODEV, the caller is expecting ENOTSUPP to trigger fallback. - ENODEV should be ignored in file_trim. Tested only on Fedora 28. --- plugins/file/file.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git

On 08/03/2018 02:28 PM, Nir Soffer wrote: > File systems not supporting FALLOC_FL_ZERO_RANGE yet fall back to manual > zeroing. > > We can avoid this by combining two fallocate calls: > > fallocate(FALLOC_FL_PUNCH_HOLE) > fallocate(0) > > Based on my tests this is much more efficient compared to manual > zeroing. The idea came from this qemu patch: >

We're currently support two paths from VFS to preallocate unwritten extents(from FS_IOC_RESVSP, or fallocate()), likewise, behavior of punching-hole should be treated as the same, this patch tries to teach file_ioctl() to handle FS_IOC_UNRESVSP, underlying filesystem like ocfs2 is wise enough to do the rest of work;-) Signed-off-by: Tristan Ye <tristan.ye at oracle.com> --- fs/ioctl.c

We're currently support two paths from VFS to preallocate unwritten extents(from FS_IOC_RESVSP, or fallocate()), likewise, behavior of punching-hole should be treated as the same, this patch tries to teach file_ioctl() to handle FS_IOC_UNRESVSP, underlying filesystem like ocfs2 is wise enough to do the rest of work;-) Signed-off-by: Tristan Ye <tristan.ye at oracle.com> --- fs/ioctl.c

Needs some work still, see in particular the commit message for patch 3. Rich.

fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with modern kernel, but when it is not, we fall back to manual zeroing. Check if the underlying file is a block device when opening the file, and fall back to ioctl(BLKZEROOUT) for aligned zero requests for a block device. Here is an example run without this change on RHEL 7.5: $ export SOCK=/tmp/nbd.sock $ export