similar to: [PATCH v2 0/4] file: Zero for block devices and older file systems

This version addresses comments on v3. Changes since v3: - Finally got spacing right (Eric) - Reorder includes (Richard) - Return 0 or -1 instead of r (Richard) - Add common/include/isaligned.h to Makefile.am (Richard) v3 was here: https://www.redhat.com/archives/libguestfs/2018-August/msg00177.html Nir Soffer (4): file: Avoid unsupported fallocate() calls file: Support zero without

This version addresses some of the comments on v2. Changes since v2: - file_zero: Add missing space in function call - is_aligned: Assert that align is indeed a power of 2 - Spelling in commit message Not changed: - Eric commented that spacing was off: https://www.redhat.com/archives/libguestfs/2018-August/msg00113.html but I could not find anything wrong. - Eric asked if ioctl.h will cause

This is the second version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v1: - Split to smaller patches - Skip linux only includes on other systems - Skip code using BLKZEROOUT if the macro is not defined - Try BLKZEROOUT only if the offset and count are aligned to device

There's a lot of code in nbdkit-file-plugin which it would be nice to reuse elsewhere. One possible approach (as outlined here) is simply to move the file callbacks (like file.pread, file.pwrite, file.zero etc) to a new mini-library. They can then be consumed by other plugins fairly easily by doing: static void * foo_open (int readonly) { struct fileops *fops; int fd, flags; /*

If we may not trim, we tried ZERO_RANGE, but this is not well supported yet, for example it is not available on NFS 4.2. ZERO_RANGE and PUNCH_HOLE are supported now on block devices, but not on RHRL 7, so we fallback to slow manual zeroing there. Change the logic to support block devices on RHEL 7, and file systems that do not support ZERO_RANGE. The new logic: - If we may trim, try PUNCH_HOLE -

Needs some work still, see in particular the commit message for patch 3. Rich.

On 08/02/2018 02:05 PM, Nir Soffer wrote: > When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or > block device on kernel < 4.9, we used to call fallocate() for every > zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When > trimming, we used to try unsupported fallocate() on every call. > > Change file handle to remember if punching holes or

Fix issues Eric found in the original patch: https://www.redhat.com/archives/libguestfs/2018-July/msg00072.html - When handling ENODEV, the caller is expecting ENOTSUPP to trigger fallback. - ENODEV should be ignored in file_trim. Tested only on Fedora 28. --- plugins/file/file.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git

On Sun, Aug 19, 2018 at 01:13:05AM +0300, Nir Soffer wrote: > When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or > block device on kernel < 4.9, we used to call fallocate() for every > zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When > trimming, we used to try unsupported fallocate() on every call. > > Change file handle to remember if

If we may not trim, we tried ZERO_RANGE, but this is not well supported yet, for example it is not available on NFS 4.2. ZERO_RANGE and PUNCH_HOLE are supported now on block devices, but not on RHRL 7, so we fallback to slow manual zeroing there. Change the logic to support block devices on RHEL 7, and file systems that do not support ZERO_RANGE. The new logic: - If we may trim, try PUNCH_HOLE -

On 08/03/2018 02:28 PM, Nir Soffer wrote: > When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or > block device on kernel < 4.9, we used to call fallocate() for every > zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When > trimming, we used to try unsupported fallocate() on every call. > > Change file handle to remember if punching holes or

Fix issues Eric found in the original patch: https://www.redhat.com/archives/libguestfs/2018-July/msg00072.html - When handling ENODEV, the caller is expecting EOPNOTSUPP to trigger fallback. - ENODEV should be ignored in file_trim. Tested only on Fedora 28 and RHEL 7.5. --- plugins/file/file.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff

Writing "file-like" plugins is hard because you have to implement your own .zero, .trim, .extents, etc, and that is very complicated. However implementations of these functions already exist in the file plugin. By factoring out the file plugin into a separate "fileops" mini-library we can reuse these implementations in other plugins. This refactoring commit creates a new

File systems not supporting FALLOC_FL_ZERO_RANGE yet fall back to manual zeroing. We can avoid this by combining two fallocate calls: fallocate(FALLOC_FL_PUNCH_HOLE) fallocate(0) Based on my tests this is much more efficient compared to manual zeroing. The idea came from this qemu patch: https://github.com/qemu/qemu/commit/1cdc3239f1bb Here is an example run with NFS 4.2 without this

File systems not supporting FALLOC_FL_ZERO_RANGE yet fall back to manual zeroing. We can avoid this by combining two fallocate calls: fallocate(FALLOC_FL_PUNHCH_HOLE) fallocate(0) Based on my tests this is much more efficient compared to manual zeroing. The idea came from this qemu patch: https://github.com/qemu/qemu/commit/1cdc3239f1bb Here is an example run with NFS 4.2 without this

On Mon, Aug 13, 2018 at 11:58 PM Nir Soffer <nsoffer@redhat.com> wrote: > On Mon, Aug 13, 2018 at 8:44 PM Eric Blake <eblake@redhat.com> wrote: > >> > #ifdef FALLOC_FL_ZERO_RANGE >> > - r = do_fallocate (h->fd, FALLOC_FL_ZERO_RANGE, offset, count); >> > - if (r == -1 && errno != EOPNOTSUPP) { >> > - nbdkit_error ("zero:

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - zero changed to:

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - zero changed to:

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - zero changed to:

When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or block device on kernel < 4.9, we used to call fallocate() for every zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When trimming, we used to try unsupported fallocate() on every call. Change file handle to remember if punching holes or zeroing range are supported, and avoid unsupported calls. - can_trim changed