similar to: [PATCH] file: Fix zero/trim with block device

On Fri, Jul 27, 2018 at 04:23:10PM -0500, Eric Blake wrote: > On 07/27/2018 04:03 PM, Nir Soffer wrote: > >When using block device on RHEL 7.5, file plugin fails to zero with this > >error (copied from strace): > > > >[pid 39551] fallocate(8, FALLOC_FL_ZERO_RANGE, 1536, 64000) = -1 ENODEV (No such device) > > > >This is expected error according to the manual:

Fix issues Eric found in the original patch: https://www.redhat.com/archives/libguestfs/2018-July/msg00072.html - When handling ENODEV, the caller is expecting ENOTSUPP to trigger fallback. - ENODEV should be ignored in file_trim. Tested only on Fedora 28. --- plugins/file/file.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff --git

Fix issues Eric found in the original patch: https://www.redhat.com/archives/libguestfs/2018-July/msg00072.html - When handling ENODEV, the caller is expecting EOPNOTSUPP to trigger fallback. - ENODEV should be ignored in file_trim. Tested only on Fedora 28 and RHEL 7.5. --- plugins/file/file.c | 33 ++++++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 9 deletions(-) diff

This is the second version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v1: - Split to smaller patches - Skip linux only includes on other systems - Skip code using BLKZEROOUT if the macro is not defined - Try BLKZEROOUT only if the offset and count are aligned to device

This is the third version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v2: - Revert file_can_trim change, since it is too late to change the value after negotiation. Changing the capability dinamically may be useful internally, but it should be done via other means. -

This version addresses comments on v3. Changes since v3: - Finally got spacing right (Eric) - Reorder includes (Richard) - Return 0 or -1 instead of r (Richard) - Add common/include/isaligned.h to Makefile.am (Richard) v3 was here: https://www.redhat.com/archives/libguestfs/2018-August/msg00177.html Nir Soffer (4): file: Avoid unsupported fallocate() calls file: Support zero without

This version addresses some of the comments on v2. Changes since v2: - file_zero: Add missing space in function call - is_aligned: Assert that align is indeed a power of 2 - Spelling in commit message Not changed: - Eric commented that spacing was off: https://www.redhat.com/archives/libguestfs/2018-August/msg00113.html but I could not find anything wrong. - Eric asked if ioctl.h will cause

On 07/27/2018 04:03 PM, Nir Soffer wrote: > When using block device on RHEL 7.5, file plugin fails to zero with this > error (copied from strace): > > [pid 39551] fallocate(8, FALLOC_FL_ZERO_RANGE, 1536, 64000) = -1 ENODEV (No such device) > > This is expected error according to the manual: > > ENODEV fd does not refer to a regular file or a directory. (If fd is a >

There's a lot of code in nbdkit-file-plugin which it would be nice to reuse elsewhere. One possible approach (as outlined here) is simply to move the file callbacks (like file.pread, file.pwrite, file.zero etc) to a new mini-library. They can then be consumed by other plugins fairly easily by doing: static void * foo_open (int readonly) { struct fileops *fops; int fd, flags; /*

On 08/02/2018 02:05 PM, Nir Soffer wrote: > When using file systems not supporting ZERO_RANGE (e.g. NFS 4.2) or > block device on kernel < 4.9, we used to call fallocate() for every > zero, fail with EOPNOTSUPP, and fallback to manual zeroing. When > trimming, we used to try unsupported fallocate() on every call. > > Change file handle to remember if punching holes or

Needs some work still, see in particular the commit message for patch 3. Rich.

We already have support in the file driver for punching holes during .zero with the may_trim flag (via FALLOC_FL_PUNCH_HOLE), so we should use the same mechanism to support .trim. Note that the NBD spec says that trim is advisory (we can return success even if we did nothing); but at the same time, it's nicer to avoid advertising the feature if we know for sure we can't do it, so we also

On 08/02/2018 02:05 PM, Nir Soffer wrote: > fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with > modern kernel, but when it is not, we fall back to manual zeroing. > > Check if the underlying file is a block device when opening the file, > and fall back to ioctl(BLKZEROOUT) for aligned zero requests for a > block device. > > +++ b/plugins/file/file.c >

If we may not trim, we tried ZERO_RANGE, but this is not well supported yet, for example it is not available on NFS 4.2. ZERO_RANGE and PUNCH_HOLE are supported now on block devices, but not on RHRL 7, so we fallback to slow manual zeroing there. Change the logic to support block devices on RHEL 7, and file systems that do not support ZERO_RANGE. The new logic: - If we may trim, try PUNCH_HOLE -

Fix some things I noticed while reviewing v1, and follow Rich's idea to add a new nbdkit_set_error() utility function with a binding for Python users to request a particular error (rather than being forced to live with whatever stale value is in errno after all the intermediate binding glue code). I could not easily find out how to register a C function callable from perl bindings, and have

On Tue, Apr 10, 2018 at 02:07:33PM +0000, Nir Soffer wrote: > This makes sense if the device is backed by a block device on oVirt side, > and the NBD support efficient zeroing. But in this case the device is backed > by an empty sparse file on NFS, and oVirt does not support yet efficient > zeroing, we just write zeros manually. > > I think should be handled on virt-v2v plugin

The upstream protocol recently promoted NBD_CMD_WRITE_ZEROES from experimental to a documented extension. Exposing support for this allows plugin writers to create sparse files when driven by a client that knows how to use the extension; meanwhile, even if a plugin does not support this extension, the server benefits from less network traffic from the client. Eric Blake (5): protocol: Support

When changing a file size with fallocate() the new size isn't being checked. In particular, the FSIZE ulimit isn't being checked, which makes fstest generic/228 fail. Simply adding a call to inode_newsize_ok() fixes this issue. Signed-off-by: Lu?s Henriques <lhenriques at suse.de> --- fs/ocfs2/file.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git

We're currently support two paths from VFS to preallocate unwritten extents(from FS_IOC_RESVSP, or fallocate()), likewise, behavior of punching-hole should be treated as the same, this patch tries to teach file_ioctl() to handle FS_IOC_UNRESVSP, underlying filesystem like ocfs2 is wise enough to do the rest of work;-) Signed-off-by: Tristan Ye <tristan.ye at oracle.com> --- fs/ioctl.c

We're currently support two paths from VFS to preallocate unwritten extents(from FS_IOC_RESVSP, or fallocate()), likewise, behavior of punching-hole should be treated as the same, this patch tries to teach file_ioctl() to handle FS_IOC_UNRESVSP, underlying filesystem like ocfs2 is wise enough to do the rest of work;-) Signed-off-by: Tristan Ye <tristan.ye at oracle.com> --- fs/ioctl.c