search for: seek_hol

On 3/28/19 11:18 AM, Richard W.M. Jones wrote: > This uses lseek SEEK_DATA/SEEK_HOLE to search for allocated data and > holes in the underlying file. > --- > plugins/file/file.c | 141 ++++++++++++++++++++++++++++++++++--- > tests/Makefile.am | 5 ++ > tests/test-file-extents.sh | 57 +++++++++++++++ > 3 files changed, 193 insertions(+), 10 de...

...all shares on that host, unrelated to disk where qBittorrent is actually writing. On Arch Linux that smbd process is using 100% of one CPU core time and seems it's blocking because of lseek calls. strace shows full of lseek(47, 1420820480, SEEK_DATA) = 1421344768 lseek(47, 1421344768, SEEK_HOLE) = 1423441920 lseek(47, 1423441920, SEEK_DATA) = 1423966208 lseek(47, 1423966208, SEEK_HOLE) = 1425014784 lseek(47, 1425014784, SEEK_DATA) = 1425539072 lseek(47, 1425539072, SEEK_HOLE) = 1426063360 lseek(47, 1426063360, SEEK_DATA) = 1426587648 lseek(47, 14...

On Wed, Apr 24, 2019 at 02:39:03PM -0500, Eric Blake wrote: > On 3/28/19 11:18 AM, Richard W.M. Jones wrote: > > This uses lseek SEEK_DATA/SEEK_HOLE to search for allocated data and > > holes in the underlying file. > > --- > > plugins/file/file.c | 141 ++++++++++++++++++++++++++++++++++--- > > tests/Makefile.am | 5 ++ > > tests/test-file-extents.sh | 57 +++++++++++++++ > > 3 files cha...

Two patches follow this message. One fixes the default implementation of SEEK_HOLE/DATA. This patch applies atop Josef's last posted patch. The second patch implements the same on ocfs2. The test tool for the same is available here. http://oss.oracle.com/~smushran/seek_data/seek_test.c It is improved since the last post. It runs cleanly on zfs, ocfs2 and ext3 (default beha...

Two patches follow this message. One fixes the default implementation of SEEK_HOLE/DATA. This patch applies atop Josef's last posted patch. The second patch implements the same on ocfs2. The test tool for the same is available here. http://oss.oracle.com/~smushran/seek_data/seek_test.c It is improved since the last post. It runs cleanly on zfs, ocfs2 and ext3 (default beha...

On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > This uses lseek SEEK_DATA/SEEK_HOLE to search for allocated data and > holes in the underlying file. > --- > plugins/file/file.c | 139 ++++++++++++++++++++++++++++++++++++++++---- > 1 file changed, 129 insertions(+), 10 deletions(-) > > -int file_debug_zero; /* to enable: -D file.zero=1 */ > +/* A...

The NBD protocol is adding an extension to let servers advertise initialization state to the client: whether the image contains holes, and whether it is known to read as all zeroes. For file-based plugins, we are already probing lseek(SEEK_HOLE) to learn if extents are supported; a slight tweak to remember if that result is EOF tells us if we are sparse, and a similar lseek(SEEK_DATA) returning ENXIO tells us if the entire file is a hole, with at most two lseek calls during open (rather than one lseek for each of .can_extents, .init_spar...

This uses lseek SEEK_DATA/SEEK_HOLE to search for allocated data and holes in the underlying file. --- plugins/file/file.c | 139 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 129 insertions(+), 10 deletions(-) diff --git a/plugins/file/file.c b/plugins/file/file.c index 628f8fb..22fdfcd 100644 --- a/plugins/file/fi...

By referring to http://linux.die.net/man/2/lseek, return ENXIO only when "offset beyond EOF" for either SEEK_DATA or SEEK_HOLE inquiry. But we return it in case of internal issue too if btrfs_get_extent_fiemap() failed due to other issues. This will confuse the user applications to be expecting ENXIO when trying to find a specific data or hole location once it has occurred. Thanks Dave for pointing that out in XFS threa...

Author: perrin Repository: /hg/zfs-crypto/gate Revision: 39a587f9abc6273f4188996d9e4de4e0ea126efa Log message: 6303398 lseek(., offset, SEEK_HOLE) asserts with tainted offsets Files: update: usr/src/uts/common/fs/ufs/ufs_bmap.c

This uses lseek SEEK_DATA/SEEK_HOLE to search for allocated data and holes in the underlying file. --- plugins/file/file.c | 141 ++++++++++++++++++++++++++++++++++--- tests/Makefile.am | 5 ++ tests/test-file-extents.sh | 57 +++++++++++++++ 3 files changed, 193 insertions(+), 10 deletions(-) diff --git a/plugi...

...at 03:43:58PM -0600, Eric Blake wrote: > @@ -214,6 +217,52 @@ file_open (int readonly) > h->can_fallocate = true; > h->can_zeroout = h->is_block_device; > > + h->can_extents = false; > + h->init_sparse = false; > + h->init_zero = false; > +#ifdef SEEK_HOLE > + if (!h->is_block_device) { > + off_t r; > + > + /* A simple test to see whether SEEK_DATA/SEEK_HOLE are likely to work on > + * the current filesystem, and to see if the image is sparse or zero. > + */ > + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lseek_loc...

Copies somewhat from the file plugin, with the difference that we always provide an .extents even if one or more of the split files does not support SEEK_HOLE. Testing is possible on a file system that supports sparse files when using nbdsh. Signed-off-by: Eric Blake <eblake@redhat.com> --- I'm going ahead and committing this, but am also posting this for review in case we later find any problems with what I just pushed. plugins/split/Mak...

...from any thread model, including + * NBDKIT_THREAD_MODEL_PARALLEL (in fact, parallel is the recommended + * thread model). + * + * See plugins/file/file.c for an example. + */ + +#ifndef NBDKIT_FILEOPS_H +#define NBDKIT_FILEOPS_H + +#include <config.h> + +#include <stdbool.h> + +/* For SEEK_HOLE. */ +#include <unistd.h> +#include <sys/types.h> + +/* Either use this as the per-connection handle, or it must appear as + * the first member of the per-connection handle. Don’t access these + * fields directly from your plugin. + */ +struct fileops { + int fd; + int sector_size; +...

There's a lot of code in nbdkit-file-plugin which it would be nice to reuse elsewhere. One possible approach (as outlined here) is simply to move the file callbacks (like file.pread, file.pwrite, file.zero etc) to a new mini-library. They can then be consumed by other plugins fairly easily by doing: static void * foo_open (int readonly) { struct fileops *fops; int fd, flags; /*

https://bugzilla.samba.org/show_bug.cgi?id=8918 Summary: Use fiemap to quickly detect zero ranges of source file Product: rsync Version: 3.1.0 Platform: All OS/Version: All Status: NEW Severity: enhancement Priority: P5 Component: core AssignedTo: wayned at samba.org

...lo All, I''d like to know if there is an utility like `Filefrag'' shipped with e2fsprogs on linux, which is used to fetch the extents mapping info of a file(especially a sparse file) located on ZFS? I am working on efficient sparse file detection and backup through lseek(SEEK_DATA/SEEK_HOLE) on ZFS, and I need to verify the result by comparing the original sparse file and the copied file, so if there is such a tool available, it can be used to analyze the start offset and length of each data extent. Thanks in advance! -Jeff

Needs some work still, see in particular the commit message for patch 3. Rich.

...fully allocated image. > This may be a bug in nbdkit or qemu-img. Calling fallocate(0) forces allocation; so anything explicitly written to 0 won't be sparse when this mode is used. There's also a question of whether your source file accurately reports holes to begin with (poor tmpfs SEEK_HOLE performance is still a common problem). But I don't see that as getting in the way of this patch going in. > --- > plugins/file/file.c | 32 ++++++++++++++++++++++++++++++++ > 1 file changed, 32 insertions(+) > LGTM -- Eric Blake, Principal Software Engineer Red Hat, Inc....

..., 0, 0, POSIX_FADV_RANDOM|POSIX_FADV_DONTNEED); /* Iterate over blocks. */ iter_blocks (idx, nr_threads, filename, fd, outputfile, ofd); + /* discard ranges that were allocated but not written */ + data_start = 0; + while (data_start < size) { + hole_start = lseek (ofd, data_start, SEEK_HOLE); + if (hole_start == (off_t) -1) + error (EXIT_FAILURE, errno, "lseek: %s", outputfile); + if (hole_start == size) + break; + data_start = lseek (ofd, hole_start, SEEK_DATA); + if (data_start == (off_t) -1) { + if (errno == ENXIO) + data_start = size; +...