search for: falloc_fl_

On RHEL 7.5 we need to include <linux/falloc.h> for FALLOC_FL_* macros. Without the macros, fallocate is never used and we fall back to manual zeroing. Here are examples runs with this change with a local file on ext4: $ export SOCK=/tmp/nbd.sock $ export FILE=/var/tmp/nbd.img $ export BLOCK=/dev/loop2 $ src/nbdkit -f plugins/file/.libs/nbdkit-file-plugin.s...

On RHEL 7.5 we need to include <linux/falloc.h> for FALLOC_FL_* macros. Without the macros, fallocate is never used and we fall back to manual zeroing. Here are examples runs with this change with a local file on ext4: $ export SOCK=/tmp/nbd.sock $ export FILE=/var/tmp/nbd.img $ export BLOCK=/dev/loop2 $ src/nbdkit -f plugins/file/.libs/nbdkit-file-plugin.s...

On 07/30/2018 01:04 PM, Nir Soffer wrote: > On RHEL 7.5 we need to include <linux/falloc.h> for FALLOC_FL_* macros. Rather, on any Linux system that pre-dates glibc 2.18, where the flags were finally supported directly in <fcntl.h> > Without the macros, fallocate is never used and we fall back to manual > zeroing. > > +++ b/plugins/file/file.c > @@ -42,6 +42,10 @@ > #inclu...

On 07/29/2018 06:35 AM, Nir Soffer wrote: > On RHEL 7.5 we need to include <linux/falloc.h> for FALLOC_FL_* macros. > Without the macros, fallocate is never used and we fall back to manual > zeroing. > > @@ -41,6 +41,7 @@ > #include <sys/types.h> > #include <sys/stat.h> > #include <errno.h> > +#include <linux/falloc.h> /* For FALLOC_FL_* on RHEL,...

On 08/02/2018 02:05 PM, Nir Soffer wrote: > fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with > modern kernel, but when it is not, we fall back to manual zeroing. > > Check if the underlying file is a block device when opening the file, > and fall back to ioctl(BLKZEROOUT) for aligned zero requests for a > block device. > &gt...

On Thu, Aug 2, 2018 at 10:39 PM Eric Blake <eblake@redhat.com> wrote: > On 08/02/2018 02:05 PM, Nir Soffer wrote: > > fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with > > modern kernel, but when it is not, we fall back to manual zeroing. > > > > Check if the underlying file is a block device when opening the file, > > and fall back to ioctl(BLKZEROOUT) for aligned zero requests for a > >...

...tems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v2: - Revert file_can_trim change, since it is too late to change the value after negotiation. Changing the capability dinamically may be useful internally, but it should be done via other means. - Do not depend on FALLOC_FL_* when including <linux/fs.h>. - Add common/includes/isaligned.h for is_aligned helper, implemented in a more efficient way with bitwise math. - If getting sector size fail, fall back to safe guess instead of hard failure. - More efficient alignment check using bitwise math. - For BLKZEROO...

...c +++ b/plugins/file/file.c @@ -33,6 +33,7 @@ #include <config.h> +#include <stdbool.h> #include <stdio.h> #include <stdlib.h> #include <string.h> @@ -42,6 +43,8 @@ #include <sys/stat.h> #include <errno.h> #include <linux/falloc.h> /* For FALLOC_FL_* on RHEL, glibc < 2.18 */ +#include <sys/ioctl.h> +#include <linux/fs.h> #include <nbdkit-plugin.h> @@ -116,6 +119,10 @@ file_config_complete (void) /* The per-connection handle. */ struct handle { int fd; + bool is_block_device; + bool can_punch_hole; + bool can_z...

This is the second version to support efficient zero for block devices on older kernels (e.g. RHEL 7.5), and file systems that do not support yet FALLOC_FS_ZERO_RANGE (e.g. NFS 4.2). Changes since v1: - Split to smaller patches - Skip linux only includes on other systems - Skip code using BLKZEROOUT if the macro is not defined - Try BLKZEROOUT only if the offset and count are aligned to device

...ized flags. If the server does not advertise the new feature, clients can safely fall back to assuming that writing zeroes is no faster than normal writes. Note that the Linux fallocate(2) interface may or may not be powerful enough to easily determine if zeroing will be efficient - in particular, FALLOC_FL_ZERO_RANGE in isolation does NOT give that insight; for block devices, it is known that ioctl(BLKZEROOUT) does NOT have a way for userspace to probe if it is efficient or slow. But with enough demand, the kernel may add another FALLOC_FL_ flag to use with FALLOC_FL_ZERO_RANGE, and/or appropriate io...

...clude <config.h> > > +#include <stdbool.h> > #include <stdio.h> > #include <stdlib.h> > #include <string.h> > @@ -42,6 +43,8 @@ > #include <sys/stat.h> > #include <errno.h> > #include <linux/falloc.h> /* For FALLOC_FL_* on RHEL, glibc < 2.18 */ > +#include <sys/ioctl.h> > +#include <linux/fs.h> Does this need a configure-time probe to see if it exists, since it will break compilation on BSD systems? Same question to linux/falloc.h. Actually, linux/falloc.h doesn't see any use in the c...

fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with modern kernel, but when it is not, we fall back to manual zeroing. Check if the underlying file is a block device when opening the file, and fall back to ioctl(BLKZEROOUT) for aligned zero requests for a block device. Here is an example run without th...

...bool.h> #include <string.h> +#include <inttypes.h> #include <fcntl.h> #include <unistd.h> #include <sys/types.h> @@ -44,6 +45,8 @@ #include <sys/ioctl.h> #include <errno.h> +#include <pthread.h> + #if defined(__linux__) && !defined(FALLOC_FL_PUNCH_HOLE) #include <linux/falloc.h> /* For FALLOC_FL_*, glibc < 2.18 */ #endif @@ -68,7 +71,11 @@ static char *filename = NULL; -int file_debug_zero; /* to enable: -D file.zero=1 */ +/* Any callbacks using lseek must be protected by this lock. */ +static pthread_mutex_...

fallocate(FALLOC_FL_ZERO_RANGE) is supported for block devices with modern kernel, but when it is not, we fall back to manual zeroing. For block device, try also to use ioctl(BLKZEROOUT) if offset and count are aligned to block device sector size. Here is an example run without this change on RHEL 7.5: $ export SOCK...

fallocate(FALLOC_FL_ZERO_RANGE) is supportd for block devices with modern kernel, but when it is not, we fall back to manual zeroing. For block device, try also to use ioctl(BLKZEROOUT) if offset and count are aligned to block device sector size. Here is an example run without this change on RHEL 7.5: $ export SOCK=...

...c +++ b/plugins/file/file.c @@ -33,6 +33,7 @@ #include <config.h> +#include <stdbool.h> #include <stdio.h> #include <stdlib.h> #include <string.h> @@ -42,6 +43,8 @@ #include <sys/stat.h> #include <errno.h> #include <linux/falloc.h> /* For FALLOC_FL_* on RHEL, glibc < 2.18 */ +#include <sys/ioctl.h> +#include <linux/fs.h> #include <nbdkit-plugin.h> @@ -116,6 +119,10 @@ file_config_complete (void) /* The per-connection handle. */ struct handle { int fd; + bool is_block_device; + bool can_punch_hole; + bool can_z...

...bool.h> #include <string.h> +#include <inttypes.h> #include <fcntl.h> #include <unistd.h> #include <sys/types.h> @@ -44,6 +45,8 @@ #include <sys/ioctl.h> #include <errno.h> +#include <pthread.h> + #if defined(__linux__) && !defined(FALLOC_FL_PUNCH_HOLE) #include <linux/falloc.h> /* For FALLOC_FL_*, glibc < 2.18 */ #endif @@ -68,7 +71,11 @@ static char *filename = NULL; -int file_debug_zero; /* to enable: -D file.zero=1 */ +/* Any callbacks using lseek must be protected by this lock. */ +static pthread_mutex_...

...does not advertise the new > feature, clients can safely fall back to assuming that writing zeroes > is no faster than normal writes. > Note that the Linux fallocate(2) interface may or may not be powerful > enough to easily determine if zeroing will be efficient - in > particular, FALLOC_FL_ZERO_RANGE in isolation does NOT give that > insight; for block devices, it is known that ioctl(BLKZEROOUT) does > NOT have a way for userspace to probe if it is efficient or slow. But > with enough demand, the kernel may add another FALLOC_FL_ flag to use > with FALLOC_FL_ZERO_RANGE, a...

...e new feature, clients can safely fall back to assuming that writing zeroes is no faster than normal writes (whether or not the assumption actually holds). Note that the Linux fallocate(2) interface may or may not be powerful enough to easily determine if zeroing will be efficient - in particular, FALLOC_FL_ZERO_RANGE in isolation does NOT give that insight; likewise, for block devices, it is known that ioctl(BLKZEROOUT) does NOT have a way for userspace to probe if it is efficient or slow. But with enough demand, the kernel may add another FALLOC_FL_ flag to use with FALLOC_FL_ZERO_RANGE, and/or appr...

...clude <stdbool.h> +#include <stdint.h> +#include <inttypes.h> +#include <fcntl.h> +#include <unistd.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/ioctl.h> + +#include <pthread.h> + +#if defined (__linux__) && !defined (FALLOC_FL_PUNCH_HOLE) +#include <linux/falloc.h> /* For FALLOC_FL_*, glibc < 2.18 */ +#endif + +#if defined (__linux__) +#include <linux/fs.h> /* For BLKZEROOUT */ +#endif + +#define NBDKIT_API_VERSION 2 +#include <nbdkit-plugin.h> + +#include "cleanup.h" +#include &quot...