search for: do_extents

...* the current filesystem. + */ + pthread_mutex_lock (&lseek_lock); + r = lseek (h->fd, 0, SEEK_HOLE); + pthread_mutex_unlock (&lseek_lock); + if (r == -1) { + nbdkit_debug ("extents disabled: lseek: SEEK_HOLE: %m"); + return 0; + } + return 1; +} + +static int +do_extents (void *handle, uint32_t count, uint64_t offset, + uint32_t flags, struct nbdkit_extents *extents) +{ + struct handle *h = handle; + const bool req_one = flags & NBDKIT_FLAG_REQ_ONE; + uint64_t end = offset + count; + + do { + off_t pos; + + pos = lseek (h->fd, offset, SE...

...* the current filesystem. + */ + pthread_mutex_lock (&lseek_lock); + r = lseek (h->fd, 0, SEEK_HOLE); + pthread_mutex_unlock (&lseek_lock); + if (r == -1) { + nbdkit_debug ("extents disabled: lseek: SEEK_HOLE: %m"); + return 0; + } + return 1; +} + +static int +do_extents (void *handle, uint32_t count, uint64_t offset, + uint32_t flags, struct nbdkit_extents *extents) +{ + struct handle *h = handle; + const bool req_one = flags & NBDKIT_FLAG_REQ_ONE; + uint64_t end = offset + count; + + do { + off_t pos; + + pos = lseek (h->fd, offset, SE...

...size (h->fd); } else { /* Regular file. */ struct stat statbuf; @@ -607,13 +604,8 @@ static int file_extents (void *handle, uint32_t count, uint64_t offset, uint32_t flags, struct nbdkit_extents *extents) { - int r; - - pthread_mutex_lock (&lseek_lock); - r = do_extents (handle, count, offset, flags, extents); - pthread_mutex_unlock (&lseek_lock); - - return r; + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lseek_lock); + return do_extents (handle, count, offset, flags, extents); } #endif /* SEEK_HOLE */ diff --git a/plugins/memory/memory.c b/plugins/memory/mem...

...files[i].can_extents = false; +#endif } h->size = offset; nbdkit_debug ("total size=%" PRIu64, h->size); @@ -319,6 +341,104 @@ split_cache (void *handle, uint32_t count, uint64_t offset, uint32_t flags) } #endif /* HAVE_POSIX_FADVISE */ +#ifdef SEEK_HOLE +static int64_t +do_extents (struct file *file, uint32_t count, uint64_t offset, + bool req_one, struct nbdkit_extents *extents) +{ + int64_t r = 0; + uint64_t end = offset + count; + + do { + off_t pos; + + pos = lseek (file->fd, offset, SEEK_DATA); + if (pos == -1) { + if (errno == ENXIO) { +...

...E etc is likely to work on + * the current filesystem. + */ + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lseek_lock); + r = lseek (fops->fd, 0, SEEK_HOLE); + if (r == -1) { + nbdkit_debug ("extents disabled: lseek: SEEK_HOLE: %m"); + return 0; + } + return 1; +} + +static int +do_extents (void *handle, uint32_t count, uint64_t offset, + uint32_t flags, struct nbdkit_extents *extents) +{ + struct fileops *fops = handle; + const bool req_one = flags & NBDKIT_FLAG_REQ_ONE; + uint64_t end = offset + count; + + do { + off_t pos; + + pos = lseek (fops->fd, off...

There's a lot of code in nbdkit-file-plugin which it would be nice to reuse elsewhere. One possible approach (as outlined here) is simply to move the file callbacks (like file.pread, file.pwrite, file.zero etc) to a new mini-library. They can then be consumed by other plugins fairly easily by doing: static void * foo_open (int readonly) { struct fileops *fops; int fd, flags; /*

...our optimization of skipping lseek in .extents ourselves. > +static int > +file_extents (void *handle, uint32_t count, uint64_t offset, > + uint32_t flags, struct nbdkit_extents *extents) > +{ > + int r; > + > + pthread_mutex_lock (&lseek_lock); > + r = do_extents (handle, count, offset, flags, extents); > + pthread_mutex_unlock (&lseek_lock); > + But this would be the spot where we could optimize by returning all data when our initial lseek() probe proved the file is not sparse. Otherwise, looks good. -- Eric Blake, Principal Software Enginee...

Needs some work still, see in particular the commit message for patch 3. Rich.

There's more that can be done (in particular, use of CLEANUP_FREE), but this is enough to at least see if I'm on the right track. I couldn't figure out an obvious difference between common/include and common/utils, but it looks like the former is for things that are inlineable via .h only, while the latter is when you need to link in a convenience library, so this landed in the

...: lseek: SEEK_HOLE: %m"); @@ -628,7 +712,7 @@ static int file_extents (void *handle, uint32_t count, uint64_t offset, uint32_t flags, struct nbdkit_extents *extents) { - ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lseek_lock); + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lock); return do_extents (handle, count, offset, flags, extents); } #endif /* SEEK_HOLE */ @@ -662,6 +746,7 @@ static struct nbdkit_plugin plugin = { .config_help = file_config_help, .magic_config_key = "file", .dump_plugin = file_dump_plugin, + .list_exports = file_list_exports,...

...1) { - nbdkit_debug ("extents disabled: lseek: SEEK_HOLE: %m"); - return 0; - } - return 1; + return h->can_extents; } static int @@ -622,6 +661,23 @@ file_extents (void *handle, uint32_t count, uint64_t offset, ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lseek_lock); return do_extents (handle, count, offset, flags, extents); } + +/* Initial state. */ +static int +file_init_sparse (void *handle) +{ + struct handle *h = handle; + + return h->init_sparse; +} + +static int +file_init_zero (void *handle) +{ + struct handle *h = handle; + + return h->init_zero; +} #endif /...

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

I want to post this but mainly for discussion and early review. It's not safe for these patches to all go upstream yet (because not all filters have been checked/adjusted), but if any patches were to go upstream then probably 1 & 2 only are safe. File, VDDK, memory and data plugins all work, although I have only done minimal testing on them. The current tests, such as they are, all

Here's changes to the file plugin (which I'm happy with) and a new exportname filter (which is still at RFC stage; I need to finish implementing strict mode in .open, and add tests). I also discovered that we really want .list_exports and .open to know when they are used on plaintext vs. tls clients for --tls=on, and we may want to split out a new .default_export callback rather than

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.

I will be following up to this email with four separate threads each addressed to the appropriate single list, with proposed changes to: - the NBD protocol - qemu: both server and client - libnbd: client - nbdkit: server The feature in question adds a new optional NBD_INFO_ packet to the NBD_OPT_GO portion of handshake, adding up to 16 bits of information that the server can advertise to the