search for: extent

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 107 ++++++++ docs/nbdkit-plugin.pod | 103 +++++++ include/nbdkit-common.h | 10 +- include/nbdkit-filter.h | 25 +- include/nbdkit-plugin.h | 6 +- server/internal.h | 5 + server/extents.c | 5...

On 3/19/19 11:34 AM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- > + void nbdkit_extents_free (struct nbdkit_extents_map *); > + > +Frees an existing extents map. Is this like free() where it is safe to call on NULL? > + > +=head3 Iterating over nbdkit_extents_map > + > +...

This extends the vector library with an insert function. It is more expensive than appending, but this does not affect existing code using vector and can be used in new places without making those uses more expensive. We use this function in nbdkit-extentlist-filter, nbdkit-eval-plugin and the sparse library. This is refactoring, so should not affect functionality at all. However during the rewrite of eval it revealed a fencepost error in the loop iterating over method_scripts which is fixed here. --- common/sparse/Makefile.am | 1 + commo...

I created a btrfs file system with a single 420 megabyte file in it. And, when I look at the file system with btrfs-debug, I see gigantic extents, as large as 99 megabytes: > $ sudo btrfs-debug-tree /dev/sdb | grep extent > ... > dev extent chunk_tree 3 > dev extent chunk_tree 3 > extent data disk byte 80084992 nr 99958784 > extent data offset 0 nr 99958784 ram 99958784 >...

--- server/extents.c | 49 ++++++++++++++++++------------------------------ 1 file changed, 18 insertions(+), 31 deletions(-) diff --git a/server/extents.c b/server/extents.c index 2d609652..4ab5946c 100644 --- a/server/extents.c +++ b/server/extents.c @@ -42,6 +42,7 @@ #include <assert.h> #include "...

...lters have been checked/adjusted), but if any patches were to go upstream then probably 1 & 2 only are safe. File, VDDK, memory and data plugins all work, although I have only done minimal testing on them. The current tests, such as they are, all pass. We probably do need more testing of the extents feature specifically. I have only tested against qemu (3.1.0) not against any other clients (if any exist that support this feature?) For reference the original plan which I'm working to is here: https://www.redhat.com/archives/libguestfs/2019-March/msg00017.html https://www.redhat.com/archi...

...1,6 @@ General ideas for improvements * common/utils/vector.h could be extended and used in other places: - there are some more possible places in the server (anywhere using realloc is suspect) - - nbdkit-vddk-plugin, where it builds the new argv - allow insertion, and use it in nbdkit-extentlist-filter - allow insertion and keep sorted, use it in nbdkit-eval-plugin - same, and use it in common/sparse/sparse.c -- 2.25.0

On 3/26/19 4:17 PM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- > +=head2 C<.extents> > + > + int extents (void *handle, uint32_t count, uint64_t offset, > + uint32_t flags, struct nbdkit_extents *extents); > + > +During the data serving phase, this callback...

We have several filters that would benefit from reporting extents to the client that are always aligned to boundaries chosen by the filter, regardless of whether the plugin reports extents at a finer granularity. Add a new helper function to make the work easier, without having to duplicate code in each filter. Any alignment block that straddles more than one...

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 89 +++++++++++++++++++ docs/nbdkit-plugin.pod | 96 +++++++++++++++++++++ include/nbdkit-common.h | 10 ++- include/nbdkit-filter.h | 22 ++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 +...

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 92 +++++++++++++++++++++ docs/nbdkit-plugin.pod | 99 +++++++++++++++++++++++ include/nbdkit-common.h | 10 ++- include/nbdkit-filter.h | 22 +++++- include/nbdkit-plugin.h | 6 +- server/internal.h |...

This new pair of callbacks allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 83 ++++++++++++++++ docs/nbdkit-plugin.pod | 97 +++++++++++++++++++ include/nbdkit-common.h | 10 +- include/nbdkit-filter.h | 22 ++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 + serv...

Ultimately, both the blocksize and swab filters want to return aligned extents to the client. I'm posting this as a snapshot of my work in progress on how I plan to get there (it's not quite working yet, but I'm done for today and wanted to at least document my ideas). I might also add a convenience function for nbdkit_extents_offset, since we have a number of...

[11558.527680] ------------[ cut here ]------------ [11558.527708] kernel BUG at fs/btrfs/transaction.c:1337! [11558.527730] invalid opcode: 0000 [#1] PREEMPT SMP [11558.527764] CPU 1 [11558.527776] Modules linked in: loop nls_cp437 vfat fat dm_mod xfs exportfs jfs usb_storage uas fuse ext4 jbd2 mbcache snd_hda_codec_hdmi snd_hda_codec_realtek arc4 iwlwifi snd_hda_intel snd_hda_codec uvcvideo

This tentative commit implements extents/can_extents, roughly as discussed in the previous thread here: https://www.redhat.com/archives/libguestfs/2019-March/msg00017.html I can't say that I'm a big fan of having the plugin allocate an extents array. There are no other plugin callbacks currently where we require the plugin to...

On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- I see you've already made a couple of tweaks to your block-status branch since posting this (skipping 0-legnth add_extent, and coalescing consecutive add_extents with the same type), but here's some more review: > +++ b...

...th gkrellm. Also, after about an hour of balancing, /dev/sdb is still completely empty. Any idea of why this is happening? Last few lines of dmesg: floppy0: disk absent or changed during operation end_request: I/O error, dev fd0, sector 128 end_request: I/O error, dev fd0, sector 0 btrfs: found 69 extents btrfs: relocating block group 1897465380864 flags 9 btrfs: found 95 extents btrfs: found 95 extents btrfs: relocating block group 1895317897216 flags 9 btrfs: found 81 extents btrfs: found 81 extents btrfs: relocating block group 1893170413568 flags 9 btrfs: found 93 extents btrfs: found 93 extent...

On Tue, Mar 12, 2019 at 11:20:25AM +0000, Richard W.M. Jones wrote: >This pair of calls allows plugins to describe which extents in the >virtual disk are allocated, holes or zeroes. >--- > docs/nbdkit-filter.pod | 22 +++++++++++ > docs/nbdkit-plugin.pod | 86 +++++++++++++++++++++++++++++++++++++++++ > include/nbdkit-common.h | 14 ++++++- > include/nbdkit-filter.h | 12 +++++- > include/nbdkit-plugin.h...

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 22 +++++++++++ docs/nbdkit-plugin.pod | 86 +++++++++++++++++++++++++++++++++++++++++ include/nbdkit-common.h | 14 ++++++- include/nbdkit-filter.h | 12 +++++- include/nbdkit-plugin.h | 6 ++- server/internal.h...

Hello, I''m currently trying to understand how compression in btrfs works. I could not find any detailed description about it. So here are my questions. 1. How is decided what to compress and what not? After a fast test with a 2g image file, I''ve looked into the extents of that file with find-new and it turned out that only some of the first extents were compressed. The file was simply copied with cp. 2. I compared the extents of that mentioned file from a non-compressed fs and from a compressed fs and see much more and much smaller extents in the compressed fs....