search for: extenting

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 107 ++++++++ docs/nbdkit-plugin.pod | 103 +++++++ include/nbdkit-common.h | 10 +- include/nbdkit-filter.h | 25 +- include/nbdkit-plugin.h | 6 +- server/internal.h | 5 + server/extents.c | 574

On 3/19/19 11:34 AM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- > + void nbdkit_extents_free (struct nbdkit_extents_map *); > + > +Frees an existing extents map. Is this like free() where it is safe to call on NULL? > + > +=head3 Iterating over nbdkit_extents_map

This extends the vector library with an insert function. It is more expensive than appending, but this does not affect existing code using vector and can be used in new places without making those uses more expensive. We use this function in nbdkit-extentlist-filter, nbdkit-eval-plugin and the sparse library. This is refactoring, so should not affect functionality at all. However during the

I created a btrfs file system with a single 420 megabyte file in it. And, when I look at the file system with btrfs-debug, I see gigantic extents, as large as 99 megabytes: > $ sudo btrfs-debug-tree /dev/sdb | grep extent > ... > dev extent chunk_tree 3 > dev extent chunk_tree 3 > extent data disk byte 80084992 nr 99958784 > extent data

--- server/extents.c | 49 ++++++++++++++++++------------------------------ 1 file changed, 18 insertions(+), 31 deletions(-) diff --git a/server/extents.c b/server/extents.c index 2d609652..4ab5946c 100644 --- a/server/extents.c +++ b/server/extents.c @@ -42,6 +42,7 @@ #include <assert.h> #include "minmax.h" +#include "vector.h" #include "internal.h"

I want to post this but mainly for discussion and early review. It's not safe for these patches to all go upstream yet (because not all filters have been checked/adjusted), but if any patches were to go upstream then probably 1 & 2 only are safe. File, VDDK, memory and data plugins all work, although I have only done minimal testing on them. The current tests, such as they are, all

--- plugins/vddk/vddk.c | 41 +++++++++++++++++++++++++---------------- TODO | 1 - 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/plugins/vddk/vddk.c b/plugins/vddk/vddk.c index 87c0d146..d1a3015f 100644 --- a/plugins/vddk/vddk.c +++ b/plugins/vddk/vddk.c @@ -51,6 +51,7 @@ #include "isaligned.h" #include "minmax.h" #include

On 3/26/19 4:17 PM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- > +=head2 C<.extents> > + > + int extents (void *handle, uint32_t count, uint64_t offset, > + uint32_t flags, struct nbdkit_extents *extents); > + > +During the data serving phase, this

We have several filters that would benefit from reporting extents to the client that are always aligned to boundaries chosen by the filter, regardless of whether the plugin reports extents at a finer granularity. Add a new helper function to make the work easier, without having to duplicate code in each filter. Any alignment block that straddles more than one plugin extent is reported as a

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 89 +++++++++++++++++++ docs/nbdkit-plugin.pod | 96 +++++++++++++++++++++ include/nbdkit-common.h | 10 ++- include/nbdkit-filter.h | 22 ++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 + server/extents.c | 186

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 92 +++++++++++++++++++++ docs/nbdkit-plugin.pod | 99 +++++++++++++++++++++++ include/nbdkit-common.h | 10 ++- include/nbdkit-filter.h | 22 +++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 + server/extents.c | 171

This new pair of callbacks allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 83 ++++++++++++++++ docs/nbdkit-plugin.pod | 97 +++++++++++++++++++ include/nbdkit-common.h | 10 +- include/nbdkit-filter.h | 22 ++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 + server/extents.c | 210

Ultimately, both the blocksize and swab filters want to return aligned extents to the client. I'm posting this as a snapshot of my work in progress on how I plan to get there (it's not quite working yet, but I'm done for today and wanted to at least document my ideas). I might also add a convenience function for nbdkit_extents_offset, since we have a number of filters that repeat the

[11558.527680] ------------[ cut here ]------------ [11558.527708] kernel BUG at fs/btrfs/transaction.c:1337! [11558.527730] invalid opcode: 0000 [#1] PREEMPT SMP [11558.527764] CPU 1 [11558.527776] Modules linked in: loop nls_cp437 vfat fat dm_mod xfs exportfs jfs usb_storage uas fuse ext4 jbd2 mbcache snd_hda_codec_hdmi snd_hda_codec_realtek arc4 iwlwifi snd_hda_intel snd_hda_codec uvcvideo

This tentative commit implements extents/can_extents, roughly as discussed in the previous thread here: https://www.redhat.com/archives/libguestfs/2019-March/msg00017.html I can't say that I'm a big fan of having the plugin allocate an extents array. There are no other plugin callbacks currently where we require the plugin to allocate complex data structures (or indeed do any allocation

On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > This pair of calls allows plugins to describe which extents in the > virtual disk are allocated, holes or zeroes. > --- I see you've already made a couple of tweaks to your block-status branch since posting this (skipping 0-legnth add_extent, and coalescing consecutive add_extents with the same type), but here's some more review:

Hi there... I have kernel version 2.6.36.3, compiled with gcc 4.4.5, btrfstools version 0.19+20101101 I have a btrfs filesystem (/data) consisting of two 1TB hard disks, raid0. I added in another 1TB hard drive. root@X86-64:~# btrfs filesystem show failed to read /dev/sdh failed to read /dev/sdg failed to read /dev/sdf failed to read /dev/sde failed to read /dev/sr0 failed to read /dev/fd0u800

On Tue, Mar 12, 2019 at 11:20:25AM +0000, Richard W.M. Jones wrote: >This pair of calls allows plugins to describe which extents in the >virtual disk are allocated, holes or zeroes. >--- > docs/nbdkit-filter.pod | 22 +++++++++++ > docs/nbdkit-plugin.pod | 86 +++++++++++++++++++++++++++++++++++++++++ > include/nbdkit-common.h | 14 ++++++- > include/nbdkit-filter.h | 12 +++++-

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 22 +++++++++++ docs/nbdkit-plugin.pod | 86 +++++++++++++++++++++++++++++++++++++++++ include/nbdkit-common.h | 14 ++++++- include/nbdkit-filter.h | 12 +++++- include/nbdkit-plugin.h | 6 ++- server/internal.h | 7 +++- server/filters.c |

Hello, I''m currently trying to understand how compression in btrfs works. I could not find any detailed description about it. So here are my questions. 1. How is decided what to compress and what not? After a fast test with a 2g image file, I''ve looked into the extents of that file with find-new and it turned out that only some of the first extents were compressed. The file was