Libguestfs - Feb 2016 - Question: resize: non-sparse copying of extended partition

Hello everyone!

I faced an issue using virt-resize. I have an image with
extended -> logical -> pv -> lv -> ext4
(although I think LVM does not matter).

And image is 65G virtual size and 1G actualSize.
When I call virt-resize to 60G, it blindly copies the extended partition 
content,
but does it non-sparse (resize.ml:1231), irrespective to --no-sparse option.
So the destination image is 60G actual size.

Is it a bug, or it was made intentionally (why?) ?
TY for answers!

-- 
Your sincerely,
Maxim Perevedentsev

On Tue, Feb 02, 2016 at 12:09:49PM +0300, Maxim Perevedentsev
wrote:
> Hello everyone!
> 
> I faced an issue using virt-resize. I have an image with
> extended -> logical -> pv -> lv -> ext4
> (although I think LVM does not matter).
> 
> And image is 65G virtual size and 1G actualSize.
> When I call virt-resize to 60G, it blindly copies the extended
> partition content,
> but does it non-sparse (resize.ml:1231), irrespective to --no-sparse
option.
> So the destination image is 60G actual size.
> 
> Is it a bug, or it was made intentionally (why?) ?
> TY for answers!
I think it's a bug.  We should add the ~sparse flag to
g#copy_device_to_device depending on the state of the 'sparse'
(ie. command line) flag.

Does this untested patch fix it?

diff --git a/resize/resize.ml b/resize/resize.ml
index 8ff4793..6ac1019 100644
--- a/resize/resize.ml
+++ b/resize/resize.ml
@@ -1228,7 +1228,9 @@ read the man page virt-resize(1).
             *)
            let srcoffset = p.p_part.G.part_start in
            let destoffset = p.p_target_start *^ 512L in
-           g#copy_device_to_device ~srcoffset ~destoffset ~size:copysize
"/dev/sda" "/dev/sdb"
+           g#copy_device_to_device ~srcoffset ~destoffset ~size:copysize
+                                   ~sparse
+                                   "/dev/sda" "/dev/sdb"
         )
       | OpIgnore | OpDelete -> ()
   in


Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

Greetings,

I'm playing around an idea and I'd like to ask you some questions.

I'd like to extract the MFT table from a disk image file. The idea is to 
employ it to build a sort of reverse lookup table which, given a 
cluster, could retrieve the corresponding file with the related metadata.

Such table could be used to optimize the analysis of disk snapshots in 
order to collect the changes which happened on the disk. As the disk 
snapshots contains only the new or modified clusters, I could avoid 
exploring the whole FS content and focus on what has really changed on disk.

Did you explore the concept anyhow? Is there a way I can use libguestfs 
to locate and extract the MFT table from a disk image?

Thank you.

On Tue, Feb 02, 2016 at 07:40:12PM +0200, noxdafox
wrote:
> Greetings,
> 
> I'm playing around an idea and I'd like to ask you some questions.
> 
> I'd like to extract the MFT table from a disk image file. The idea
> is to employ it to build a sort of reverse lookup table which, given
> a cluster, could retrieve the corresponding file with the related
> metadata.
> 
> Such table could be used to optimize the analysis of disk snapshots
> in order to collect the changes which happened on the disk. As the
> disk snapshots contains only the new or modified clusters, I could
> avoid exploring the whole FS content and focus on what has really
> changed on disk.
> 
> Did you explore the concept anyhow?
No.
> Is there a way I can use libguestfs to locate and extract the MFT
> table from a disk image?
If there's an ntfsprogs command that does this (ntfsinfo --mft maybe?)
then it's really easy to extract the output from that command.  You
could hack it together using `debug sh', search this page:

  http://libguestfs.org/guestfs-faq.1.html

... but if you wanted to do it "properly" then you could add an API
modelled on one of the `FileOut' APIs, eg:

  https://github.com/libguestfs/libguestfs/blob/master/daemon/base64.c#L100

For information on adding APIs, see:

  http://libguestfs.org/guestfs-hacking.1.html#adding-a-new-api

This question of how do you find which disk block is associated with a
particular file comes up often enough that I have looked at it various
times on my blog:

 
https://rwmj.wordpress.com/2014/02/21/use-guestfish-and-nbdkit-to-examine-physical-disk-locations/

  https://rwmj.wordpress.com/2014/11/23/mapping-files-to-disk/

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top