Libguestfs - Nov 2015 - Re: Fwd: [Bug 1277705] virt-sparsify --in-place should not sparsify a snapshot

[Let's discuss this upstream]

On Wed, Nov 04, 2015 at 12:18:48PM +0200, Yaniv Kaul
wrote:
> I'm missing something here - what will happen to the tree structure?
> Will we lose it? So essentially it performs a merge?
In copying mode:

  virt-sparsify disk1 disk2

creates an overlay on top of disk1, writes zeroes to the overlay in
the parts of disk1 which are not used (disk1 is not modified by this),
and then copies the result to disk2.  It doesn't even consider any
snapshots or backing files of disk1.

	- - -

This doesn't matter because you're proposing to use --in-place which
works completely differently -- literally: it's a different code path.

  virt-sparsify --in-place disk

opens `disk', mounts each filesystem it finds in turn, and runs fstrim
on them.

The question for this bug is does that do anything useful if `disk'
has backing files?

Here is a non-sparse Fedora 22 disk image:

$ virt-builder fedora-22
$ mv fedora-22.img fedora-22.img.sparse
$ cp --sparse=never fedora-22.img.sparse fedora-22.img
$ du -sh fedora-22.img
6.1G fedora-22.img

Let's add a snapshot on top:

$ qemu-img create -f qcow2 -o compat=1.1 -b fedora-22.img overlay.qcow2
$ du -sh fedora-22.img overlay.qcow2 
6.1G fedora-22.img
196K overlay.qcow2

and sparsify the overlay:

$ virt-sparsify --in-place overlay.qcow2 
$ du -sh fedora-22.img overlay.qcow2 
6.1G fedora-22.img
3.2M overlay.qcow2

All that happened was that the overlay got bigger (because it's now
storing a bunch of qcow2 zero clusters marking the places in the
backing file which are zero).

In *theory* it should be possible to commit the changes to the backing
file, making the backing file sparse.  But in reality that doesn't
work:

$ qemu-img commit overlay.qcow2 
Image committed.
$ du -sh fedora-22.img overlay.qcow2 
6.1G fedora-22.img
260K overlay.qcow2

So really there's no use for virt-sparsify on a snapshot (although you
could also argue this is a bug or missing feature in qemu-img).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

On Wed, Nov 4, 2015 at 12:49 PM, Richard W.M. Jones <rjones@redhat.com>
wrote:
> [Let's discuss this upstream]
>
> On Wed, Nov 04, 2015 at 12:18:48PM +0200, Yaniv Kaul wrote:
> > I'm missing something here - what will happen to the tree
structure?
> > Will we lose it? So essentially it performs a merge?
>
> In copying mode:
>
>   virt-sparsify disk1 disk2
>
> creates an overlay on top of disk1, writes zeroes to the overlay in
> the parts of disk1 which are not used (disk1 is not modified by this),
> and then copies the result to disk2.  It doesn't even consider any
> snapshots or backing files of disk1.
>
>         - - -
>
> This doesn't matter because you're proposing to use --in-place
which
> works completely differently -- literally: it's a different code path.
>
>   virt-sparsify --in-place disk
>
> opens `disk', mounts each filesystem it finds in turn, and runs fstrim
> on them.
>
> The question for this bug is does that do anything useful if `disk'
> has backing files?
>
> Here is a non-sparse Fedora 22 disk image:
>
> $ virt-builder fedora-22
> $ mv fedora-22.img fedora-22.img.sparse
> $ cp --sparse=never fedora-22.img.sparse fedora-22.img
> $ du -sh fedora-22.img
> 6.1G fedora-22.img
>
> Let's add a snapshot on top:
>
> $ qemu-img create -f qcow2 -o compat=1.1 -b fedora-22.img overlay.qcow2
> $ du -sh fedora-22.img overlay.qcow2
> 6.1G fedora-22.img
> 196K overlay.qcow2
>
> and sparsify the overlay:
>
> $ virt-sparsify --in-place overlay.qcow2
> $ du -sh fedora-22.img overlay.qcow2
> 6.1G fedora-22.img
> 3.2M overlay.qcow2
>
> All that happened was that the overlay got bigger (because it's now
> storing a bunch of qcow2 zero clusters marking the places in the
> backing file which are zero).
>
> In *theory* it should be possible to commit the changes to the backing
> file, making the backing file sparse.  But in reality that doesn't
> work:
>
> $ qemu-img commit overlay.qcow2
> Image committed.
> $ du -sh fedora-22.img overlay.qcow2
> 6.1G fedora-22.img
> 260K overlay.qcow2
>
> So really there's no use for virt-sparsify on a snapshot (although you
> could also argue this is a bug or missing feature in qemu-img).
>
Indeed, as in real life, I expect that in any level of the snapshot tree
there are opportunities to sparsify blocks.
Perhaps I should run 'zerofree' when the VM is up, so the blocks become
zero'ed on the right snapshot? Not sure how that would help a lot, though.
It might on newer storage, which will recognize zero blocks as free on the
underlying storage level.
Y.


> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-builder quickly builds VMs from scratch
> http://libguestfs.org/virt-builder.1.html
>

On Wed, Nov 04, 2015 at 01:10:04PM +0200, Yaniv Kaul
wrote:
> On Wed, Nov 4, 2015 at 12:49 PM, Richard W.M. Jones
<rjones@redhat.com>
> wrote:
> > All that happened was that the overlay got bigger (because it's
now
> > storing a bunch of qcow2 zero clusters marking the places in the
> > backing file which are zero).
                           ^^^
Here I should have more accurately written "unused".
> Perhaps I should run 'zerofree' when the VM is up, so the blocks
become
> zero'ed on the right snapshot? Not sure how that would help a lot,
though.
> It might on newer storage, which will recognize zero blocks as free on the
> underlying storage level.
This shouldn't make any difference, since fstrim finds all unused
space in filesystems, whether zeroes or deleted files.
> > In *theory* it should be possible to commit the changes to the backing
> > file, making the backing file sparse.  But in reality that doesn't
> > work:
> >
> > $ qemu-img commit overlay.qcow2
> > Image committed.
> > $ du -sh fedora-22.img overlay.qcow2
> > 6.1G fedora-22.img
> > 260K overlay.qcow2
> >
> > So really there's no use for virt-sparsify on a snapshot (although
you
> > could also argue this is a bug or missing feature in qemu-img).
> >
> 
> Indeed, as in real life, I expect that in any level of the snapshot tree
> there are opportunities to sparsify blocks.
The trouble is you can't run virt-sparsify on the backing files -
you'll just end up with a corrupted disk.  This is actually because
virt-sparsify mounts the filesystems within the backing file, and the
mount could replay the journal.

I think this is a qemu bug or missing feature.  It should be possible
to "push" the trimmed clusters to the backing file.  I'll ask
Paolo.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

On Wed, Nov 04, 2015 at 10:49:43AM +0000, Richard W.M. Jones
wrote:
> $ qemu-img commit overlay.qcow2 
> Image committed.
> $ du -sh fedora-22.img overlay.qcow2 
> 6.1G fedora-22.img
> 260K overlay.qcow2
> 
> So really there's no use for virt-sparsify on a snapshot (although you
> could also argue this is a bug or missing feature in qemu-img).
I had a bit of discussion with Kevin Wolf on IRC, and he thinks
'qemu-img commit' ought to discard clusters in the backing file when
committing zero clusters.  IOW the above command should have
sparsified the backing file, and if it doesn't, it may be a qemu bug.

In the light of that, I'll drop my patch for RHBZ#1277705, because
maybe we can make this work.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/