Libguestfs - Aug 2018 - Re: [PATCH] v2v: Add --print-estimate option to print source size estimate.

On Tue, Aug 14, 2018 at 09:31:06PM +0300, Nir Soffer
wrote:
> On Tue, Aug 14, 2018 at 8:29 PM Richard W.M. Jones
<rjones@redhat.com>
> wrote:
> 
> > This option prints the estimated size of the data that will be copied
> > from the source disk.
> >
> > For interest, the test prints:
> >
> > 3747840 ../test-data/phony-guests/windows.img
> > Estimate: 3710976
> >
> 
> Why not use qemu-img measure on the overlay?
Yes this would be better, but oddly it doesn't work properly when
the output format is set to 'raw':

  /usr/bin/qemu-img 'measure' '-f' 'qcow2' '-O'
'raw' '--output=human'
'/home/rjones/d/libguestfs/tmp/v2vovl62b7c2.qcow2'
  required size: 6442450944
  fully allocated size: 6442450944

However it's OK if the output format is set to 'qcow2':

  /usr/bin/qemu-img 'measure' '-f' 'qcow2' '-O'
'qcow2' '--output=human'
'/home/rjones/d/libguestfs/tmp/v2vovla53d7c.qcow2'
  required size: 1047986176
  fully allocated size: 6443696128

I guess it ignores sparseness of raw images, but I can't see a way to
specify that on the command line.

OTOH the qcow2 figure is probably a good enough guess for our purposes
(ie. estimating how much data will be copied).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

Am 15.08.2018 um 12:26 hat Richard W.M. Jones
geschrieben:
> On Tue, Aug 14, 2018 at 09:31:06PM +0300, Nir Soffer wrote:
> > On Tue, Aug 14, 2018 at 8:29 PM Richard W.M. Jones
<rjones@redhat.com>
> > wrote:
> > 
> > > This option prints the estimated size of the data that will be
copied
> > > from the source disk.
> > >
> > > For interest, the test prints:
> > >
> > > 3747840 ../test-data/phony-guests/windows.img
> > > Estimate: 3710976
> > >
> > 
> > Why not use qemu-img measure on the overlay?
> 
> Yes this would be better, but oddly it doesn't work properly when
> the output format is set to 'raw':
> 
>   /usr/bin/qemu-img 'measure' '-f' 'qcow2'
'-O' 'raw' '--output=human'
'/home/rjones/d/libguestfs/tmp/v2vovl62b7c2.qcow2'
>   required size: 6442450944
>   fully allocated size: 6442450944
> 
> However it's OK if the output format is set to 'qcow2':
> 
>   /usr/bin/qemu-img 'measure' '-f' 'qcow2'
'-O' 'qcow2' '--output=human'
'/home/rjones/d/libguestfs/tmp/v2vovla53d7c.qcow2'
>   required size: 1047986176
>   fully allocated size: 6443696128
> 
> I guess it ignores sparseness of raw images, but I can't see a way to
> specify that on the command line.
> 
> OTOH the qcow2 figure is probably a good enough guess for our purposes
> (ie. estimating how much data will be copied).
'qemu-img measure' calculates the resulting file size, not the number of
used blocks. I think this is because its main purpose is to create block
devices (like LVs) of the right size, so sparseness on a filesystem
level doesn't buy you anything.

Kevin

(Adding Stefan who implemented the subcommand)

On Wed, Aug 15, 2018 at 12:44:44PM +0200, Kevin Wolf
wrote:
> Am 15.08.2018 um 12:26 hat Richard W.M. Jones geschrieben:
> > On Tue, Aug 14, 2018 at 09:31:06PM +0300, Nir Soffer wrote:
> > > On Tue, Aug 14, 2018 at 8:29 PM Richard W.M. Jones
<rjones@redhat.com>
> > > wrote:
> > > 
> > > > This option prints the estimated size of the data that will
be copied
> > > > from the source disk.
> > > >
> > > > For interest, the test prints:
> > > >
> > > > 3747840 ../test-data/phony-guests/windows.img
> > > > Estimate: 3710976
> > > >
> > > 
> > > Why not use qemu-img measure on the overlay?
> > 
> > Yes this would be better, but oddly it doesn't work properly when
> > the output format is set to 'raw':
> > 
> >   /usr/bin/qemu-img 'measure' '-f' 'qcow2'
'-O' 'raw' '--output=human'
'/home/rjones/d/libguestfs/tmp/v2vovl62b7c2.qcow2'
> >   required size: 6442450944
> >   fully allocated size: 6442450944
> > 
> > However it's OK if the output format is set to 'qcow2':
> > 
> >   /usr/bin/qemu-img 'measure' '-f' 'qcow2'
'-O' 'qcow2' '--output=human'
'/home/rjones/d/libguestfs/tmp/v2vovla53d7c.qcow2'
> >   required size: 1047986176
> >   fully allocated size: 6443696128
> > 
> > I guess it ignores sparseness of raw images, but I can't see a way
to
> > specify that on the command line.
> > 
> > OTOH the qcow2 figure is probably a good enough guess for our purposes
> > (ie. estimating how much data will be copied).
> 
> 'qemu-img measure' calculates the resulting file size, not the
number of
> used blocks. I think this is because its main purpose is to create block
> devices (like LVs) of the right size, so sparseness on a filesystem
> level doesn't buy you anything.
But if I run ‘qemu-img convert ... -O raw output.raw’ then output.raw
will be a sparse file, and that's the file size I'd expect measure to
give us for "required size" (of course "fully allocated
size" would be
the virtual size, and that is correct).

It does look as if the block/raw-format.c:raw_measure function is wrong.

In any case we can use the qcow2 estimate for our purposes as it will
be near enough what we need (a rough estimate of the size of the copy).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v