libvirt users - Oct 2015 - Re: Questions about qcow2 file size management

Le 2015-10-14 12:09, Sven Schwedas a écrit :
> On 2015-10-14 11:34, Jérôme wrote:
> 
>> In other words, can I have this on a 60Go drive?
>> total 53G
>> 7,4G -rw------- 1 root root 11G oct.   8 06:34 prod_151008.qcow2
>> 7,4G -rw------- 1 root root 11G oct.   9 06:37 prod_151009.qcow2
>> 7,5G -rw------- 1 root root 11G oct.  10 06:41 prod_151010.qcow2
>> 7,5G -rw------- 2 root root 11G oct.  11 06:49 prod_151011.qcow2
>> 7,5G -rw------- 1 root root 11G oct.  12 06:44 prod_151012.qcow2
>> 7,5G -rw------- 1 root root 11G oct.  13 06:27 prod_151013.qcow2
>> 7,7G -rw------- 1 root root 11G oct.  14 06:55 prod_151014.qcow2
>>
>> (Total allocated is 53 < 60 but sum of 11G sizes is 77 G.)
>>
>> If so, then I guess I get the point.
> 
> Yes. You just might need to figure out what to do if VMs suddenly start
> to need more storage after all.
Sure. I don't think I'll do this, but I wanted to be sure I understood
the mechanism correctly.

Thanks for the confirmation.
>> 2/ Backup size
>>
>> Is there a simple way to do backups that only take the actual allocated
>> space (7,7 G in my example) instead of the max (10 G)?
> 
> I believe the mechanism you're searching for is sparse file handling.
> Both rsync and cp can copy sparse files correctly; rsync does not do it
> by default (--sparse), and cp's man page indicates that its automatic
> sparse file detection is buggy (but manually specifying --sparse=always
> should work).
Exactly. I missed the "sparse" keyword for my research.

I modified the cp and rsync commands in my backup script and now I get
sparse files copied as sparse files, so my backup space won't get filled
with zeros.
 
>> 3/ Playing with allocated space
>>
>> My VM apparently has an allocated size of 7.7 G
>>
>> 7,7G -rw------- 1 libvirt-qemu libvirt-qemu  11G oct.  14 10:04
>> prod.qcow2
>>
>> However, loging into it shows much less than 7.7 G is used:
> 
> TRIM is needed to indicate that space is reclaimable. This needs file
> system support (no fat32, e.g.), and support by QEMU('s I/O driver).
> Once Qemu is set up to accept TRIM, you can either enable the discard
> mount option, or set up an fstrim cron job.
OK, I think I get the idea. I'll dig into this.

Thanks !

-- 
Jérôme

Le 2015-10-14 13:27, Jérôme a écrit :
>> TRIM is needed to indicate that space is reclaimable. This needs file
>> system support (no fat32, e.g.), and support by QEMU('s I/O
driver).
>> Once Qemu is set up to accept TRIM, you can either enable the discard
>> mount option, or set up an fstrim cron job.
Thanks. I read stuff about that and it's becoming clearer now. 

These pages are the most complete explanations I found:
http://dustymabe.com/2013/06/11/recover-space-from-vm-disk-images-by-using-discardfstrim/
http://dustymabe.com/2013/06/26/enabling-qemu-guest-agent-anddddd-fstrim-again/

Unfortunately, after stopping the VM, editing the config file to add
"discard=unmap" and booting the VM again, fstrim does not work:

fstrim: /: the discard operation is not supported

The doc cited above says the feature needs virtio-scsi so I modified the
guest parameters using virt-manager to set virtio-scsi instead of virtio
serial.

I don't know how to set "discard="unmap" with virt-manager.
Is it
possible?

I can modify the .xml file using vi, but I'd rather stick to
virt-manager if possible, for me and my colleagues. However, for those
tests, I diff the .xml file everytime to check what happens underneath.

Once virtio-scsi is used, whether or not I set "discard="unmap"
(through
manually .xml file edition), as long as virtio-scsi is specified, the
fstrim command works or seems to:

#fstrim -v / 
/: 1.3 GiB (1372569600 bytes) trimmed

However, in any case, I don't see any change in the space used by the VM
file as seen from the host. It does not shrink.

Anything else I should do?

Anything I could check?

Thanks.

-- 
Jérôme

[Just now noticing this thread]

On 10/14/2015 08:43 AM, Jérôme wrote:
> Le 2015-10-14 13:27, Jérôme a écrit :
> 
>>> TRIM is needed to indicate that space is reclaimable. This needs
file
>>> system support (no fat32, e.g.), and support by QEMU('s I/O
driver).
>>> Once Qemu is set up to accept TRIM, you can either enable the
discard
>>> mount option, or set up an fstrim cron job.
> 
> Thanks. I read stuff about that and it's becoming clearer now. 
> 
> These pages are the most complete explanations I found:
>
http://dustymabe.com/2013/06/11/recover-space-from-vm-disk-images-by-using-discardfstrim/
>
http://dustymabe.com/2013/06/26/enabling-qemu-guest-agent-anddddd-fstrim-again/
> However, in any case, I don't see any change in the space used by the
VM
> file as seen from the host. It does not shrink.
> 
> Anything else I should do?
> 
> Anything I could check?
You should really look into using 'virt-resize' and
'virt-sparsify',
both part of the libguestfs tools suite, because they make this job a
LOT less error-prone and automatable.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org