libvirt users - Jan 2020 - Volume file permissions and huge volume downloads

Hi all:

I am using the libvirt version that comes with Ubuntu 18.04.3 LTS.

I want to backup a virtual machine in a foolproof way:

- Gracefully shutdown the VM.
- Backup the disk image.
- Restart the VM.

I wrote the following script to do that:

https://github.com/rdiez/Tools/blob/master/VirtualMachineManager/BackupVm.sh

Writing that script was difficult enough because of the virsh limitations (in my
opinion) described in the comments. But at least it works.

The main problem is that libvirt sets the ownership and permissions of volume
files in such a way that a standard user cannot access them,
even if it is member of the 'libvirt' group.

While the VM is running, volume file permissions are like this:
   -rw-r----- 1 libvirt-qemu kvm [...] /var/lib/libvirt/images/YourVmDisk.qcow2
When the VM is shutoff:
   -rw-r----- 1 root root [...] /var/lib/libvirt/images/YourVmDisk.qcow2

The trouble is, I want to access that .qcow2 file when the VM is shutoff.

I would really like not to run my script as root. I could not find a way to
specify the permissions for the .qcow2 files, so I tried editing
the group ownership for the whole pool:

virsh pool-edit default

<target>
   <path>/var/lib/libvirt/images</path>
   <permissions>
     <mode>0711</mode>
     <owner>0</owner>
     <group>131</group>
   </permissions>
</target>

Setting <group> to "libvirt" did not work (XML validation
error), so I tried the numeric ID of that group (131).

That did not work. When I restart libvirt with "service libvirtd
restart", my changes to the pool XML file disappear (!).


Later on, I found out that you can download a volume like this:

   virsh  vol-download  --pool default  YourVmDisk.qcow2  YourVmDisk.qcow2

The trouble is that I created the virtual disk with a maximum size of 120 GB. I
copied it around a few times, so I think it has lost any
"sparseness" inside. Command "virsh vol-download" takes ages
and downloads all those 120 GB. My script uses "qemu-img convert",
which
discards all unused space and writes just 17 GB of data, and that is without
turning compression on. The whole backup takes seconds instead
of minutes.


Can somebody help me with these issues?

Thanks in advance,
   rdiez

On Fri, Jan 17, 2020 at 04:30:09PM +0100, R. Diez wrote:
>Hi all:
>
>I am using the libvirt version that comes with Ubuntu 18.04.3 LTS.
>
>I want to backup a virtual machine in a foolproof way:
>
>- Gracefully shutdown the VM.
>- Backup the disk image.
>- Restart the VM.
>
>I wrote the following script to do that:
>
>https://github.com/rdiez/Tools/blob/master/VirtualMachineManager/BackupVm.sh
>
>Writing that script was difficult enough because of the virsh limitations
(in my opinion) described in the comments. But at least it works.
>
>The main problem is that libvirt sets the ownership and permissions of
volume files in such a way that a standard user cannot access them,
>even if it is member of the 'libvirt' group.
>
Yes, you are letting libvirt do its dynamic_ownership thing.  You can turn that
off, but you don't want to do that as it would create more problems for you
than
it would solve issues.  But don't worry.
>While the VM is running, volume file permissions are like this:
>   -rw-r----- 1 libvirt-qemu kvm [...]
/var/lib/libvirt/images/YourVmDisk.qcow2
>When the VM is shutoff:
>   -rw-r----- 1 root root [...] /var/lib/libvirt/images/YourVmDisk.qcow2
>
If you have new enough libvirt and just change the permissions on your disk file
once to who you want to have access to that when it is not used in any VM, then
(if your FS supports it etc.) it will be reverted to whatever the permissions
were before.
>The trouble is, I want to access that .qcow2 file when the VM is shutoff.
>
>I would really like not to run my script as root. I could not find a way to
specify the permissions for the .qcow2 files, so I tried editing
>the group ownership for the whole pool:
>
>virsh pool-edit default
>
><target>
>   <path>/var/lib/libvirt/images</path>
The pool represents the directory here, so that won't help you,
>   <permissions>
>     <mode>0711</mode>
>     <owner>0</owner>
>     <group>131</group>
>   </permissions>
also the permissions are not parsed in this case I think.  I might be wrong in
this part and it would change something if you did pool-build or something, but
I got carried away.
></target>
>
>Setting <group> to "libvirt" did not work (XML validation
error), so I tried the numeric ID of that group (131).
>
>That did not work. When I restart libvirt with "service libvirtd
restart", my changes to the pool XML file disappear (!).
>
OK, so I was right.
>
>Later on, I found out that you can download a volume like this:
>
>   virsh  vol-download  --pool default  YourVmDisk.qcow2  YourVmDisk.qcow2
>
>The trouble is that I created the virtual disk with a maximum size of 120
GB. I copied it around a few times, so I think it has lost any
>"sparseness" inside. Command "virsh vol-download" takes
ages and downloads all those 120 GB. My script uses "qemu-img
convert", which
>discards all unused space and writes just 17 GB of data, and that is without
turning compression on. The whole backup takes seconds instead
>of minutes.
>
qemu-img convert does an expensive scan for zeroes.  It's good that it works
even for non-sparse volumes, but you can still do better.  You can sparsify the
image where it is by using virt-sparsify.  Then, you can use vol-download and
specify --sparse.

Look at the manual for each command, sudo responsively and have a nice day.
>
>Can somebody help me with these issues?
>
I hope I did, but there will be others to help as well, so feel free to ask if
that is not enough.  But don't forget there is lot hidden in those manuals,
for
example.
>Thanks in advance,
>   rdiez
>
>