libvirt users - Aug 2017 - external snapshot is missing object secrets

Hello,

I have virtual machines running with a ceph storage backend.

When creating an external qcow2 snapshot with a libvirt version without
support for the new object secret passing, the backing file info would
list the ceph secret in plain,e.g.

# virsh snapshot-create-as vm-123 --no-metadata --disk-only --diskspec
sda,file=/var/lib/libvirt/qemu/snapshot/vm-123-wrapper.qcow2

# qemu-img info /var/lib/libvirt/qemu/snapshot/vm-123-wrapper.qcow2
...
backing file:
rbd:vms_pool0/disk-123:id=libvirt:key=SECRET:auth_supported=cephx\;none:mon_host=192.168.1.1\:6789\;192.168.1.11\:6789\;192.168.1.21\:6789

While this is problematic from a security perspective (and one of the
reasons for the new method), it enabled starting the virtual machine
again in case it died or got powered off.

With the newer libvirt object secret passing the backing file
of the qcow image only references the secret id.

# qemu-img info /var/lib/libvirt/qemu/snapshot/vm-123-wrapper.qcow2
...
backing file: json:{"driver": "raw", "file":
{"password-secret":
"scsi0-0-0-0-secret0", "pool": "vms_pool",
"image": "disk-123",
"driver": "rbd", "user": "libvirt",
"=keyvalue-pairs":
"[\"auth_supported\", \"cephx;none\",
\"mon_host\",
\"192.168.1.1:6789;192.168.1.11:6789;192.168.1.21:6789\"]"}}

This is fine as long as the virtual machine is running and with
qemu-2.10 it is even possible to block-commit this external snapshot
(Yeah!).

However, should the VM die or be powered off, it is now longer possible
to start the domain or at least recover the data:

Could not open backing file: No secret with id 'scsi0-0-0-0-secret0'

I guess this problems happens with any disk type that is accessed with
object secrets, which is why I would consider this a bug.

The question is, should/can this be fixed in libvirt or qemu?

I think libvirt should create the snapshot file with the object secret
stored in a persistent file (and reference this file in the backing file
definition)


Best regards,
Markus

On Fri, Aug 25, 2017 at 09:49:10 +0200, Markus Schade
wrote:
> Hello,
> 
> I have virtual machines running with a ceph storage backend.
> 
> When creating an external qcow2 snapshot with a libvirt version without
> support for the new object secret passing, the backing file info would
> list the ceph secret in plain,e.g.
> 
> # virsh snapshot-create-as vm-123 --no-metadata --disk-only --diskspec
> sda,file=/var/lib/libvirt/qemu/snapshot/vm-123-wrapper.qcow2
> 
> # qemu-img info /var/lib/libvirt/qemu/snapshot/vm-123-wrapper.qcow2
> ...
> backing file:
>
rbd:vms_pool0/disk-123:id=libvirt:key=SECRET:auth_supported=cephx\;none:mon_host=192.168.1.1\:6789\;192.168.1.11\:6789\;192.168.1.21\:6789
> 
> While this is problematic from a security perspective (and one of the
> reasons for the new method), it enabled starting the virtual machine
> again in case it died or got powered off.
> 
> With the newer libvirt object secret passing the backing file
> of the qcow image only references the secret id.
> 
> # qemu-img info /var/lib/libvirt/qemu/snapshot/vm-123-wrapper.qcow2
> ...
> backing file: json:{"driver": "raw", "file":
{"password-secret":
> "scsi0-0-0-0-secret0", "pool": "vms_pool",
"image": "disk-123",
> "driver": "rbd", "user": "libvirt",
"=keyvalue-pairs":
> "[\"auth_supported\", \"cephx;none\",
\"mon_host\",
> \"192.168.1.1:6789;192.168.1.11:6789;192.168.1.21:6789\"]"}}
> 
> This is fine as long as the virtual machine is running and with
> qemu-2.10 it is even possible to block-commit this external snapshot
> (Yeah!).
> 
> However, should the VM die or be powered off, it is now longer possible
> to start the domain or at least recover the data:
> 
> Could not open backing file: No secret with id
'scsi0-0-0-0-secret0'
Yes, the issue here is that even if qemu records the secret alias,
libvirt does not remember it in the XML and thus does not create the
objects.
> I guess this problems happens with any disk type that is accessed with
> object secrets, which is why I would consider this a bug.
> 
> The question is, should/can this be fixed in libvirt or qemu?
I'm currently working on this. This needs to be fixed in libvirt by
tracking the full backing chain.
> 
> I think libvirt should create the snapshot file with the object secret
> stored in a persistent file (and reference this file in the backing file
> definition)
Libvirt will store the authentication credentials in the XML along with
the backing chain. With that you can specify the authentication for
every image, so a new start of the VM should work as expected.

The {"password-secret": field in the backing chain then will be
ignored
and libvirt will create a new secret object with the proper name.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20170913/0462ef23/attachment.sig>