Running libvirtd inside chroot (mock to be precise)

Marcin Sobczyk
Wed, 13 May 2020 10:11:08 -0000

Hi,

I was wondering whether it's possible to run libvirtd inside a chroot
environment.
The assumption is that only one instance of libvirtd would be running on
the machine at a time, but still, inside chroot.

Currently in my chroot env I have:

- /dev/kvm added with mknod
- /dev/vhost-net added with mknod
- mounted:
   - /dev/net
   - /dev/shm
   - /run/dbus

When I run libvirtd in non-daemon mode I get one error:

2020-05-13 10:06:27.480+0000: 28555: error : virNetDevBridgeCreate:484 :
error creating bridge interface virbr0: File exists

but it seems to me that libvirtd can live with that.

When I try to run a VM however I get a qemu error reported:

2020-05-13 08:56:51.643+0000: starting up libvirt version: 4.5.0,
package: 35.3.module_el8.1.0+297+df420408 (CentOS Buildsys
<bugs@centos.org>, 2020-04-10-11:06:38, ), qemu version:
2.12.0qemu-kvm-2.12.0-88.module_el8.1.0+297+df420408.3, kernel:
4.18.0-80.el8.x86_64, hostname: .....
LC_ALL=C
PATH=/var/lib/ci_toolbox:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin
HOME=/root/ovirt-system-tests USER=root LOGNAME=root QEMU_AUDIO_DRV=none
/usr/libexec/qemu-kvm -name
guest=992415aa-lago-basic-suite-master-engine,debug-threads=on -S
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-992415aa-lago-basic-/master-key.aes
-machine
pc-i440fx-rhel7.6.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off
-cpu host -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1
-object iothread,id=iothread1 -numa node,nodeid=0,cpus=0,mem=2048 -numa
node,nodeid=1,cpus=1,mem=2048 -uuid 10ed4b42-bb14-4b03-a411-c62fb1181c2d
-display none -no-user-config -nodefaults -chardev
socket,id=charmonitor,fd=31,server,nowait -mon
chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
-global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot
menu=off,strict=on -device
virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=2,bus=pci.0,addr=0x5
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
file=/root/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-engine_root.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,serial=1,cache=writeback,discard=unmap
-device
virtio-blk-pci,iothread=iothread1,scsi=off,num-queues=1,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on
-drive
file=/root/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-engine_nfs.raw,format=raw,if=none,id=drive-scsi0-0-0-0,serial=2,cache=writeback,discard=unmap
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,write-cache=on
-drive
file=/root/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-engine_iscsi.raw,format=raw,if=none,id=drive-scsi0-0-0-2,serial=3,cache=writeback,discard=unmap
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2,write-cache=on
-netdev tap,fds=33:34,id=hostnet0,vhost=on,vhostfds=35:36 -device
virtio-net-pci,mq=on,vectors=6,netdev=hostnet0,id=net0,mac=54:52:c0:a8:c8:02,bus=pci.0,addr=0x3
-netdev tap,fds=37:38,id=hostnet1,vhost=on,vhostfds=39:40 -device
virtio-net-pci,mq=on,vectors=6,netdev=hostnet1,id=net1,mac=54:52:c0:a8:ca:02,bus=pci.0,addr=0x4
-chardev socket,id=charchannel0,fd=41,server,nowait -device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
-chardev pty,id=charconsole0 -device
virtconsole,chardev=charconsole0,id=console0 -device
cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object
rng-random,id=objrng0,filename=/dev/urandom -device
virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x9 -sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny
-msg timestamp=on
2020-05-13 08:56:51.643+0000: Domain id=1 is tainted: high-privileges
2020-05-13 08:56:51.643+0000: Domain id=1 is tainted: host-cpu
libvirt: QEMU Driver error : internal error: Unable to find any /dev mount
2020-05-13 08:56:51.654+0000: shutting down, reason=failed

And this one currently is blocking me.
Did anyone try such a setup?
Any help would be appreciated.

Regards, Marcin