-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi lxc folks, the symptom my libvirt LXC container suffers from is: root@depot:/dev/net# ls -la total 0 drwxr-xr-x 2 root root 40 Jun 29 16:26 . drwxr-xr-x 5 root root 480 Jun 29 16:26 .. root@depot:/dev/net# mknod tun c 10 200 mknod: `tun': Operation not permitted The host is an up-to-date AMD64 Ubuntu raring on 3.8.0-25-generic that was formerly installed from precise and then upgraded. The guest is Ubuntu precise; however, I see the same symptom in another raring container on the same host. What I tried to resolve this: 1) On the host, I echoed various stuff to the cgroup device files: cd /sys/fs/cgroup/devices/libvirt/lxc echo "c 10:200 rwm" > devices.allow echo "c 10:200 rwm" > depot/devices.allow echo a > depot/devices.allow ... and I see the successful results in depot/devices.list, but no success. 2) I inserted a line "/dev/net/tun rwk," into /etc/apparmor.d/abstractions/lxc/container-base - no change. (I know, it seems kind of pointless - because it's about permissions to a device, not a path. You may deduce my desperation from this ... SELinux is not active. Mounts on /dev look normal to me: devfs on /dev type tmpfs (rw,mode=0755) devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) devpts on /dev/ptmx type devpts (rw,nosuid,relatime,gid=5,mode=620,ptmxmode=666) cgroup on /sys/fs/cgroup/devices type cgroup (rw,relatime,devices) Now comes the weird part: Once I umount /dev inside the container, the "hidden" /dev appears which contains a usable /dev/net/tun. So the mknod problem is probably due to the dropped capabilities - but why/how mounts the container a more restricted /dev on top of the prepared one ...? Thanks & regards Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlHYdysACgkQiMyIQtYO79yUMQCfUrbaAWcZsqYkopRL6F2evWdq XnMAoOq6/wopgBGZniWDGnIiYBEsFX61 =BIr9 -----END PGP SIGNATURE-----
Daniel P. Berrange
2013-Jul-08 09:41 UTC
Re: [libvirt-users] Permission problem with /dev/net/tun
On Sat, Jul 06, 2013 at 09:59:39PM +0200, Thomas Karcher wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi lxc folks, > > the symptom my libvirt LXC container suffers from is: > > root@depot:/dev/net# ls -la > total 0 > drwxr-xr-x 2 root root 40 Jun 29 16:26 . > drwxr-xr-x 5 root root 480 Jun 29 16:26 .. > > root@depot:/dev/net# mknod tun c 10 200 > mknod: `tun': Operation not permitted > > The host is an up-to-date AMD64 Ubuntu raring on 3.8.0-25-generic that > was formerly installed from precise and then upgraded. The guest is > Ubuntu precise; however, I see the same symptom in another raring > container on the same host. > > What I tried to resolve this: > > 1) On the host, I echoed various stuff to the cgroup device files: > > cd /sys/fs/cgroup/devices/libvirt/lxc > echo "c 10:200 rwm" > devices.allow > echo "c 10:200 rwm" > depot/devices.allow > echo a > depot/devices.allow > > ... and I see the successful results in depot/devices.list, but no > success. > > 2) I inserted a line "/dev/net/tun rwk," into > /etc/apparmor.d/abstractions/lxc/container-base - no change. (I know, > it seems kind of pointless - because it's about permissions to a > device, not a path. You may deduce my desperation from this ... > > SELinux is not active. Mounts on /dev look normal to me: > > devfs on /dev type tmpfs (rw,mode=0755) > devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) > devpts on /dev/ptmx type devpts > (rw,nosuid,relatime,gid=5,mode=620,ptmxmode=666) > cgroup on /sys/fs/cgroup/devices type cgroup (rw,relatime,devices) > > Now comes the weird part: Once I umount /dev inside the container, the > "hidden" /dev appears which contains a usable /dev/net/tun. So the > mknod problem is probably due to the dropped capabilities - but > why/how mounts the container a more restricted /dev on top of the > prepared one ...?Allowing the container direct access to the hosts' /dev would be a security flaw, so libvirt sets up a private /dev for the container. Allowing the container to use mknod would also be insecure, so we blocking mknod using both cgroups device ACL, and also droping the CAP_MKNOD capability. http://libvirt.org/drvlxc.html#devnodes Any device that the container is authorized to access per the XML configuration, will be pre-created in the container's /dev. To explicitly allow /dev/net/tun you need to tell libvirt about it. http://libvirt.org/formatdomain.html#elementsHostDevCaps Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Thomas Karcher
2013-Jul-08 21:51 UTC
Re: [libvirt-users] Permission problem with /dev/net/tun
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Daniel, On 07/08/2013 11:41 AM, Daniel P. Berrange wrote:>> the symptom my libvirt LXC container suffers from is: >> root@depot:/dev/net# ls -la total 0 drwxr-xr-x 2 root root 40 >> Jun 29 16:26 . drwxr-xr-x 5 root root 480 Jun 29 16:26 .. >> root@depot:/dev/net# mknod tun c 10 200 mknod: `tun': Operation >> not permitted > Allowing the container direct access to the hosts' /dev would be a > security flaw, so libvirt sets up a private /dev for the > container. Allowing the container to use mknod would also be > insecure, so we blocking mknod using both cgroups device ACL, and > also droping the CAP_MKNOD capability. > http://libvirt.org/drvlxc.html#devnodesGood to know.> Any device that the container is authorized to access per the XML > configuration, will be pre-created in the container's /dev. To > explicitly allow /dev/net/tun you need to tell libvirt about it. > http://libvirt.org/formatdomain.html#elementsHostDevCapsThanks! I extended the 'devices' section as follows: <hostdev mode='capabilities' type='misc'> <source> <char>/dev/net/tun</char> </source> </hostdev> ... because even though /dev/net/tun is used for networking, it appears as a character device. (Btw: The documentation says in the hostdev section: ''For block/character device passthrough mode is always "capabilities" and type is "block" for a block device, "char" for a character device and "net" for a host network interface.'' When I specify type='char', I get an error from virsh.) With this XML, I can define the container. But upon start, I get the following error message: Fehler: internal error guest failed to start: PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=f3602503-9603-24aa-7dd8-fccc830a802b LIBVIRT_LXC_UUID=f3602503-9603-24aa-7dd8-fccc830a802b LIBVIRT_LXC_NAME=depot /sbin/init 2013-07-08 21:36:50.735+0000: 1: info : libvirt version: 1.0.2 2013-07-08 21:36:50.735+0000: 1: error : lxcContainerSetupHostdevCapsMisc:1490 : Unable to create device /dev/net/tun: No such file or directory 2013-07-08 21:36:50.744+0000: 19537: info : libvirt version: 1.0.2 2013-07-08 21:36:50.744+0000: 19537: error : virCommandWait:2287 : internal error Child process (ip link set veth6 netns 19538) unexpected exit status 2: RTNETLINK answers: No such process 2013-07-08 21:36:50.786+0000: 19537: error : virCommandWait:2287 : internal error Child process (ip link del veth4) unexpected exit status 1: Cannot find device "veth4" On the host, /dev/net/tun exists as character device: root@main:~# ls -la /dev/net/tun crw-rw-rwT 1 root root 10, 200 Jul 8 23:45 /dev/net/tun What am I doing wrong ...? Thanks Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlHbNGwACgkQiMyIQtYO79zOwgCdFVzn0JopHK+ZY2ZshgZnuz6L Yx8An3BL/2sfLTFSs39yNkB0FXzq9K/4 =xRtO -----END PGP SIGNATURE-----