mxs kolo
2018-Apr-10 12:01 UTC
[libvirt-users] unprivileged container has wrong owners inside in /proc/ and /sys
We run unprivileged lxc containers (libvirt based) with next config: ... <idmap> <uid start='0' target='65535' count='65535'/> <gid start='0' target='65535' count='65535'/> </idmap> ... <devices> <emulator>/usr/libexec/libvirt_lxc</emulator> <filesystem type='mount' accessmode='passthrough'> <source dir='/var/lib/libvirt/lxc/test1'/> <target dir='/'/> </filesystem> ... </devices> Before start we need mount container's LVM: mount /dev/data/test1 /var/lib/libvirt/lxc/test1 And shift uid/gid: ./uidmapshift -b /var/lib/libvirt/lxc/test1 0 65535 65535 (uidmapshift.c is simple utility, found in LXD community) As result, our FS permissions look from hardware node as: [root@ops-node01.infra]# ls -la /var/lib/libvirt/lxc/test1/ total 8 dr-xr-xr-x 19 65535 65535 275 Apr 9 18:55 . drwxrwxrwx 3 root root 26 Apr 9 19:32 .. -rw-r--r-- 1 65535 65535 0 Mar 30 18:20 .autorelabel lrwxrwxrwx 1 65535 65535 7 Mar 2 2017 bin -> usr/bin dr-xr-xr-x 2 65535 65535 6 Nov 5 2016 boot drwxr-xr-x 4 65535 65535 206 Mar 30 18:03 dev drwxr-xr-x 59 65535 65535 4096 Apr 9 18:57 etc drwxr-xr-x 2 65535 65535 6 Nov 5 2016 home lrwxrwxrwx 1 65535 65535 7 Mar 2 2017 lib -> usr/lib lrwxrwxrwx 1 65535 65535 9 Mar 2 2017 lib64 -> usr/lib64 drwxr-xr-x 2 65535 65535 6 Nov 5 2016 media drwxr-xr-x 2 65535 65535 6 Nov 5 2016 mnt drwxr-xr-x 2 65535 65535 6 Apr 9 18:55 .oldroot drwxr-xr-x 3 65535 65535 24 Mar 2 2017 opt dr-xr-xr-x 2 65535 65535 6 Nov 5 2016 proc dr-xr-x--- 4 65535 65535 172 Apr 9 18:57 root drwxr-xr-x 13 65535 65535 178 Mar 30 18:03 run lrwxrwxrwx 1 65535 65535 8 Mar 2 2017 sbin -> usr/sbin drwxr-xr-x 2 65535 65535 21 Mar 30 18:03 selinux drwxr-xr-x 2 65535 65535 6 Nov 5 2016 srv dr-xr-xr-x 2 65535 65535 6 Nov 5 2016 sys drwxrwxrwt 7 65535 65535 93 Apr 6 21:08 tmp drwxr-xr-x 13 65535 65535 155 Mar 2 2017 usr drwxr-xr-x 18 65535 65535 254 Mar 30 18:20 var Container running, but some FS objects inside get owner 65534.65534: [root@test1 ~]# find / -uid 65534 -ls | wc -l 40084 It's mostly /sys and /proc objects: [root@test1 ~]# ls -la / total 8 dr-xr-xr-x 19 root root 275 Apr 9 18:55 . dr-xr-xr-x 19 root root 275 Apr 9 18:55 .. -rw-r--r-- 1 root root 0 Mar 30 18:20 .autorelabel drwxr-xr-x 2 root root 6 Apr 9 18:55 .oldroot lrwxrwxrwx 1 root root 7 Mar 2 2017 bin -> usr/bin dr-xr-xr-x 2 root root 6 Nov 5 2016 boot drwxr-xr-x 6 root root 440 Apr 9 20:40 dev drwxr-xr-x 59 root root 4096 Apr 9 18:57 etc drwxr-xr-x 2 root root 6 Nov 5 2016 home lrwxrwxrwx 1 root root 7 Mar 2 2017 lib -> usr/lib lrwxrwxrwx 1 root root 9 Mar 2 2017 lib64 -> usr/lib64 drwxr-xr-x 2 root root 6 Nov 5 2016 media drwxr-xr-x 2 root root 6 Nov 5 2016 mnt drwxr-xr-x 3 root root 24 Mar 2 2017 opt dr-xr-xr-x 348 65534 65534 0 Apr 9 20:40 proc dr-xr-x--- 4 root root 172 Apr 9 18:57 root drwxrwxrwt 15 root root 360 Apr 9 20:40 run lrwxrwxrwx 1 root root 8 Mar 2 2017 sbin -> usr/sbin drwxr-xr-x 2 root root 21 Mar 30 18:03 selinux drwxr-xr-x 2 root root 6 Nov 5 2016 srv dr-xr-xr-x 13 65534 65534 0 Apr 9 20:40 sys drwxrwxrwt 7 root root 93 Apr 6 21:08 tmp drwxr-xr-x 13 root root 155 Mar 2 2017 usr drwxr-xr-x 18 root root 254 Mar 30 18:20 var [root@test1 ~]# find /sys/ -uid 65534 -ls | wc -l 36871 [root@t1 ~]# find /proc -uid 65534 -ls | wc -l 3200 [root@test1 ~]# find /dev -uid 65534 -ls 2121875 0 drwxrwxrwt 2 65534 65534 40 Apr 10 13:57 /dev/mqueue It's feature ? Look like libvirt make all this objects from real root with owner root.root You can't call virLXCControllerSetupUserns() before make this objects, right ? Because you need root permissions for some operations. After you set uid range for process 65535-131070, we can't show object with uid.gid 0 and then kernel use values from /proc/sys/kernel/overflowuid and /proc/sys/kernel/overflowgid. It's can probably affect some process in container ? As solution is shift uid/gid _after_ make all objects and _before_ enter userNS ? As workaround we can try set 0 for overflowuid and overflowgid p.s. We use type='mount' instead type='block' because in unprivileged container mount() call failed, more detailes in https://bugzilla.redhat.com/show_bug.cgi?id=1328946 b.r. Maxim Kozin