Hello!
I'm testing user namespaces and I have quite some problem getting them to
work.
First of all, I have user namespaces support enabled in kernel:
offlinehacker:~/ $ uname -r
3.10.10
offlinehacker:~/ $ ls /proc/self/ns/
ipc@ mnt@ net@ pid@ user@ uts@
I created simple ubuntu rootfs and when I start container without idmap, so
without user namespace mappings, it works just fine:
Libivrt config:
<domain type='lxc'>
<name>helloworld</name>
<memory>102400</memory>
<os>
<type>exe</type>
<init>/bin/dash</init>
</os>
<!--<idmap>
<uid start='0' target='499' count='10'/>
<gid start='0' target='100' count='10'/>
</idmap>-->
<devices>
<console type='pty'/>
<filesystem type='mount'>
<source dir='/home/offlinehacker/rootfs'/>
<target dir='/'/>
</filesystem>
</devices>
</domain>
This is how my rootfs looks like:
offlinehacker:~/ $ ls -la rootfs
drwxr-xr-x 23 offlinehacker users 4096 sep 5 19:06 ./
drwxr-xr-x 59 offlinehacker users 4096 sep 5 19:06 ../
drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 bin/
drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 boot/
drwxr-xr-x 4 offlinehacker users 4096 avg 27 14:11 dev/
drwxr-xr-x 86 offlinehacker users 4096 sep 5 18:20 etc/
drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 home/
lrwxrwxrwx 1 offlinehacker users 33 avg 27 14:10 initrd.img ->
/boot/initrd.img-3.2.0-52-virtual
drwxr-xr-x 18 offlinehacker users 4096 avg 27 14:10 lib/
drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:10 lib64/
drwx------ 2 offlinehacker users 4096 avg 27 14:11 lost+found/
drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 media/
drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 mnt/
drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 opt/
-rw-r--r-- 1 offlinehacker vboxusers 231671365 avg 27 14:12
precise-server-cloudimg-amd64-root.tar.gz
drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 proc/
drwx------ 2 offlinehacker users 4096 sep 3 23:47 root/
drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 run/
drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 sbin/
drwxr-xr-x 2 offlinehacker users 4096 mar 5 2012 selinux/
drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 srv/
drwxr-xr-x 2 offlinehacker users 4096 apr 14 2012 sys/
drwxrwxrwt 2 offlinehacker users 4096 sep 4 01:39 tmp/
drwxr-xr-x 10 offlinehacker users 4096 avg 27 14:09 usr/
drwxr-xr-x 12 offlinehacker users 4096 sep 5 18:10 var/
lrwxrwxrwx 1 offlinehacker users 29 avg 27 14:10 vmlinuz ->
boot/vmlinuz-3.2.0-52-virtual
And this is who I am:
offlinehacker:~/ $ id
uid=499(offlinehacker) gid=67(libvirtd)
groups=100(users),1(wheel),57(networkmanager),67(libvirtd)
When I create the container with idmap uncommented I get the following
error:
offlinehacker:~/ $ virsh -c lxc:/// create helloworld.xml
error: Failed to create domain from helloworld.xml
error: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000:
19036: debug
And these are the logs:
sep 05 19:08:52 laptop libvirtd[1542]: server=0x7fc8a60ddd60
client=0x7fc8a60e68d0 msg=0x7fc8a60e9380 rerr=0x7fc89a32cd40
args=0x7fc88800b4a0 ret=0x7fc88800a1c0
sep 05 19:08:52 laptop libvirtd[1542]: priv=0x7fc8a60e91f0 conn=(nil)
sep 05 19:08:52 laptop libvirtd[1542]: name=lxc:///
sep 05 19:08:57 laptop libvirtd[1542]: Cannot recv data: Connection reset
by peer
sep 05 19:08:58 laptop libvirtd[1542]: internal error: guest failed to
start: 2013-09-05 19:08:57.781+0000: 19036: debug
Looks like .oldroot, dev, proc and sys gets created with mapped
permissions(499:100), but container fails to start.
Any help or direction how to get this working would be appriciated,
Thanks, Jaka Hudoklin!
On 09/06/2013 03:15 AM, Jaka Hudoklin wrote:> Hello! > > I'm testing user namespaces and I have quite some problem getting them to work. > > First of all, I have user namespaces support enabled in kernel: > > offlinehacker:~/ $ uname -r > 3.10.10 > offlinehacker:~/ $ ls /proc/self/ns/ > ipc@ mnt@ net@ pid@ user@ uts@ > > I created simple ubuntu rootfs and when I start container without idmap, so without user namespace mappings, it works just fine: > > Libivrt config: > > <domain type='lxc'> > <name>helloworld</name> > <memory>102400</memory> > <os> > <type>exe</type> > <init>/bin/dash</init> > </os> > <!--<idmap> > <uid start='0' target='499' count='10'/> > <gid start='0' target='100' count='10'/> > </idmap>--> > <devices> > <console type='pty'/> > <filesystem type='mount'> > <source dir='/home/offlinehacker/rootfs'/> > <target dir='/'/> > </filesystem> > </devices> > </domain> >Your configuration looks good.> This is how my rootfs looks like: > > offlinehacker:~/ $ ls -la rootfs > drwxr-xr-x 23 offlinehacker users 4096 sep 5 19:06 ./ > drwxr-xr-x 59 offlinehacker users 4096 sep 5 19:06 ../ > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 bin/ > drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 boot/ > drwxr-xr-x 4 offlinehacker users 4096 avg 27 14:11 dev/ > drwxr-xr-x 86 offlinehacker users 4096 sep 5 18:20 etc/ > drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 home/ > lrwxrwxrwx 1 offlinehacker users 33 avg 27 14:10 initrd.img -> /boot/initrd.img-3.2.0-52-virtual > drwxr-xr-x 18 offlinehacker users 4096 avg 27 14:10 lib/ > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:10 lib64/ > drwx------ 2 offlinehacker users 4096 avg 27 14:11 lost+found/ > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 media/ > drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 mnt/ > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 opt/ > -rw-r--r-- 1 offlinehacker vboxusers 231671365 avg 27 14:12 precise-server-cloudimg-amd64-root.tar.gz > drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 proc/ > drwx------ 2 offlinehacker users 4096 sep 3 23:47 root/ > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 run/ > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 sbin/ > drwxr-xr-x 2 offlinehacker users 4096 mar 5 2012 selinux/ > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 srv/ > drwxr-xr-x 2 offlinehacker users 4096 apr 14 2012 sys/ > drwxrwxrwt 2 offlinehacker users 4096 sep 4 01:39 tmp/ > drwxr-xr-x 10 offlinehacker users 4096 avg 27 14:09 usr/ > drwxr-xr-x 12 offlinehacker users 4096 sep 5 18:10 var/ > lrwxrwxrwx 1 offlinehacker users 29 avg 27 14:10 vmlinuz -> boot/vmlinuz-3.2.0-52-virtual > > And this is who I am: > offlinehacker:~/ $ id > uid=499(offlinehacker) gid=67(libvirtd) groups=100(users),1(wheel),57(networkmanager),67(libvirtd)Can this user exec //home/offlinehacker/rootfs/bin/dash successfully ?> > When I create the container with idmap uncommented I get the following error: > > offlinehacker:~/ $ virsh -c lxc:/// create helloworld.xml > error: Failed to create domain from helloworld.xml > error: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug > > And these are the logs: > sep 05 19:08:52 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e68d0 msg=0x7fc8a60e9380 rerr=0x7fc89a32cd40 args=0x7fc88800b4a0 ret=0x7fc88800a1c0 > sep 05 19:08:52 laptop libvirtd[1542]: priv=0x7fc8a60e91f0 conn=(nil) > sep 05 19:08:52 laptop libvirtd[1542]: name=lxc:/// > sep 05 19:08:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer > sep 05 19:08:58 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-05 19:08:57.781+0000: 19036: debug > > Looks like .oldroot, dev, proc and sys gets created with mapped permissions(499:100), but container fails to start. >Please enable debug mode, I need more information. http://libvirt.org/logging.html Thansk
Hello! Okay i tried again with only staticly linked busybox: offlinehacker:~/ $ /home/offlinehacker/busybox/busybox BusyBox v1.17.1 (Debian 1:1.17.1-8) multi-call binary. Copyright (C) 1998-2009 Erik Andersen, Rob Landley, Denys Vlasenko and others. Licensed under GPLv2. See source distribution for full notice. .... Again my id: uid=499(offlinehacker) gid=100(users) groups=100(users),1(wheel),57(networkmanager) My rootfs tree(/home/offlinehacker/busybox): busybox ├── [offlineh users ] busybox └── [offlineh users ] busybox-static_1.17.1-8_amd64.deb It works just fine as root and these folders gets created: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [root root ] dev ├── [root root ] .oldroot ├── [root root ] proc └── [root root ] sys When i start it with idmap with clean rootfs(dev proc sys and .oldroot deleted) i get this error, and it is a little bit different now: error: Failed to create domain from helloworld.xml error: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC And log is pretty similar: sep 06 11:24:56 laptop libvirtd[1542]: EVENT_POLL_UPDATE_HANDLE: watch=241 events=1 sep 06 11:24:57 laptop libvirtd[1542]: Skip interrupt, 1 140499747788544 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: OBJECT_REF: obj=0x7fc878000c90 sep 06 11:24:57 laptop libvirtd[1542]: server=0x7fc8a60ddd60 client=0x7fc8a60e8bb0 msg=0x7fc8a60e6970 rerr=0x7fc89a32cd40 args=0x7fc8880160a0 ret=0x7fc888016030 sep 06 11:24:57 laptop libvirtd[1542]: priv=0x7fc8a60ea3a0 conn=(nil) sep 06 11:24:57 laptop libvirtd[1542]: name=lxc:/// sep 06 11:24:57 laptop libvirtd[1542]: Cannot recv data: Connection reset by peer sep 06 11:24:57 laptop libvirtd[1542]: internal error: guest failed to start: 2013-09-06 11:24:57.088+0000: 5794: debug : virFileC Rootfs after failed creation looks like this: busybox ├── [offlineh users ] busybox ├── [offlineh users ] busybox-static_1.17.1-8_amd64.deb ├── [offlineh users ] .oldroot ├── [offlineh users ] proc └── [offlineh users ] sys I have debugging enabled, at least LIBVIRT_DEBUG is set to 1 and i get much more messages. If there's any my granular debug please let me know. PS: I forgot to mention my version of libvirt is 1.1.2 Thanks, Jaka! On Fri, Sep 6, 2013 at 3:41 AM, Gao feng <gaofeng@cn.fujitsu.com> wrote:> On 09/06/2013 03:15 AM, Jaka Hudoklin wrote: > > Hello! > > > > I'm testing user namespaces and I have quite some problem getting them > to work. > > > > First of all, I have user namespaces support enabled in kernel: > > > > offlinehacker:~/ $ uname -r > > 3.10.10 > > offlinehacker:~/ $ ls /proc/self/ns/ > > ipc@ mnt@ net@ pid@ user@ uts@ > > > > I created simple ubuntu rootfs and when I start container without idmap, > so without user namespace mappings, it works just fine: > > > > Libivrt config: > > > > <domain type='lxc'> > > <name>helloworld</name> > > <memory>102400</memory> > > <os> > > <type>exe</type> > > <init>/bin/dash</init> > > </os> > > <!--<idmap> > > <uid start='0' target='499' count='10'/> > > <gid start='0' target='100' count='10'/> > > </idmap>--> > > <devices> > > <console type='pty'/> > > <filesystem type='mount'> > > <source dir='/home/offlinehacker/rootfs'/> > > <target dir='/'/> > > </filesystem> > > </devices> > > </domain> > > > > Your configuration looks good. > > > This is how my rootfs looks like: > > > > offlinehacker:~/ $ ls -la rootfs > > drwxr-xr-x 23 offlinehacker users 4096 sep 5 19:06 ./ > > drwxr-xr-x 59 offlinehacker users 4096 sep 5 19:06 ../ > > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 bin/ > > drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 boot/ > > drwxr-xr-x 4 offlinehacker users 4096 avg 27 14:11 dev/ > > drwxr-xr-x 86 offlinehacker users 4096 sep 5 18:20 etc/ > > drwxr-xr-x 3 offlinehacker users 4096 avg 27 14:11 home/ > > lrwxrwxrwx 1 offlinehacker users 33 avg 27 14:10 initrd.img > -> /boot/initrd.img-3.2.0-52-virtual > > drwxr-xr-x 18 offlinehacker users 4096 avg 27 14:10 lib/ > > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:10 lib64/ > > drwx------ 2 offlinehacker users 4096 avg 27 14:11 lost+found/ > > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 media/ > > drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 mnt/ > > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 opt/ > > -rw-r--r-- 1 offlinehacker vboxusers 231671365 avg 27 14:12 > precise-server-cloudimg-amd64-root.tar.gz > > drwxr-xr-x 2 offlinehacker users 4096 apr 19 2012 proc/ > > drwx------ 2 offlinehacker users 4096 sep 3 23:47 root/ > > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 run/ > > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:11 sbin/ > > drwxr-xr-x 2 offlinehacker users 4096 mar 5 2012 selinux/ > > drwxr-xr-x 2 offlinehacker users 4096 avg 27 14:09 srv/ > > drwxr-xr-x 2 offlinehacker users 4096 apr 14 2012 sys/ > > drwxrwxrwt 2 offlinehacker users 4096 sep 4 01:39 tmp/ > > drwxr-xr-x 10 offlinehacker users 4096 avg 27 14:09 usr/ > > drwxr-xr-x 12 offlinehacker users 4096 sep 5 18:10 var/ > > lrwxrwxrwx 1 offlinehacker users 29 avg 27 14:10 vmlinuz -> > boot/vmlinuz-3.2.0-52-virtual > > > > And this is who I am: > > offlinehacker:~/ $ id > > uid=499(offlinehacker) gid=67(libvirtd) > groups=100(users),1(wheel),57(networkmanager),67(libvirtd) > > Can this user exec //home/offlinehacker/rootfs/bin/dash successfully ? > > > > > When I create the container with idmap uncommented I get the following > error: > > > > offlinehacker:~/ $ virsh -c lxc:/// create helloworld.xml > > error: Failed to create domain from helloworld.xml > > error: internal error: guest failed to start: 2013-09-05 > 19:08:57.781+0000: 19036: debug > > > > And these are the logs: > > sep 05 19:08:52 laptop libvirtd[1542]: server=0x7fc8a60ddd60 > client=0x7fc8a60e68d0 msg=0x7fc8a60e9380 rerr=0x7fc89a32cd40 > args=0x7fc88800b4a0 ret=0x7fc88800a1c0 > > sep 05 19:08:52 laptop libvirtd[1542]: priv=0x7fc8a60e91f0 conn=(nil) > > sep 05 19:08:52 laptop libvirtd[1542]: name=lxc:/// > > sep 05 19:08:57 laptop libvirtd[1542]: Cannot recv data: Connection > reset by peer > > sep 05 19:08:58 laptop libvirtd[1542]: internal error: guest failed to > start: 2013-09-05 19:08:57.781+0000: 19036: debug > > > > Looks like .oldroot, dev, proc and sys gets created with mapped > permissions(499:100), but container fails to start. > > > > Please enable debug mode, I need more information. > > http://libvirt.org/logging.html > > Thansk >