Thomas Karcher
2013-Jul-08 21:51 UTC
Re: [libvirt-users] Permission problem with /dev/net/tun
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Daniel, On 07/08/2013 11:41 AM, Daniel P. Berrange wrote:>> the symptom my libvirt LXC container suffers from is: >> root@depot:/dev/net# ls -la total 0 drwxr-xr-x 2 root root 40 >> Jun 29 16:26 . drwxr-xr-x 5 root root 480 Jun 29 16:26 .. >> root@depot:/dev/net# mknod tun c 10 200 mknod: `tun': Operation >> not permitted > Allowing the container direct access to the hosts' /dev would be a > security flaw, so libvirt sets up a private /dev for the > container. Allowing the container to use mknod would also be > insecure, so we blocking mknod using both cgroups device ACL, and > also droping the CAP_MKNOD capability. > http://libvirt.org/drvlxc.html#devnodesGood to know.> Any device that the container is authorized to access per the XML > configuration, will be pre-created in the container's /dev. To > explicitly allow /dev/net/tun you need to tell libvirt about it. > http://libvirt.org/formatdomain.html#elementsHostDevCapsThanks! I extended the 'devices' section as follows: <hostdev mode='capabilities' type='misc'> <source> <char>/dev/net/tun</char> </source> </hostdev> ... because even though /dev/net/tun is used for networking, it appears as a character device. (Btw: The documentation says in the hostdev section: ''For block/character device passthrough mode is always "capabilities" and type is "block" for a block device, "char" for a character device and "net" for a host network interface.'' When I specify type='char', I get an error from virsh.) With this XML, I can define the container. But upon start, I get the following error message: Fehler: internal error guest failed to start: PATH=/bin:/sbin TERM=linux container=lxc-libvirt container_uuid=f3602503-9603-24aa-7dd8-fccc830a802b LIBVIRT_LXC_UUID=f3602503-9603-24aa-7dd8-fccc830a802b LIBVIRT_LXC_NAME=depot /sbin/init 2013-07-08 21:36:50.735+0000: 1: info : libvirt version: 1.0.2 2013-07-08 21:36:50.735+0000: 1: error : lxcContainerSetupHostdevCapsMisc:1490 : Unable to create device /dev/net/tun: No such file or directory 2013-07-08 21:36:50.744+0000: 19537: info : libvirt version: 1.0.2 2013-07-08 21:36:50.744+0000: 19537: error : virCommandWait:2287 : internal error Child process (ip link set veth6 netns 19538) unexpected exit status 2: RTNETLINK answers: No such process 2013-07-08 21:36:50.786+0000: 19537: error : virCommandWait:2287 : internal error Child process (ip link del veth4) unexpected exit status 1: Cannot find device "veth4" On the host, /dev/net/tun exists as character device: root@main:~# ls -la /dev/net/tun crw-rw-rwT 1 root root 10, 200 Jul 8 23:45 /dev/net/tun What am I doing wrong ...? Thanks Thomas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlHbNGwACgkQiMyIQtYO79zOwgCdFVzn0JopHK+ZY2ZshgZnuz6L Yx8An3BL/2sfLTFSs39yNkB0FXzq9K/4 =xRtO -----END PGP SIGNATURE-----
On 07/09/2013 05:51 AM, Thomas Karcher wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi Daniel, > > On 07/08/2013 11:41 AM, Daniel P. Berrange wrote: >>> the symptom my libvirt LXC container suffers from is: >>> root@depot:/dev/net# ls -la total 0 drwxr-xr-x 2 root root 40 >>> Jun 29 16:26 . drwxr-xr-x 5 root root 480 Jun 29 16:26 .. >>> root@depot:/dev/net# mknod tun c 10 200 mknod: `tun': Operation >>> not permitted >> Allowing the container direct access to the hosts' /dev would be a >> security flaw, so libvirt sets up a private /dev for the >> container. Allowing the container to use mknod would also be >> insecure, so we blocking mknod using both cgroups device ACL, and >> also droping the CAP_MKNOD capability. >> http://libvirt.org/drvlxc.html#devnodes > > Good to know. > >> Any device that the container is authorized to access per the XML >> configuration, will be pre-created in the container's /dev. To >> explicitly allow /dev/net/tun you need to tell libvirt about it. >> http://libvirt.org/formatdomain.html#elementsHostDevCaps > > Thanks! > > I extended the 'devices' section as follows: > > <hostdev mode='capabilities' type='misc'> > <source> > <char>/dev/net/tun</char> > </source> > </hostdev> > > ... because even though /dev/net/tun is used for networking, it > appears as a character device. (Btw: The documentation says in the > hostdev section: ''For block/character device passthrough mode is > always "capabilities" and type is "block" for a block device, "char" > for a character device and "net" for a host network interface.'' When > I specify type='char', I get an error from virsh.) > > With this XML, I can define the container. But upon start, I get the > following error message: > > Fehler: internal error guest failed to start: PATH=/bin:/sbin > TERM=linux container=lxc-libvirt > container_uuid=f3602503-9603-24aa-7dd8-fccc830a802b > LIBVIRT_LXC_UUID=f3602503-9603-24aa-7dd8-fccc830a802b > LIBVIRT_LXC_NAME=depot /sbin/init > 2013-07-08 21:36:50.735+0000: 1: info : libvirt version: 1.0.2 > 2013-07-08 21:36:50.735+0000: 1: error : > lxcContainerSetupHostdevCapsMisc:1490 : Unable to create device > /dev/net/tun: No such file or directory > 2013-07-08 21:36:50.744+0000: 19537: info : libvirt version: 1.0.2 > 2013-07-08 21:36:50.744+0000: 19537: error : virCommandWait:2287 : > internal error Child process (ip link set veth6 netns 19538) > unexpected exit status 2: RTNETLINK answers: No such process > > 2013-07-08 21:36:50.786+0000: 19537: error : virCommandWait:2287 : > internal error Child process (ip link del veth4) unexpected exit > status 1: Cannot find device "veth4" > > On the host, /dev/net/tun exists as character device: > > root@main:~# ls -la /dev/net/tun > crw-rw-rwT 1 root root 10, 200 Jul 8 23:45 /dev/net/tun > > What am I doing wrong ...? >You are right, it should be char device. libvirt lxc should create "net" directory atomically for the tun device.
Gao feng
2013-Jul-09 03:20 UTC
[libvirt-users] [PATCH 1/2] LXC: hostdev: introduce lxcContainerSetupHostdevCapsMakePath
This helper function is used to create parent directroy for the hostdev which will be added to the container. if the parent directory of this hostdev doesn't exist, the mknod of the hostdev will fail. Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 17 +++++++++++++++++ src/lxc/lxc_container.h | 2 ++ 2 files changed, 19 insertions(+) diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index c8420db..b954107 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -1544,6 +1544,23 @@ cleanup: } +int lxcContainerSetupHostdevCapsMakePath(char *dev) +{ + int ret = 0; + char *dir = NULL; + + if ((dir = strrchr(dev, '/'))) { + *dir = '\0'; + if ((virFileMakePath(dev) < 0) && (errno != EEXIST)) + ret = -1; + + *dir = '/'; + } + + return ret; +} + + static int lxcContainerSetupHostdevCapsStorage(virDomainDefPtr vmDef ATTRIBUTE_UNUSED, virDomainHostdevDefPtr def ATTRIBUTE_UNUSED, virSecurityManagerPtr securityDriver ATTRIBUTE_UNUSED) diff --git a/src/lxc/lxc_container.h b/src/lxc/lxc_container.h index ada72f7..f168703 100644 --- a/src/lxc/lxc_container.h +++ b/src/lxc/lxc_container.h @@ -63,6 +63,8 @@ int lxcContainerStart(virDomainDefPtr def, int lxcContainerAvailable(int features); +int lxcContainerSetupHostdevCapsMakePath(char *dev); + virArch lxcContainerGetAlt32bitArch(virArch arch); #endif /* LXC_CONTAINER_H */ -- 1.8.3.1
Gao feng
2013-Jul-09 03:20 UTC
[libvirt-users] [PATCH 2/2] LXC: hostdev: parent directroy for hostdev atomically
Create parent directroy for hostdev atomically when we start a lxc domain or attach a hostdev to a lxc domain. Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> --- src/lxc/lxc_container.c | 42 ++++++++++++++++++++++++++++-------------- src/lxc/lxc_driver.c | 14 ++++++++++++++ 2 files changed, 42 insertions(+), 14 deletions(-) diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c index b954107..a204789 100644 --- a/src/lxc/lxc_container.c +++ b/src/lxc/lxc_container.c @@ -1569,14 +1569,15 @@ static int lxcContainerSetupHostdevCapsStorage(virDomainDefPtr vmDef ATTRIBUTE_U int ret = -1; struct stat sb; mode_t mode; + char *dev = def->source.caps.u.storage.block; - if (def->source.caps.u.storage.block == NULL) { + if (dev == NULL) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("Missing storage host block path")); goto cleanup; } - if (virAsprintf(&src, "/.oldroot/%s", def->source.caps.u.storage.block) < 0) { + if (virAsprintf(&src, "/.oldroot/%s", dev) < 0) { virReportOOMError(); goto cleanup; } @@ -1591,19 +1592,25 @@ static int lxcContainerSetupHostdevCapsStorage(virDomainDefPtr vmDef ATTRIBUTE_U if (!S_ISBLK(sb.st_mode)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Storage source %s must be a block device"), - def->source.caps.u.storage.block); + dev); + goto cleanup; + } + + if (lxcContainerSetupHostdevCapsMakePath(dev) < 0) { + virReportError(errno, + _("Failed to create directory for device %s"), + dev); goto cleanup; } mode = 0700 | S_IFBLK; - VIR_DEBUG("Creating dev %s (%d,%d)", - def->source.caps.u.storage.block, + VIR_DEBUG("Creating dev %s (%d,%d)", dev, major(sb.st_rdev), minor(sb.st_rdev)); - if (mknod(def->source.caps.u.storage.block, mode, sb.st_rdev) < 0) { + if (mknod(dev, mode, sb.st_rdev) < 0) { virReportSystemError(errno, _("Unable to create device %s"), - def->source.caps.u.storage.block); + dev); goto cleanup; } @@ -1626,14 +1633,15 @@ static int lxcContainerSetupHostdevCapsMisc(virDomainDefPtr vmDef ATTRIBUTE_UNUS int ret = -1; struct stat sb; mode_t mode; + char *dev = def->source.caps.u.misc.chardev; - if (def->source.caps.u.misc.chardev == NULL) { + if (dev == NULL) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", _("Missing storage host block path")); goto cleanup; } - if (virAsprintf(&src, "/.oldroot/%s", def->source.caps.u.misc.chardev) < 0) { + if (virAsprintf(&src, "/.oldroot/%s", dev) < 0) { virReportOOMError(); goto cleanup; } @@ -1648,19 +1656,25 @@ static int lxcContainerSetupHostdevCapsMisc(virDomainDefPtr vmDef ATTRIBUTE_UNUS if (!S_ISCHR(sb.st_mode)) { virReportError(VIR_ERR_CONFIG_UNSUPPORTED, _("Storage source %s must be a character device"), - def->source.caps.u.misc.chardev); + dev); + goto cleanup; + } + + if (lxcContainerSetupHostdevCapsMakePath(dev) < 0) { + virReportError(errno, + _("Failed to create directory for device %s"), + dev); goto cleanup; } mode = 0700 | S_IFCHR; - VIR_DEBUG("Creating dev %s (%d,%d)", - def->source.caps.u.misc.chardev, + VIR_DEBUG("Creating dev %s (%d,%d)", dev, major(sb.st_rdev), minor(sb.st_rdev)); - if (mknod(def->source.caps.u.misc.chardev, mode, sb.st_rdev) < 0) { + if (mknod(dev, mode, sb.st_rdev) < 0) { virReportSystemError(errno, _("Unable to create device %s"), - def->source.caps.u.misc.chardev); + dev); goto cleanup; } diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c index 1a6d086..2661c1b 100644 --- a/src/lxc/lxc_driver.c +++ b/src/lxc/lxc_driver.c @@ -3616,6 +3616,13 @@ lxcDomainAttachDeviceHostdevStorageLive(virLXCDriverPtr driver, goto cleanup; } + if (lxcContainerSetupHostdevCapsMakePath(dst) < 0) { + virReportSystemError(errno, + _("Unable to create directroy for device %s"), + dst); + goto cleanup; + } + mode = 0700 | S_IFBLK; VIR_DEBUG("Creating dev %s (%d,%d)", @@ -3720,6 +3727,13 @@ lxcDomainAttachDeviceHostdevMiscLive(virLXCDriverPtr driver, goto cleanup; } + if (lxcContainerSetupHostdevCapsMakePath(dst) < 0) { + virReportSystemError(errno, + _("Unable to create directroy for device %s"), + dst); + goto cleanup; + } + mode = 0700 | S_IFCHR; VIR_DEBUG("Creating dev %s (%d,%d)", -- 1.8.3.1
Daniel P. Berrange
2013-Jul-09 10:19 UTC
Re: [libvirt-users] [PATCH 1/2] LXC: hostdev: introduce lxcContainerSetupHostdevCapsMakePath
On Tue, Jul 09, 2013 at 11:20:35AM +0800, Gao feng wrote:> This helper function is used to create parent directroy for > the hostdev which will be added to the container. if the > parent directory of this hostdev doesn't exist, the mknod of > the hostdev will fail. > > Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com> > --- > src/lxc/lxc_container.c | 17 +++++++++++++++++ > src/lxc/lxc_container.h | 2 ++ > 2 files changed, 19 insertions(+) > > diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c > index c8420db..b954107 100644 > --- a/src/lxc/lxc_container.c > +++ b/src/lxc/lxc_container.c > @@ -1544,6 +1544,23 @@ cleanup: > } > > > +int lxcContainerSetupHostdevCapsMakePath(char *dev) > +{ > + int ret = 0; > + char *dir = NULL; > + > + if ((dir = strrchr(dev, '/'))) { > + *dir = '\0';Modifying arguments that are passed into a function is bad practice, so I'm changing this to strdup the arg thus: +int lxcContainerSetupHostdevCapsMakePath(const char *dev) +{ + int ret = -1; + char *dir, *tmp; + + if (VIR_STRDUP(dir, dev) < 0) + return -1; + + if ((tmp = strrchr(dir, '/'))) { + *tmp = '\0'; + if (virFileMakePath(dir) < 0) { + virReportSystemError(errno, + _("Failed to create directory for '%s' dev '%s'"), + dir, dev); + goto cleanup; + } + } + + ret = 0; + +cleanup: + VIR_FREE(dir); + return ret; +} Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|