thr3ads.net - libvirt users - [libvirt-users] [PATCH 2/2] LXC: hostdev: parent directroy for hostdev atomically [Jul 2013]

If this information is useful, please help other people find it:
Share via:

Thomas Karcher

2013-Jul-08 21:51 UTC

Re: [libvirt-users] Permission problem with /dev/net/tun

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Daniel,

On 07/08/2013 11:41 AM, Daniel P. Berrange wrote:>> the symptom my libvirt LXC container suffers from is: 
>> root@depot:/dev/net# ls -la total 0 drwxr-xr-x 2 root root  40
>> Jun 29 16:26 . drwxr-xr-x 5 root root 480 Jun 29 16:26 .. 
>> root@depot:/dev/net# mknod tun c 10 200 mknod: `tun': Operation
>> not permitted
> Allowing the container direct access to the hosts' /dev would be a
> security flaw, so libvirt sets up a private /dev for the
> container. Allowing the container to use mknod would also be
> insecure, so we blocking mknod using both cgroups device ACL, and
> also droping the CAP_MKNOD capability. 
> http://libvirt.org/drvlxc.html#devnodes
Good to know.
> Any device that the container is authorized to access per the XML 
> configuration, will be pre-created in the container's /dev. To 
> explicitly allow /dev/net/tun you need to tell libvirt about it. 
> http://libvirt.org/formatdomain.html#elementsHostDevCaps
Thanks!

I extended the 'devices' section as follows:

    <hostdev mode='capabilities' type='misc'>
      <source>
        <char>/dev/net/tun</char>
      </source>
    </hostdev>

... because even though /dev/net/tun is used for networking, it
appears as a character device. (Btw: The documentation says in the
hostdev section: ''For block/character device passthrough mode is
always "capabilities" and type is "block" for a block
device, "char"
for a character device and "net" for a host network
interface.'' When
I specify type='char', I get an error from virsh.)

With this XML, I can define the container. But upon start, I get the
following error message:

Fehler: internal error guest failed to start: PATH=/bin:/sbin
TERM=linux container=lxc-libvirt
container_uuid=f3602503-9603-24aa-7dd8-fccc830a802b
LIBVIRT_LXC_UUID=f3602503-9603-24aa-7dd8-fccc830a802b
LIBVIRT_LXC_NAME=depot /sbin/init
2013-07-08 21:36:50.735+0000: 1: info : libvirt version: 1.0.2
2013-07-08 21:36:50.735+0000: 1: error :
lxcContainerSetupHostdevCapsMisc:1490 : Unable to create device
/dev/net/tun: No such file or directory
2013-07-08 21:36:50.744+0000: 19537: info : libvirt version: 1.0.2
2013-07-08 21:36:50.744+0000: 19537: error : virCommandWait:2287 :
internal error Child process (ip link set veth6 netns 19538)
unexpected exit status 2: RTNETLINK answers: No such process

2013-07-08 21:36:50.786+0000: 19537: error : virCommandWait:2287 :
internal error Child process (ip link del veth4) unexpected exit
status 1: Cannot find device "veth4"

On the host, /dev/net/tun exists as character device:

root@main:~# ls -la /dev/net/tun
crw-rw-rwT 1 root root 10, 200 Jul  8 23:45 /dev/net/tun

What am I doing wrong ...?


Thanks
Thomas

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlHbNGwACgkQiMyIQtYO79zOwgCdFVzn0JopHK+ZY2ZshgZnuz6L
Yx8An3BL/2sfLTFSs39yNkB0FXzq9K/4
=xRtO
-----END PGP SIGNATURE-----

Gao feng

2013-Jul-09 02:05 UTC

head link

Re: [libvirt-users] Permission problem with /dev/net/tun

On 07/09/2013 05:51 AM, Thomas Karcher wrote:> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Daniel,
> 
> On 07/08/2013 11:41 AM, Daniel P. Berrange wrote:
>>> the symptom my libvirt LXC container suffers from is: 
>>> root@depot:/dev/net# ls -la total 0 drwxr-xr-x 2 root root  40
>>> Jun 29 16:26 . drwxr-xr-x 5 root root 480 Jun 29 16:26 .. 
>>> root@depot:/dev/net# mknod tun c 10 200 mknod: `tun': Operation
>>> not permitted
>> Allowing the container direct access to the hosts' /dev would be a
>> security flaw, so libvirt sets up a private /dev for the
>> container. Allowing the container to use mknod would also be
>> insecure, so we blocking mknod using both cgroups device ACL, and
>> also droping the CAP_MKNOD capability. 
>> http://libvirt.org/drvlxc.html#devnodes
> 
> Good to know.
> 
>> Any device that the container is authorized to access per the XML 
>> configuration, will be pre-created in the container's /dev. To 
>> explicitly allow /dev/net/tun you need to tell libvirt about it. 
>> http://libvirt.org/formatdomain.html#elementsHostDevCaps
> 
> Thanks!
> 
> I extended the 'devices' section as follows:
> 
>     <hostdev mode='capabilities' type='misc'>
>       <source>
>         <char>/dev/net/tun</char>
>       </source>
>     </hostdev>
> 
> ... because even though /dev/net/tun is used for networking, it
> appears as a character device. (Btw: The documentation says in the
> hostdev section: ''For block/character device passthrough mode is
> always "capabilities" and type is "block" for a block
device, "char"
> for a character device and "net" for a host network
interface.'' When
> I specify type='char', I get an error from virsh.)
> 
> With this XML, I can define the container. But upon start, I get the
> following error message:
> 
> Fehler: internal error guest failed to start: PATH=/bin:/sbin
> TERM=linux container=lxc-libvirt
> container_uuid=f3602503-9603-24aa-7dd8-fccc830a802b
> LIBVIRT_LXC_UUID=f3602503-9603-24aa-7dd8-fccc830a802b
> LIBVIRT_LXC_NAME=depot /sbin/init
> 2013-07-08 21:36:50.735+0000: 1: info : libvirt version: 1.0.2
> 2013-07-08 21:36:50.735+0000: 1: error :
> lxcContainerSetupHostdevCapsMisc:1490 : Unable to create device
> /dev/net/tun: No such file or directory
> 2013-07-08 21:36:50.744+0000: 19537: info : libvirt version: 1.0.2
> 2013-07-08 21:36:50.744+0000: 19537: error : virCommandWait:2287 :
> internal error Child process (ip link set veth6 netns 19538)
> unexpected exit status 2: RTNETLINK answers: No such process
> 
> 2013-07-08 21:36:50.786+0000: 19537: error : virCommandWait:2287 :
> internal error Child process (ip link del veth4) unexpected exit
> status 1: Cannot find device "veth4"
> 
> On the host, /dev/net/tun exists as character device:
> 
> root@main:~# ls -la /dev/net/tun
> crw-rw-rwT 1 root root 10, 200 Jul  8 23:45 /dev/net/tun
> 
> What am I doing wrong ...?
> 
You are right, it should be char device.

libvirt lxc should create "net" directory atomically for the tun
device.

Gao feng

2013-Jul-09 03:20 UTC

head link

[libvirt-users] [PATCH 1/2] LXC: hostdev: introduce lxcContainerSetupHostdevCapsMakePath

This helper function is used to create parent directroy for
the hostdev which will be added to the container. if the
parent directory of this hostdev doesn't exist, the mknod of
the hostdev will fail.

Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com>
---
 src/lxc/lxc_container.c | 17 +++++++++++++++++
 src/lxc/lxc_container.h |  2 ++
 2 files changed, 19 insertions(+)

diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c
index c8420db..b954107 100644
--- a/src/lxc/lxc_container.c
+++ b/src/lxc/lxc_container.c
@@ -1544,6 +1544,23 @@ cleanup:
 }
 
 
+int lxcContainerSetupHostdevCapsMakePath(char *dev)
+{
+    int ret = 0;
+    char *dir = NULL;
+
+    if ((dir = strrchr(dev, '/'))) {
+        *dir = '\0';
+        if ((virFileMakePath(dev) < 0) && (errno != EEXIST))
+            ret = -1;
+
+        *dir = '/';
+    }
+
+    return ret;
+}
+
+
 static int lxcContainerSetupHostdevCapsStorage(virDomainDefPtr vmDef
ATTRIBUTE_UNUSED,
                                                virDomainHostdevDefPtr def
ATTRIBUTE_UNUSED,
                                                virSecurityManagerPtr
securityDriver ATTRIBUTE_UNUSED)
diff --git a/src/lxc/lxc_container.h b/src/lxc/lxc_container.h
index ada72f7..f168703 100644
--- a/src/lxc/lxc_container.h
+++ b/src/lxc/lxc_container.h
@@ -63,6 +63,8 @@ int lxcContainerStart(virDomainDefPtr def,
 
 int lxcContainerAvailable(int features);
 
+int lxcContainerSetupHostdevCapsMakePath(char *dev);
+
 virArch lxcContainerGetAlt32bitArch(virArch arch);
 
 #endif /* LXC_CONTAINER_H */
-- 
1.8.3.1

Gao feng

2013-Jul-09 03:20 UTC

head link

[libvirt-users] [PATCH 2/2] LXC: hostdev: parent directroy for hostdev atomically

Create parent directroy for hostdev atomically when we
start a lxc domain or attach a hostdev to a lxc domain.

Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com>
---
 src/lxc/lxc_container.c | 42 ++++++++++++++++++++++++++++--------------
 src/lxc/lxc_driver.c    | 14 ++++++++++++++
 2 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c
index b954107..a204789 100644
--- a/src/lxc/lxc_container.c
+++ b/src/lxc/lxc_container.c
@@ -1569,14 +1569,15 @@ static int
lxcContainerSetupHostdevCapsStorage(virDomainDefPtr vmDef ATTRIBUTE_U
     int ret = -1;
     struct stat sb;
     mode_t mode;
+    char *dev = def->source.caps.u.storage.block;
 
-    if (def->source.caps.u.storage.block == NULL) {
+    if (dev == NULL) {
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
                        _("Missing storage host block path"));
         goto cleanup;
     }
 
-    if (virAsprintf(&src, "/.oldroot/%s",
def->source.caps.u.storage.block) < 0) {
+    if (virAsprintf(&src, "/.oldroot/%s", dev) < 0) {
         virReportOOMError();
         goto cleanup;
     }
@@ -1591,19 +1592,25 @@ static int
lxcContainerSetupHostdevCapsStorage(virDomainDefPtr vmDef ATTRIBUTE_U
     if (!S_ISBLK(sb.st_mode)) {
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
                        _("Storage source %s must be a block device"),
-                       def->source.caps.u.storage.block);
+                       dev);
+        goto cleanup;
+    }
+
+    if (lxcContainerSetupHostdevCapsMakePath(dev) < 0) {
+        virReportError(errno,
+                       _("Failed to create directory for device %s"),
+                       dev);
         goto cleanup;
     }
 
     mode = 0700 | S_IFBLK;
 
-    VIR_DEBUG("Creating dev %s (%d,%d)",
-              def->source.caps.u.storage.block,
+    VIR_DEBUG("Creating dev %s (%d,%d)", dev,
               major(sb.st_rdev), minor(sb.st_rdev));
-    if (mknod(def->source.caps.u.storage.block, mode, sb.st_rdev) < 0) {
+    if (mknod(dev, mode, sb.st_rdev) < 0) {
         virReportSystemError(errno,
                              _("Unable to create device %s"),
-                             def->source.caps.u.storage.block);
+                             dev);
         goto cleanup;
     }
 
@@ -1626,14 +1633,15 @@ static int
lxcContainerSetupHostdevCapsMisc(virDomainDefPtr vmDef ATTRIBUTE_UNUS
     int ret = -1;
     struct stat sb;
     mode_t mode;
+    char *dev = def->source.caps.u.misc.chardev;
 
-    if (def->source.caps.u.misc.chardev == NULL) {
+    if (dev == NULL) {
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
                        _("Missing storage host block path"));
         goto cleanup;
     }
 
-    if (virAsprintf(&src, "/.oldroot/%s",
def->source.caps.u.misc.chardev) < 0) {
+    if (virAsprintf(&src, "/.oldroot/%s", dev) < 0) {
         virReportOOMError();
         goto cleanup;
     }
@@ -1648,19 +1656,25 @@ static int
lxcContainerSetupHostdevCapsMisc(virDomainDefPtr vmDef ATTRIBUTE_UNUS
     if (!S_ISCHR(sb.st_mode)) {
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
                        _("Storage source %s must be a character
device"),
-                       def->source.caps.u.misc.chardev);
+                       dev);
+        goto cleanup;
+    }
+
+    if (lxcContainerSetupHostdevCapsMakePath(dev) < 0) {
+        virReportError(errno,
+                       _("Failed to create directory for device %s"),
+                       dev);
         goto cleanup;
     }
 
     mode = 0700 | S_IFCHR;
 
-    VIR_DEBUG("Creating dev %s (%d,%d)",
-              def->source.caps.u.misc.chardev,
+    VIR_DEBUG("Creating dev %s (%d,%d)", dev,
               major(sb.st_rdev), minor(sb.st_rdev));
-    if (mknod(def->source.caps.u.misc.chardev, mode, sb.st_rdev) < 0) {
+    if (mknod(dev, mode, sb.st_rdev) < 0) {
         virReportSystemError(errno,
                              _("Unable to create device %s"),
-                             def->source.caps.u.misc.chardev);
+                             dev);
         goto cleanup;
     }
 
diff --git a/src/lxc/lxc_driver.c b/src/lxc/lxc_driver.c
index 1a6d086..2661c1b 100644
--- a/src/lxc/lxc_driver.c
+++ b/src/lxc/lxc_driver.c
@@ -3616,6 +3616,13 @@ lxcDomainAttachDeviceHostdevStorageLive(virLXCDriverPtr
driver,
         goto cleanup;
     }
 
+    if (lxcContainerSetupHostdevCapsMakePath(dst) < 0) {
+        virReportSystemError(errno,
+                             _("Unable to create directroy for device
%s"),
+                             dst);
+        goto cleanup;
+    }
+
     mode = 0700 | S_IFBLK;
 
     VIR_DEBUG("Creating dev %s (%d,%d)",
@@ -3720,6 +3727,13 @@ lxcDomainAttachDeviceHostdevMiscLive(virLXCDriverPtr
driver,
         goto cleanup;
     }
 
+    if (lxcContainerSetupHostdevCapsMakePath(dst) < 0) {
+        virReportSystemError(errno,
+                             _("Unable to create directroy for device
%s"),
+                             dst);
+        goto cleanup;
+    }
+
     mode = 0700 | S_IFCHR;
 
     VIR_DEBUG("Creating dev %s (%d,%d)",
-- 
1.8.3.1

Daniel P. Berrange

2013-Jul-09 10:19 UTC

head link

Re: [libvirt-users] [PATCH 1/2] LXC: hostdev: introduce lxcContainerSetupHostdevCapsMakePath

On Tue, Jul 09, 2013 at 11:20:35AM +0800, Gao feng
wrote:> This helper function is used to create parent directroy for
> the hostdev which will be added to the container. if the
> parent directory of this hostdev doesn't exist, the mknod of
> the hostdev will fail.
> 
> Signed-off-by: Gao feng <gaofeng@cn.fujitsu.com>
> ---
>  src/lxc/lxc_container.c | 17 +++++++++++++++++
>  src/lxc/lxc_container.h |  2 ++
>  2 files changed, 19 insertions(+)
> 
> diff --git a/src/lxc/lxc_container.c b/src/lxc/lxc_container.c
> index c8420db..b954107 100644
> --- a/src/lxc/lxc_container.c
> +++ b/src/lxc/lxc_container.c
> @@ -1544,6 +1544,23 @@ cleanup:
>  }
>  
>  
> +int lxcContainerSetupHostdevCapsMakePath(char *dev)
> +{
> +    int ret = 0;
> +    char *dir = NULL;
> +
> +    if ((dir = strrchr(dev, '/'))) {
> +        *dir = '\0';
Modifying arguments that are passed into a function is bad
practice, so I'm changing this to strdup the arg thus:


+int lxcContainerSetupHostdevCapsMakePath(const char *dev)
+{
+    int ret = -1;
+    char *dir, *tmp;
+
+    if (VIR_STRDUP(dir, dev) < 0)
+        return -1;
+
+    if ((tmp = strrchr(dir, '/'))) {
+        *tmp = '\0';
+        if (virFileMakePath(dir) < 0) {
+            virReportSystemError(errno,
+                                 _("Failed to create directory for
'%s' dev '%s'"),
+                                 dir, dev);
+            goto cleanup;
+        }
+    }
+
+    ret = 0;
+
+cleanup:
+    VIR_FREE(dir);
+    return ret;
+}


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Possibly Parallel Threads

Search for more apparently analagous threads

libvirt users - Jul 2013 - [PATCH 2/2] LXC: hostdev: parent directroy for hostdev atomically

Re: [libvirt-users] Permission problem with /dev/net/tun

Re: [libvirt-users] Permission problem with /dev/net/tun

[libvirt-users] [PATCH 1/2] LXC: hostdev: introduce lxcContainerSetupHostdevCapsMakePath

[libvirt-users] [PATCH 2/2] LXC: hostdev: parent directroy for hostdev atomically

Re: [libvirt-users] [PATCH 1/2] LXC: hostdev: introduce lxcContainerSetupHostdevCapsMakePath

Possibly Parallel Threads