thr3ads.net - libvirt users - Re: [libvirt-users] LXC + USB passthrough = Operation not permitted [Apr 2014]

If this information is useful, please help other people find it:
Share via:

Filip Maj

2014-Apr-16 22:25 UTC

Re: [libvirt-users] LXC + USB passthrough = Operation not permitted

Yeah, AppArmor is enabled, but I put everything (that I could find) into
complain mode:

$ sudo apparmor_status
apparmor module is loaded.
12 profiles are loaded.
3 profiles are in enforce mode.
   lxc-container-default
   lxc-container-default-with-mounting
   lxc-container-default-with-nesting
9 profiles are in complain mode.
   /sbin/dhclient
   /usr/bin/lxc-start
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/libvirt/virt-aa-helper
   /usr/sbin/libvirtd
   /usr/sbin/ntpd
   /usr/sbin/rsyslogd
   /usr/sbin/tcpdump
3 processes have profiles defined.
0 processes are in enforce mode.
2 processes are in complain mode.
   /usr/sbin/libvirtd (30419)
   /usr/sbin/ntpd (3418)
1 processes are unconfined but have a profile defined.
   /usr/sbin/rsyslogd (626)

And still get issues. From libvirtd.log:

2014-04-16 22:19:10.855+0000: 30419: info : libvirt version: 1.2.2
2014-04-16 22:19:10.855+0000: 30419: error : virNetSocketReadWire:1446 :
Cannot recv data: Connection reset by peer
2014-04-16 22:19:10.940+0000: 30420: error : virLXCProcessStart:1299 :
internal error: guest failed to start: Unable to create device
//var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not
permitted

2014-04-16 22:19:10.964+0000: 30420: warning :
virLXCDomainReAttachHostUsbDevices:388 : Unable to find device 000.000 in
list of active USB devices

Thanks in advance for any help, Daniel!

Cheers,
Fil


On Tue, Apr 15, 2014 at 1:33 AM, Daniel P. Berrange
<berrange@redhat.com>wrote:
> On Fri, Apr 11, 2014 at 05:32:28PM -0700, Filip Maj wrote:
> > Hi!
> >
> > First post, kind of a noobie. I've been working with LXC and
libvirt for
> a
> > few months now. Trying to do some interesting things with containers
and
> > Android devices :D
> > Here's my entire domain definition:
> >
> > <domain type='lxc'>
> >   <name>oshi32134</name>
> >   <uuid>xxxxx</uuid>
> >   <memory unit='KiB'>3145728</memory>
> >   <currentMemory
unit='KiB'>3145728</currentMemory>
> >   <vcpu placement='static'>1</vcpu>
> >   <resource>
> >     <partition>/machine</partition>
> >   </resource>
> >   <os>
> >     <type arch='i686'>exe</type>
> >     <init>/sbin/init</init>
> >   </os>
> >   <clock offset='utc'/>
> >   <on_poweroff>destroy</on_poweroff>
> >   <on_reboot>restart</on_reboot>
> >   <on_crash>destroy</on_crash>
> >   <devices>
> >     <emulator>/usr/lib/libvirt/libvirt_lxc</emulator>
> >     <filesystem type='mount'
accessmode='passthrough'>
> >       <source dir='/some/valid/filesystem/location'/>
> >       <target dir='/'/>
> >     </filesystem>
> >     <filesystem type='mount'
accessmode='passthrough'>
> >       <source dir='/another/valid/filesystem/location'/>
> >       <target dir='/mnt/android'/>
> >     </filesystem>
> >     <interface type='bridge'>
> >       <mac address='xx:xx:xx:xx:xx:xx'/>
> >       <source bridge='br1'/>
> >     </interface>
> >     <console type='pty'>
> >       <target type='lxc' port='0'/>
> >     </console>
> >     <hostdev mode='capabilities' type='misc'>
> >       <source>
> >         <char>/dev/kvm</char>
> >       </source>
> >     </hostdev>
> >     <hostdev mode='subsystem' type='usb'
managed='yes'>
> >       <source>
> >         <vendor id='0x04e8'/>
> >         <product id='0x6860'/>
> >   </source>
> >     </hostdev>
> >   </devices>
> > </domain>
>
> Your config looks fine here.
>
> >
> > Everything worked fine until I added the USB <hostdev> element.
I'm
> > essentially trying to get access to a physical Android device
connected
> to
> > the host from inside a container. When I go to start the container, I
get
> > an error about Operation not permitted. Here's the relevant bits
from
> > /var/log/libvirt/lxc/machine.log:
> >
> > 2014-04-11 22:46:40.491+0000: starting up
> > PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
> > LIBVIRT_DEBUG=3 LIBVIRT_LOG_OUTPUTS=3:stderr
/usr/lib/libvirt/libvirt_lxc
> > --name oshi32134 --console 24 --security=none --handshake 27
--background
> > --veth vnet1
> > 2014-04-11 22:46:40.597+0000: 685: info : libvirt version: 1.2.2
> > 2014-04-11 22:46:40.597+0000: 685: error :
> > virLXCControllerSetupHostdevSubsysUSB:1390 : Unable to create device
> > //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not
> > permitted
> > Unable to create device
> > //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not
> > permitted
>
> Do you have AppArmour enabled on the machine. That seems like the
> most likely thing that would result in libvirt getting that permission
> error.
>
> Regards,
> Daniel
> --
> |: http://berrange.com      -o-   
http://www.flickr.com/photos/dberrange/:|
> |: http://libvirt.org              -o-            
http://virt-manager.org:|
> |: http://autobuild.org       -o-        
http://search.cpan.org/~danberr/:|
> |: http://entangle-photo.org       -o-      
http://live.gnome.org/gtk-vnc:|
>

Filip Maj

2014-Apr-16 22:58 UTC

head link

Re: [libvirt-users] LXC + USB passthrough = Operation not permitted

To follow up on this a little bit, tail'ing kern.log while trying to get
our little container up doesn't yield anything with apparmor complaining,
so, unless I'm looking in the wrong spots for apparmor logs (which I
don't
think so, as I see other apparmor-related log entries in kern.log), I am
not entirely sure this is an apparmor issue at this point.


On Wed, Apr 16, 2014 at 3:25 PM, Filip Maj <fil@saucelabs.com> wrote:
> Yeah, AppArmor is enabled, but I put everything (that I could find) into
> complain mode:
>
> $ sudo apparmor_status
> apparmor module is loaded.
> 12 profiles are loaded.
> 3 profiles are in enforce mode.
>    lxc-container-default
>    lxc-container-default-with-mounting
>    lxc-container-default-with-nesting
> 9 profiles are in complain mode.
>    /sbin/dhclient
>    /usr/bin/lxc-start
>    /usr/lib/NetworkManager/nm-dhcp-client.action
>    /usr/lib/connman/scripts/dhclient-script
>    /usr/lib/libvirt/virt-aa-helper
>    /usr/sbin/libvirtd
>    /usr/sbin/ntpd
>    /usr/sbin/rsyslogd
>    /usr/sbin/tcpdump
> 3 processes have profiles defined.
> 0 processes are in enforce mode.
> 2 processes are in complain mode.
>    /usr/sbin/libvirtd (30419)
>    /usr/sbin/ntpd (3418)
> 1 processes are unconfined but have a profile defined.
>    /usr/sbin/rsyslogd (626)
>
> And still get issues. From libvirtd.log:
>
> 2014-04-16 22:19:10.855+0000: 30419: info : libvirt version: 1.2.2
> 2014-04-16 22:19:10.855+0000: 30419: error : virNetSocketReadWire:1446 :
> Cannot recv data: Connection reset by peer
> 2014-04-16 22:19:10.940+0000: 30420: error : virLXCProcessStart:1299 :
> internal error: guest failed to start: Unable to create device
> //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not
> permitted
>
> 2014-04-16 22:19:10.964+0000: 30420: warning :
> virLXCDomainReAttachHostUsbDevices:388 : Unable to find device 000.000 in
> list of active USB devices
>
> Thanks in advance for any help, Daniel!
>
> Cheers,
> Fil
>
>
> On Tue, Apr 15, 2014 at 1:33 AM, Daniel P. Berrange
<berrange@redhat.com>wrote:
>
>> On Fri, Apr 11, 2014 at 05:32:28PM -0700, Filip Maj wrote:
>> > Hi!
>> >
>> > First post, kind of a noobie. I've been working with LXC and
libvirt
>> for a
>> > few months now. Trying to do some interesting things with
containers and
>> > Android devices :D
>> > Here's my entire domain definition:
>> >
>> > <domain type='lxc'>
>> >   <name>oshi32134</name>
>> >   <uuid>xxxxx</uuid>
>> >   <memory unit='KiB'>3145728</memory>
>> >   <currentMemory
unit='KiB'>3145728</currentMemory>
>> >   <vcpu placement='static'>1</vcpu>
>> >   <resource>
>> >     <partition>/machine</partition>
>> >   </resource>
>> >   <os>
>> >     <type arch='i686'>exe</type>
>> >     <init>/sbin/init</init>
>> >   </os>
>> >   <clock offset='utc'/>
>> >   <on_poweroff>destroy</on_poweroff>
>> >   <on_reboot>restart</on_reboot>
>> >   <on_crash>destroy</on_crash>
>> >   <devices>
>> >     <emulator>/usr/lib/libvirt/libvirt_lxc</emulator>
>> >     <filesystem type='mount'
accessmode='passthrough'>
>> >       <source
dir='/some/valid/filesystem/location'/>
>> >       <target dir='/'/>
>> >     </filesystem>
>> >     <filesystem type='mount'
accessmode='passthrough'>
>> >       <source
dir='/another/valid/filesystem/location'/>
>> >       <target dir='/mnt/android'/>
>> >     </filesystem>
>> >     <interface type='bridge'>
>> >       <mac address='xx:xx:xx:xx:xx:xx'/>
>> >       <source bridge='br1'/>
>> >     </interface>
>> >     <console type='pty'>
>> >       <target type='lxc' port='0'/>
>> >     </console>
>> >     <hostdev mode='capabilities'
type='misc'>
>> >       <source>
>> >         <char>/dev/kvm</char>
>> >       </source>
>> >     </hostdev>
>> >     <hostdev mode='subsystem' type='usb'
managed='yes'>
>> >       <source>
>> >         <vendor id='0x04e8'/>
>> >         <product id='0x6860'/>
>> >   </source>
>> >     </hostdev>
>> >   </devices>
>> > </domain>
>>
>> Your config looks fine here.
>>
>> >
>> > Everything worked fine until I added the USB <hostdev>
element. I'm
>> > essentially trying to get access to a physical Android device
connected
>> to
>> > the host from inside a container. When I go to start the
container, I
>> get
>> > an error about Operation not permitted. Here's the relevant
bits from
>> > /var/log/libvirt/lxc/machine.log:
>> >
>> > 2014-04-11 22:46:40.491+0000: starting up
>> > PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
>> > LIBVIRT_DEBUG=3 LIBVIRT_LOG_OUTPUTS=3:stderr
>> /usr/lib/libvirt/libvirt_lxc
>> > --name oshi32134 --console 24 --security=none --handshake 27
>> --background
>> > --veth vnet1
>> > 2014-04-11 22:46:40.597+0000: 685: info : libvirt version: 1.2.2
>> > 2014-04-11 22:46:40.597+0000: 685: error :
>> > virLXCControllerSetupHostdevSubsysUSB:1390 : Unable to create
device
>> > //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation
not
>> > permitted
>> > Unable to create device
>> > //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation
not
>> > permitted
>>
>> Do you have AppArmour enabled on the machine. That seems like the
>> most likely thing that would result in libvirt getting that permission
>> error.
>>
>> Regards,
>> Daniel
>> --
>> |: http://berrange.com      -o-
>> http://www.flickr.com/photos/dberrange/ :|
>> |: http://libvirt.org              -o-
>> http://virt-manager.org :|
>> |: http://autobuild.org       -o-
>> http://search.cpan.org/~danberr/ :|
>> |: http://entangle-photo.org       -o-
>> http://live.gnome.org/gtk-vnc :|
>>
>
>

Filip Maj

2014-Apr-17 00:11 UTC

head link

Re: [libvirt-users] LXC + USB passthrough = Operation not permitted

Further followups!

We are correlating DEBUG-level output from libvirt with the libvirt 1.2.2
code to try to figure out what libvirt is doing under the hood.

Even though we have the log level set to 1 (info) in our libvirtd.conf, we
are not seeing the VIR_DEBUG() [1] statements being printed out. There are
tons of other presumably-debug lines of output showing up in our log. We
are sort of expecting to see the output from [1] in our logs somewhere,
given the type of usb mounting we are trying to do?

[1]
http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/lxc/lxc_controller.c;h=c05dfec6428cad927cd5751004a4f3afc67899de;hb=HEAD#l1396


On Wed, Apr 16, 2014 at 3:58 PM, Filip Maj <fil@saucelabs.com> wrote:
> To follow up on this a little bit, tail'ing kern.log while trying to
get
> our little container up doesn't yield anything with apparmor
complaining,
> so, unless I'm looking in the wrong spots for apparmor logs (which I
don't
> think so, as I see other apparmor-related log entries in kern.log), I am
> not entirely sure this is an apparmor issue at this point.
>
>
> On Wed, Apr 16, 2014 at 3:25 PM, Filip Maj <fil@saucelabs.com> wrote:
>
>> Yeah, AppArmor is enabled, but I put everything (that I could find)
into
>> complain mode:
>>
>> $ sudo apparmor_status
>> apparmor module is loaded.
>> 12 profiles are loaded.
>> 3 profiles are in enforce mode.
>>    lxc-container-default
>>    lxc-container-default-with-mounting
>>    lxc-container-default-with-nesting
>> 9 profiles are in complain mode.
>>    /sbin/dhclient
>>    /usr/bin/lxc-start
>>    /usr/lib/NetworkManager/nm-dhcp-client.action
>>    /usr/lib/connman/scripts/dhclient-script
>>    /usr/lib/libvirt/virt-aa-helper
>>    /usr/sbin/libvirtd
>>    /usr/sbin/ntpd
>>    /usr/sbin/rsyslogd
>>    /usr/sbin/tcpdump
>> 3 processes have profiles defined.
>> 0 processes are in enforce mode.
>> 2 processes are in complain mode.
>>    /usr/sbin/libvirtd (30419)
>>    /usr/sbin/ntpd (3418)
>> 1 processes are unconfined but have a profile defined.
>>    /usr/sbin/rsyslogd (626)
>>
>> And still get issues. From libvirtd.log:
>>
>> 2014-04-16 22:19:10.855+0000: 30419: info : libvirt version: 1.2.2
>> 2014-04-16 22:19:10.855+0000: 30419: error : virNetSocketReadWire:1446
:
>> Cannot recv data: Connection reset by peer
>> 2014-04-16 22:19:10.940+0000: 30420: error : virLXCProcessStart:1299 :
>> internal error: guest failed to start: Unable to create device
>> //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003: Operation not
>> permitted
>>
>> 2014-04-16 22:19:10.964+0000: 30420: warning :
>> virLXCDomainReAttachHostUsbDevices:388 : Unable to find device 000.000
in
>> list of active USB devices
>>
>> Thanks in advance for any help, Daniel!
>>
>> Cheers,
>> Fil
>>
>>
>> On Tue, Apr 15, 2014 at 1:33 AM, Daniel P. Berrange
<berrange@redhat.com>wrote:
>>
>>> On Fri, Apr 11, 2014 at 05:32:28PM -0700, Filip Maj wrote:
>>> > Hi!
>>> >
>>> > First post, kind of a noobie. I've been working with LXC
and libvirt
>>> for a
>>> > few months now. Trying to do some interesting things with
containers
>>> and
>>> > Android devices :D
>>> > Here's my entire domain definition:
>>> >
>>> > <domain type='lxc'>
>>> >   <name>oshi32134</name>
>>> >   <uuid>xxxxx</uuid>
>>> >   <memory unit='KiB'>3145728</memory>
>>> >   <currentMemory
unit='KiB'>3145728</currentMemory>
>>> >   <vcpu placement='static'>1</vcpu>
>>> >   <resource>
>>> >     <partition>/machine</partition>
>>> >   </resource>
>>> >   <os>
>>> >     <type arch='i686'>exe</type>
>>> >     <init>/sbin/init</init>
>>> >   </os>
>>> >   <clock offset='utc'/>
>>> >   <on_poweroff>destroy</on_poweroff>
>>> >   <on_reboot>restart</on_reboot>
>>> >   <on_crash>destroy</on_crash>
>>> >   <devices>
>>> >    
<emulator>/usr/lib/libvirt/libvirt_lxc</emulator>
>>> >     <filesystem type='mount'
accessmode='passthrough'>
>>> >       <source
dir='/some/valid/filesystem/location'/>
>>> >       <target dir='/'/>
>>> >     </filesystem>
>>> >     <filesystem type='mount'
accessmode='passthrough'>
>>> >       <source
dir='/another/valid/filesystem/location'/>
>>> >       <target dir='/mnt/android'/>
>>> >     </filesystem>
>>> >     <interface type='bridge'>
>>> >       <mac address='xx:xx:xx:xx:xx:xx'/>
>>> >       <source bridge='br1'/>
>>> >     </interface>
>>> >     <console type='pty'>
>>> >       <target type='lxc' port='0'/>
>>> >     </console>
>>> >     <hostdev mode='capabilities'
type='misc'>
>>> >       <source>
>>> >         <char>/dev/kvm</char>
>>> >       </source>
>>> >     </hostdev>
>>> >     <hostdev mode='subsystem' type='usb'
managed='yes'>
>>> >       <source>
>>> >         <vendor id='0x04e8'/>
>>> >         <product id='0x6860'/>
>>> >   </source>
>>> >     </hostdev>
>>> >   </devices>
>>> > </domain>
>>>
>>> Your config looks fine here.
>>>
>>> >
>>> > Everything worked fine until I added the USB <hostdev>
element. I'm
>>> > essentially trying to get access to a physical Android device
>>> connected to
>>> > the host from inside a container. When I go to start the
container, I
>>> get
>>> > an error about Operation not permitted. Here's the
relevant bits from
>>> > /var/log/libvirt/lxc/machine.log:
>>> >
>>> > 2014-04-11 22:46:40.491+0000: starting up
>>> >
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
>>> > LIBVIRT_DEBUG=3 LIBVIRT_LOG_OUTPUTS=3:stderr
>>> /usr/lib/libvirt/libvirt_lxc
>>> > --name oshi32134 --console 24 --security=none --handshake 27
>>> --background
>>> > --veth vnet1
>>> > 2014-04-11 22:46:40.597+0000: 685: info : libvirt version:
1.2.2
>>> > 2014-04-11 22:46:40.597+0000: 685: error :
>>> > virLXCControllerSetupHostdevSubsysUSB:1390 : Unable to create
device
>>> > //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003:
Operation not
>>> > permitted
>>> > Unable to create device
>>> > //var/run/libvirt/lxc/oshi32134.dev/bus/usb//002//003:
Operation not
>>> > permitted
>>>
>>> Do you have AppArmour enabled on the machine. That seems like the
>>> most likely thing that would result in libvirt getting that
permission
>>> error.
>>>
>>> Regards,
>>> Daniel
>>> --
>>> |: http://berrange.com      -o-
>>> http://www.flickr.com/photos/dberrange/ :|
>>> |: http://libvirt.org              -o-
>>> http://virt-manager.org :|
>>> |: http://autobuild.org       -o-
>>> http://search.cpan.org/~danberr/ :|
>>> |: http://entangle-photo.org       -o-
>>> http://live.gnome.org/gtk-vnc :|
>>>
>>
>>
>

Reasonably Related Threads

Search for more apparently analagous threads

libvirt users - Apr 2014 - Re: LXC + USB passthrough = Operation not permitted

Re: [libvirt-users] LXC + USB passthrough = Operation not permitted

Re: [libvirt-users] LXC + USB passthrough = Operation not permitted

Re: [libvirt-users] LXC + USB passthrough = Operation not permitted

Reasonably Related Threads