Re: [Xen-users] using USB Host Controller in domU

Andreas Seuss
Wed, 03 Aug 2005 15:23:11 -0000

Hi all,

thanks Marcus. Seems like I missed to make my domU-Kernel a privileged
guest.

CONFIG_XEN_PRIVILEGED_GUEST=y


Now it works ;-).

Marcus Brown wrote:

>Hi Andreas,
>Thanks for asking, I figure it is time to start organising some of my notes.
>This is what I''ve got so far, perhaps you''ll find
something in them?
>
>*****************************************************************************************************************
>Using PCI devices in a driver domain (a privileged domU?)
>===================================================================>The
examples below use the following domains.
>wench:        dom0 on P3/650 on Asus P2BVM, 512MB RAM
>"Production" Xen Server
>brutus:        dom0 on P2/350 on Asus P2B, 192MB ECC RAM
>"Development" Xen Server
>BluetoothProxy:    domU on wench (in use)
>XenFirewall:    domU on brutus (WIP)
>
>These notes are mostly to clean up my own documentation, and they
>_hopefully_ contain
>all the information I need to repeat the process.
>It is not necessarily HOWTO CREATE DRIVER DOMAINS, but how I got mine to
>work.
>
>
>1. Hide the device from the hypervisor (dom0)
>
>EXAMPLE A: brutus
>-----------------------------
>This machine has a hidden USB Host Controller, and two network cards,
>(the second) one being hidden from brutus.
>It has a custom initrd as the root is on LVM.
>brutus:~# lspci
>0000:00:00.0 Host bridge: Intel Corp. 440BX/ZX/DX - 82443BX/ZX/DX Host
>bridge (rev 03)
>0000:00:01.0 PCI bridge: Intel Corp. 440BX/ZX/DX - 82443BX/ZX/DX AGP
>bridge (rev 03)
>0000:00:04.0 ISA bridge: Intel Corp. 82371AB/EB/MB PIIX4 ISA (rev 02)
>0000:00:04.1 IDE interface: Intel Corp. 82371AB/EB/MB PIIX4 IDE (rev 01)
>0000:00:04.3 Bridge: Intel Corp. 82371AB/EB/MB PIIX4 ACPI (rev 02)
>0000:00:0b.0 Ethernet controller: 3Com Corporation 3c905B 100BaseTX
>[Cyclone] (rev 30)
>0000:01:00.0 VGA compatible controller: Silicon Integrated Systems [SiS]
>86C326 5598/6326 (rev 0b)
>
>brutus:~# tail /boot/grub/menu.lst
>title           Debian Xen
>root            (hd0,0)
>kernel          /xen-2.0-testing.gz dom0_mem=81920 root=/dev/ram0
>lvm2root=/dev/mapper/vg0-root physdev_dom0_hide=(00:0c.0)(00:04:2) ro
>console=tty0 noreboot
>module          /vmlinuz-2.6.11.12-xen0 root=/dev/mapper/vg0-root
>console=tty0 ro noreboot
>module          /initrd.img-2.6.11.12-xen0
>savedefault
>boot
>
>EXAMPLE B: wench
>------------------------------
>This machine has a hidden USB host controller
>wench:~# lspci
>0000:00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX
>Host bridge (rev 03)
>0000:00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX
>AGP bridge (rev 03)
>0000:00:06.0 Multimedia audio controller: ESS Technology ES1969 Solo-1
>Audiodrive (rev 01)
>0000:00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 02)
>0000:00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE
>(rev 01)
>0000:00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 02)
>0000:00:0b.0 Ethernet controller: 3Com Corporation 3c905B 100BaseTX
>[Cyclone] (rev 64)
>0000:01:00.0 VGA compatible controller: ATI Technologies Inc 3D Rage IIC
>AGP (rev 7a)
>
>wench:~# tail /boot/grub/menu.lst
>title           Xen 2 - Kernel 2.6.11.12
>root            (hd0,0)
>#kernel          /xen-2.0-testing.gz dom0_mem=131072 root=/dev/hda3 ro
>console=tty0
>kernel          /xen-2.0-testing.gz dom0_mem=131072 root=/dev/hda3 ro
>console=tty0 physdev_dom0_hide=(00:07.2)
>                # Marcus appends phys_hide option so BluetoothProxy domU
>can have USB Host Controller
>module          /vmlinuz-2.6.11.12-xen0 root=/dev/hda3 ro console=tty0
>savedefault
>boot
>
>===================================================================>2.
Configure kernel (& modules) for the driver domain
>
>EXAMPLE A: Firewall on brutus
>----------------------------------------------
>brutus:~# egrep "XEN|PCI|NET|VORTEX"
>/boot/config-2.6.11.12-XenFirewall  | grep -v "#"
>CONFIG_XEN=y
>CONFIG_ARCH_XEN=y
>CONFIG_XEN_PRIVILEGED_GUEST=y
>CONFIG_XEN_PHYSDEV_ACCESS=y
>CONFIG_XEN_BLKDEV_BACKEND=y
>CONFIG_XEN_NETDEV_BACKEND=y
>CONFIG_XEN_BLKDEV_FRONTEND=y
>CONFIG_XEN_NETDEV_FRONTEND=y
>CONFIG_XEN_WRITABLE_PAGETABLES=y
>CONFIG_XEN_SCRUB_PAGES=y
>CONFIG_XENARCH="i386"
>CONFIG_PCI=y
>CONFIG_PCI_DIRECT=y
>CONFIG_PCI_LEGACY_PROC=y
>CONFIG_PCI_NAMES=y
>CONFIG_HOTPLUG_PCI=m
>CONFIG_HOTPLUG_PCI_FAKE=m
>CONFIG_NET=y
>CONFIG_INET=y
>CONFIG_NET_IPIP=m
>CONFIG_NET_IPGRE=m
>CONFIG_INET_TUNNEL=m
>CONFIG_NETFILTER=y
>CONFIG_BRIDGE_NETFILTER=y
>CONFIG_IP_NF_TARGET_NETMAP=m
>CONFIG_NET_SCHED=y
>CONFIG_NET_SCH_CLK_JIFFIES=y
>CONFIG_NET_SCH_CBQ=m
>CONFIG_NET_SCH_HTB=m
>CONFIG_NET_SCH_HFSC=m
>CONFIG_NET_SCH_PRIO=m
>CONFIG_NET_SCH_RED=m
>CONFIG_NET_SCH_SFQ=m
>CONFIG_NET_SCH_TEQL=m
>CONFIG_NET_SCH_TBF=m
>CONFIG_NET_SCH_GRED=m
>CONFIG_NET_SCH_DSMARK=m
>CONFIG_NET_SCH_NETEM=m
>CONFIG_NET_SCH_INGRESS=m
>CONFIG_NET_QOS=y
>CONFIG_NET_ESTIMATOR=y
>CONFIG_NET_CLS=y
>CONFIG_NET_CLS_TCINDEX=m
>CONFIG_NET_CLS_ROUTE4=m
>CONFIG_NET_CLS_ROUTE=y
>CONFIG_NET_CLS_FW=m
>CONFIG_NET_CLS_U32=m
>CONFIG_NET_CLS_IND=y
>CONFIG_NET_CLS_RSVP=m
>CONFIG_NET_CLS_RSVP6=m
>CONFIG_NET_CLS_ACT=y
>CONFIG_NET_ACT_POLICE=m
>CONFIG_NET_ACT_GACT=m
>CONFIG_NET_ACT_MIRRED=m
>CONFIG_NET_ACT_IPT=m
>CONFIG_NET_ACT_PEDIT=m
>CONFIG_NETDEVICES=y
>CONFIG_NET_ETHERNET=y
>CONFIG_NET_VENDOR_3COM=y
>CONFIG_VORTEX=m
>CONFIG_NET_PCI=y
>CONFIG_SECURITY_NETWORK=y
>
>EXAMPLE B: BluetoothProxy on wench
>----------------------------------------------
>wench:~# egrep "XEN|PCI|USB|HCI|BT|VID"
>/boot/config-2.6.11.12-BluetoothProxy  | grep -v "#"
>CONFIG_XEN=y
>CONFIG_ARCH_XEN=y
>CONFIG_XEN_PRIVILEGED_GUEST=y
>CONFIG_XEN_PHYSDEV_ACCESS=y
>CONFIG_XEN_BLKDEV_BACKEND=y
>CONFIG_XEN_BLKDEV_FRONTEND=y
>CONFIG_XEN_NETDEV_FRONTEND=y
>CONFIG_XEN_WRITABLE_PAGETABLES=y
>CONFIG_XEN_SCRUB_PAGES=y
>CONFIG_XENARCH="i386"
>CONFIG_PCI=y
>CONFIG_PCI_DIRECT=y
>CONFIG_PCI_LEGACY_PROC=y
>CONFIG_PCI_NAMES=y
>CONFIG_HOTPLUG_PCI=m
>CONFIG_HOTPLUG_PCI_FAKE=m
>CONFIG_BT=m
>CONFIG_BT_L2CAP=m
>CONFIG_BT_SCO=m
>CONFIG_BT_RFCOMM=m
>CONFIG_BT_RFCOMM_TTY=y
>CONFIG_BT_BNEP=m
>CONFIG_BT_BNEP_MC_FILTER=y
>CONFIG_BT_BNEP_PROTO_FILTER=y
>CONFIG_BT_HIDP=m
>CONFIG_BT_HCIUSB=m
>CONFIG_BT_HCIUSB_SCO=y
>CONFIG_BT_HCIVHCI=m
>CONFIG_VIDEO_DEV=m
>CONFIG_USB=m
>CONFIG_USB_DEVICEFS=y
>CONFIG_USB_ARCH_HAS_HCD=y
>CONFIG_USB_ARCH_HAS_OHCI=y
>CONFIG_USB_UHCI_HCD=m
>CONFIG_USB_STV680=m
>
>===================================================================>3.
Configuration of driver domain
>
>wench:~# grep = /etc/xen/BluetoothProxy
>#===========================================================================>kernel
= "/boot/vmlinuz-2.6.11.12-BluetoothProxy"
>pci = [''00,7,2'']
>memory = 80
>name = "BluetoothProxy"
>disk = [ ''phy:/dev/vg0/BluetoothProxy,/dev/hda3,w'']
>root = "/dev/hda3 ro"
>extra = "4"
>restart = ''onreboot''
>#===========================================================================>
>
>===================================================================>4.
Confirming results
>
>EXAMPLE A: firewall on brutus
>---------------------------------------------
>tba
>
>EXAMPLE B: BluetoothProxy on wench
>----------------------------------------------------------
>BluetoothProxy:~# lspci
>0000:00:04.2 USB Controller: Intel Corp. 82371AB/EB/MB PIIX4 USB (rev 01)
>(Yep, that''s ALL it comes up with!:))
>
>BluetoothProxy:~# lsmod
>Module                  Size  Used by
>stv680                 26060  0
>ipt_MASQUERADE          2624  0
>iptable_nat            19836  1 ipt_MASQUERADE
>ip_conntrack           39224  2 ipt_MASQUERADE,iptable_nat
>ip_tables              19904  2 ipt_MASQUERADE,iptable_nat
>rfcomm                 35132  0
>bnep                   12256  2
>crc32                   3808  1 bnep
>l2cap                  22340  9 rfcomm,bnep
>videodev                7360  1 stv680
>hci_usb                13928  2
>bluetooth              44164  8 rfcomm,bnep,l2cap,hci_usb
>uhci_hcd               30416  0
>usbcore               107672  4 stv680,hci_usb,uhci_hcd
>
>BluetoothProxy:~# lsusb
>Bus 001 Device 004: ID 0553:0202 STMicroelectronics Imaging Division
>(VLSI Vision) Aiptek PenCam 1
>Bus 001 Device 003: ID 0db0:1967 Micro Star International Bluetooth Dongle
>Bus 001 Device 001: ID 0000:0000
>
>Using iPaq to connect to BluetoothProxy NAP service, we see:
>BluetoothProxy:~#  tail /var/log/syslog
>Aug  2 20:31:08 localhost hcid[1739]: link_key_request
>(sba=00:10:DC:E9:71:7C, dba=00:04:3E:A3:C7:73)
>Aug  2 20:31:08 localhost pand[9838]: New connection from
>00:04:3E:A3:C7:73 bnep0
>
>Testing the camera with pencam2 (must remember to null out stv680 in
>modules)
>This is the look you give when it all just works:
>http://marcusbrutus.cust.internode.on.net/Computers/Me/snapshot002
>
>===================================================================>5.
References
>
>5.1 Driver Domain Configuration
>The Xen Users'' Manual, chapter 7.3
>http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html#SECTION03230000000000000000
>
>Flat Format: Include a list of PCI device addresses of the format:
>pci = [''x,y,z'', ...]
>
>where each element in the list is a string specifying the components of
>the PCI device address, separated by commas. The components (x, y and z)
>of the list may be formatted as either decimal or hexadecimal.
>
>*****************************************************************************************************************
>
>Andreas Seuss wrote:
>
>
>
>>Hi Marcus,
>>
>>I read your mail on the Xen Users mailing list. What I try to achieve
>>is to share a pci network interface card. I compiled a kernel
>>according to your config shown below except that I used the NIC
>>modules. I run xen-3.0 (devel). When I include the pci =
[''00,08,00'']
>>statement into my config file I get the following error when trying to
>>create the domain:
>>
>>Error: Error creating domain: global name ''bus'' is not
defined
>>
>>I also set the ''physdev_dom0_hide=(00:08.0)'' in the
menu list. Tried
>>it on the kernel line and on the module line (not both on the same
>>time;-))
>>but it does not seem to work. I can still see all pci devices. Maybe
>>xou could paste the respective line from your menu.lst?
>>
>>
>
>The hide stanza goes in the kernel line.
>Can you confirm the PCI ID with lspci in dom0?
>
>
>
>>Which documentation did you use? The user manual? It contains only
>>sparse information on that topic I think. Thanks in advance.
>>
>>
>
>Yes, the user manual :) (see References above).
>Not as verbose as my attempt, huh? :)
>
>
>
>>Andreas.
>>
>>
>>Marcus Brown wrote:
>>
>>
>>
>>>Heh,
>>>ipaq turned off, bluez-utils not installed, no IRQ for USB... etc
>>>sorry to trouble the list :) ... thanks for reading!!
>>>
>>>ie. PCI + USB working afaik (not fully tested but looks ok)
>>>
>>>M.
>>>
>>>Marcus Brown wrote:
>>>
>>>
>>>
>>>
>>>
>>>>OK, PCI export working now.
>>>>
>>>>Except that I still can''t configure the bnep0
interface.
>>>>Hot plugging seems to work, but dmesg gives:
>>>>  SIOCSIFADDR: No such device
>>>>  bnep0: ERROR while getting interface flags: No such device
>>>>  SIOCSIFNETMASK: No such device
>>>>  bnep0: ERROR while getting interface flags: No such device
>>>>  Failed to bring up bnep0.
>>>>
>>>># lspci
>>>>0000:00:07.2 USB Controller: Intel Corp. 82371AB/EB/MB PIIX4 USB
>>>>(rev 01)
>>>># lsusb
>>>>Bus 001 Device 002: ID 0db0:1967 Micro Star International
Bluetooth
>>>>Dongle
>>>>Bus 001 Device 001: ID 0000:0000
>>>># lsmod
>>>>Module                  Size  Used by
>>>>bnep                   12256  0
>>>>crc32                   3808  1 bnep
>>>>l2cap                  22340  1 bnep
>>>>videodev                7360  0
>>>>hci_usb                13928  0
>>>>bluetooth              44164  3 bnep,l2cap,hci_usb
>>>>uhci_hcd               30416  0
>>>>usbcore               107672  3 hci_usb,uhci_hcd
>>>>
>>>>Any suggestions?
>>>>
>>>>(Yes, I talk to myself in the blue room too !! :))
>>>>
>>>>Marcus Brown wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>Dear, oh dear!!
>>>>>
>>>>>pci = [''00,07,2''] in domU config!!!
>>>>>(I''m going blind, I think)
>>>>>
>>>>>doh!
>>>>>
>>>>>lol
>>>>>
>>>>>Marcus Brown wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>Hi all.
>>>>>>
>>>>>>I''ve been attempting to export the USB Host
Controller to a domU,
>>>>>>so that I can use a USB Bluetooth adapter (and possibly
a USB
>>>>>>camera).
>>>>>>I''ve been using xen-testing on Debian (dom0 is
sid, domU is sarge)
>>>>>>
>>>>>>I''ve managed to get the domU to boot using a
few of the following
>>>>>>settings:
>>>>>>
>>>>>>The kernel line in grub''s menu.lst now
includes:
>>>>>>    physdev_dom0_hide=(00:07.2)
>>>>>>The domU kernel config has:
>>>>>>     XEN  --->
>>>>>>          [*] Physical device access
>>>>>>          [*]   Block-device backend driver
>>>>>>          [ ]   Network-device backend driver
>>>>>>         X86 Processor Configuration  --->
>>>>>>           Bus options (PCI, PCMCIA, EISA, MCA, ISA)
--->
>>>>>>            [*] PCI support
>>>>>>            [*] Legacy /proc/pci interface
>>>>>>            [*] PCI device name database
>>>>>>         Device Drivers  --->
>>>>>>          Networking support  --->
>>>>>>           <M> Bluetooth subsystem support
--->
>>>>>>            --- Bluetooth subsystem support
>>>>>>            <M>   L2CAP protocol support
>>>>>>            <M>   SCO links support
>>>>>>            <M>   RFCOMM protocol support
>>>>>>             [*]     RFCOMM TTY support
>>>>>>            <M>   BNEP protocol support
>>>>>>             [*]     Multicast filter support
>>>>>>             [*]     Protocol filter support
>>>>>>            <M>   HIDP protocol support
>>>>>>            Bluetooth device drivers  --->
>>>>>>             <M> HCI USB driver
>>>>>>              [*]   SCO (voice) support
>>>>>>             <M> HCI VHCI (Virtual HCI device)
driver
>>>>>>Device Drivers  --->
>>>>>>           Multimedia devices  --->
>>>>>>            <M> Video For Linux
>>>>>>           USB support  --->
>>>>>>            <M> Support for Host-side USB
>>>>>>             [*]   USB device filesystem
>>>>>>             <M> UHCI HCD (most Intel and VIA)
support
>>>>>>            <M> USB STV680 (Pencam) Camera support
>>>>>>
>>>>>>After booting the domU, the USB modules can be loaded
manually,
>>>>>>however, there are no PCI devices found, or USB devices
listed.
>>>>>>
>>>>>>I have come to the conclusion (from the xen-users
manual) that my
>>>>>>xend-config.sxp is in error. I''ve tried
appending:
>>>>>>(backend (blkif))
>>>>>>(device (pci (bus 00) (dev 07) (func 2)))
>>>>>>
>>>>>>(this could explain why I now get errors from
''xm list -v'' now:))
>>>>>>
>>>>>>but during boot of the domU I still get the following:
>>>>>>Starting hotplug subsystem:
>>>>>>   pci
>>>>>>cat: /sys/bus/pci/devices/*/class: No such file or
directory
>>>>>>cat: /sys/bus/pci/devices/*/vendor: No such file or
directory
>>>>>>cat: /sys/bus/pci/devices/*/device: No such file or
directory
>>>>>>cat: /sys/bus/pci/devices/*/subsystem_vendor: No such
file or
>>>>>>directory
>>>>>>cat: /sys/bus/pci/devices/*/subsystem_device: No such
file or
>>>>>>directory
>>>>>>Bad PCI agent invocation
>>>>>>
>>>>>>Can anyone see something obviously wrong here?
>>>>>>The users manual talks about placing these options
>>>>>>"somewhere within the vm element of its
configuration"
>>>>>>but I''m unable to find information on how to do
this.
>>>>>>
>>>>>>Thanks for any pointers,
>>>>>>
>>>>>>Marcus.
>>>>>>
>>>>>>_______________________________________________
>>>>>>Xen-users mailing list
>>>>>>Xen-users@lists.xensource.com
>>>>>>http://lists.xensource.com/xen-users
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>_______________________________________________
>>>>>Xen-users mailing list
>>>>>Xen-users@lists.xensource.com
>>>>>http://lists.xensource.com/xen-users
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>_______________________________________________
>>>>Xen-users mailing list
>>>>Xen-users@lists.xensource.com
>>>>http://lists.xensource.com/xen-users
>>>>
>>>>
>>>>
>>>>
>>>>
>>>_______________________________________________
>>>Xen-users mailing list
>>>Xen-users@lists.xensource.com
>>>http://lists.xensource.com/xen-users