Sorry to duplicate my issues onto the list - I thought I had looked
through this thread before and seen a different error, but apparently
Adam and I are running into much the same thing. I'm seeing exactly the
same "Failed to assign device to IOMMU" along with the "Device
not owned
by PCI Back End or pci-stub" in these scenarios. I have not gone back
to previous versions, but I'm running openSuSE 11.3, so it's not quite
that easy for me to go back to 3.3.x on that platform.
-Nick
On Thu, 2010-10-21 at 16:16 +0100, Adam Moss wrote:> Hi all,
>
> I have worked with Michael on this problem, here are some further
> details that may help troubleshoot the issue:
>
> The error seen when starting DomU is in fact "Failed to assign device
> to IOMMU".
>
> We're using the following system setup for our Xen environment:
>
> SunBlade 6000 chassis
> SunBlade X6270 blade
> SuSE ES11 SP1 64bit Dom0 (Xen 4.0.0 - Kernel 2.6.32.12-0.7-xen )
> SuSE ES11 SP1 64bit DomU
>
> The lspci -vt output at Dom0 looks like this:
>
> ###
> -[0000:00]-+-00.0 Intel Corporation 5520 I/O Hub to ESI Port
> +-01.0-[0000:01-06]----00.0 LSI Logic / Symbios Logic
> SAS1068E PCI-Express Fusion-MPT SAS
> +-03.0-[0000:07-0c]--
> +-05.0-[0000:0d-12]----00.0-[0000:0e]----00.0 Freescale
> Semiconductor Inc Device 0091
> +-07.0-[0000:13-18]--
> +-09.0-[0000:19-1e]--
> +-13.0 Intel Corporation 5520/5500/X58 I/O Hub I/OxAPIC
> Interrupt Controller
> +-14.0 Intel Corporation 5520/5500/X58 I/O Hub System
> Management Registers
> +-14.1 Intel Corporation 5520/5500/X58 I/O Hub GPIO and
> Scratch Pad Registers
> +-14.2 Intel Corporation 5520/5500/X58 I/O Hub Control
> Status and RAS Registers
> +-14.3 Intel Corporation 5520/5500/X58 I/O Hub Throttle
> Registers
> +-16.0 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-16.1 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-16.2 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-16.3 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-16.4 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-16.5 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-16.6 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-16.7 Intel Corporation 5520/5500/X58 Chipset QuickData
> Technology Device
> +-1a.0 Intel Corporation 82801JI (ICH10 Family) USB UHCI
> Controller #4
> +-1a.1 Intel Corporation 82801JI (ICH10 Family) USB UHCI
> Controller #5
> +-1a.2 Intel Corporation 82801JI (ICH10 Family) USB UHCI
> Controller #6
> +-1a.7 Intel Corporation 82801JI (ICH10 Family) USB2 EHCI
> Controller #2
> +-1c.0-[0000:1f]--+-00.0 Intel Corporation 82575EB Gigabit
> Network Connection
> | \-00.1 Intel Corporation 82575EB Gigabit
> Network Connection
> +-1d.0 Intel Corporation 82801JI (ICH10 Family) USB UHCI
> Controller #1
> +-1d.1 Intel Corporation 82801JI (ICH10 Family) USB UHCI
> Controller #2
> +-1d.2 Intel Corporation 82801JI (ICH10 Family) USB UHCI
> Controller #3
> +-1d.7 Intel Corporation 82801JI (ICH10 Family) USB2 EHCI
> Controller #1
> +-1e.0-[0000:20]----05.0 ASPEED Technology, Inc. ASPEED
> Graphics Family
> +-1f.0 Intel Corporation 82801JIR (ICH10R) LPC Interface
> Controller
> +-1f.2 Intel Corporation 82801JI (ICH10 Family) SATA AHCI
> Controller
> \-1f.3 Intel Corporation 82801JI (ICH10 Family) SMBus
> Controller
> ###
>
> The following commands are used to hide the single device
> (0000:0e:00.0) we want to pass back to DomU:
>
> ###
> modprobe pciback
>
> echo -n "0000:0e:00.0" > /sys/bus/pci/drivers/pciback/new_slot
> echo -n "0000:0e:00.0" > /sys/bus/pci/drivers/pciback/bind
> ###
>
> After this the device is shown correctly in the assignable devices
> list.
>
> ###
> SunBlade2:/etc/xen/vm # xm pci-list-assignable-devices
> 0000:0e:00.0
> ###
>
> The DomU is then started/created using the following configuration
> file:
>
> ###
> import os, re
> arch = os.uname()[4]
> if re.search('64', arch):
> arch_libdir = 'lib64'
> else:
> arch_libdir = 'lib'
> kernel = "/usr/lib/xen/boot/hvmloader"
> builder = 'hvm'
> device_model = '/usr/' + arch_libdir +
'/xen/bin/qemu-dm'
> name = "node1"
> vcpus = 1
> memory = 1024
> pae = 1
> acpi = 1
> apic = 1
>
> boot = "dc"
> vif =['mac=00:11:22:33:00:04, bridge=br0, model=e1000']
> disk =['phy:sda5,hda,w']
>
> on_poweroff = 'destroy'
> on_reboot = 'restart'
> on_crash = 'restart'
>
> snapshot = 0
> sdl = 0
> opengl = 1
> vnc = 1
> vnclisten = "127.0.0.1"
> vncdisplay = 4
> vncunused = 0
> vncpasswd = ''
> nographic = 0
> stdvga = 0
> serial = 'pty'
>
> pci = ['0000:0e:00.0']
> ###
>
> After running the create command the "failed to assign to IOMMU"
error
> is seen:
>
> ###
> SunBlade2:/etc/xen/vm # xm create node1.conf
> Using config file "./node1.conf".
> Error: Failed to assign device to IOMMU
> (0000:0e:00.0@100,msitranslate=1,power_mgmt=0)
> ###
>
> We have enabled the debug traces (iommu=verbose loglvl=all
> guest_loglvl=all) which show the following at the point of failure
> (Please ignore the time stamps):
>
> /var/log/messages
> --------------------------
>
> Mar 10 02:57:05 SunBlade2 kernel: [ 1495.751322]
> (cdrom_add_media_watch()
>
file=/usr/src/packages/BUILD/kernel-xen-2.6.32.12/linux-2.6.32/drivers/xen/blkback/cdrom.c,
line=108) nodename:backend/vbd/2/768
> Mar 10 02:57:05 SunBlade2 kernel: [ 1495.751327] (cdrom_is_type()
>
file=/usr/src/packages/BUILD/kernel-xen-2.6.32.12/linux-2.6.32/drivers/xen/blkback/cdrom.c,
line=95) type:0
> Mar 10 02:57:05 SunBlade2 kernel: [ 1495.936015] pcieport
> 0000:00:05.0: AER: Uncorrected (Non-Fatal) error received: id=0028
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.023732] pcieport
> 0000:00:05.0: PCIE Bus Error: severity=Uncorrected (Non-Fatal),
> type=Transaction Layer, id=0028(Requester ID)
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.023737] pcieport
> 0000:00:05.0: device [8086:340c] error
> status/mask=00004000/00000000
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.023742] pcieport
> 0000:00:05.0: [14] Completion Timeout (First)
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024193] pcieport
> 0000:00:05.0: broadcast error_detected message
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024198] pciback 0000:0e:00.0:
> pciback device is not found/assigned
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024201] pcieport
> 0000:00:05.0: broadcast mmio_enabled message
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024203] pciback 0000:0e:00.0:
> pciback device is not found/assigned
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024205] pcieport
> 0000:00:05.0: broadcast resume message
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024208] pciback 0000:0e:00.0:
> pciback device is not found/assigned
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024213] pcieport
> 0000:00:05.0: AER driver successfully recovered
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024216] pcieport
> 0000:00:05.0: AER: Uncorrected (Non-Fatal) error received: id=0028
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024305] pcieport
> 0000:00:05.0: can't find device of ID0028
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024307] pcieport
> 0000:00:05.0: AER: Uncorrected (Non-Fatal) error received: id=0028
> Mar 10 02:57:05 SunBlade2 kernel: [ 1496.024395] pcieport
> 0000:00:05.0: can't find device of ID0028
> Mar 10 02:57:05 SunBlade2 kernel: [ 1495.587191] device tap2.0 entered
> promiscuous mode
> Mar 10 02:57:05 SunBlade2 kernel: [ 1495.587205] br0: port 3(tap2.0)
> entering forwarding state
> Mar 10 02:57:05 SunBlade2 kernel: [ 1495.771897] br0: port 3(tap2.0)
> entering disabled state
> Mar 10 02:57:06 SunBlade2 kernel: [ 1495.959814] device tap2.0 left
> promiscuous mode
>
> /var/log/xen/xend.log
> ----------------------------
>
> [2011-03-10 02:58:42 5613] DEBUG (XendDomainInfo:825)
> XendDomainInfo.hvm_pci_device_insert_dev:
> 0000:0e:00.0@100,msitranslate=1,power_mgmt=0
> [2011-03-10 02:58:42 5613] ERROR (XendDomainInfo:2957)
> XendDomainInfo.initDomain: exception occurred
> Traceback (most recent call last):
> File
"/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
> line 2949, in _initDomain
> self._createDevices()
> File
> "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
line
> 2440, in _createDevices
> self.pci_device_configure_boot()
> File
> "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
line
> 641, in pci_device_configure_boot
> self.pci_device_configure(dev_sxp, first_dev = first)
> File
> "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
line
> 934, in pci_device_configure
> vdevfn = self.hvm_pci_device_insert(dev_config)
> File
> "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
line
> 800, in hvm_pci_device_insert
> return self.hvm_pci_device_insert_dev(new_dev)
> File
> "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
line
> 828, in hvm_pci_device_insert_dev
> raise VmError("Failed to assign device to IOMMU (%s)" %
bdf_str)
> VmError: Failed to assign device to IOMMU
> (0000:0e:00.0@100,msitranslate=1,power_mgmt=0)
> [2011-03-10 02:58:42 5613] ERROR (XendDomainInfo:486) VM start failed
> Traceback (most recent call last):
> File
> "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
line
> 472, in start
> XendTask.log_progress(31, 60, self._initDomain)
> File "/usr/lib64/python2.6/site-packages/xen/xend/XendTask.py",
line
> 209, in log_progress
> retval = func(*args, **kwds)
> File
> "/usr/lib64/python2.6/site-packages/xen/xend/XendDomainInfo.py",
line
> 2960, in _initDomain
> raise exn
> VmError: Failed to assign device to IOMMU
> (0000:0e:00.0@100,msitranslate=1,power_mgmt=0)
>
>
> xm dmes
> -------------
>
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to
switch
> input to Xen)
> (XEN) Freed 184kB init memory.
> (XEN) mtrr: type mismatch for fb000000,800000 old: write-back new:
> write-combining
> (XEN) [VT-D]iommu.c:1461: d0:PCI: unmap bdf = e:0.0
> (XEN) [VT-D]iommu.c:1332: d1:PCI: map bdf = e:0.0
> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device (e:0.0)
> failed
> (XEN) event_channel.c:192:d0 EVTCHNOP failure: domain 1, error -22
> (XEN) [VT-D]iommu.c:1461: d0:PCI: unmap bdf = e:0.0
> (XEN) [VT-D]iommu.c:1332: d2:PCI: map bdf = e:0.0
> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device (e:0.0)
> failed
> (XEN) [VT-D]iommu.c:1461: d0:PCI: unmap bdf = e:0.0
> (XEN) [VT-D]iommu.c:1332: d3:PCI: map bdf = e:0.0
> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device (e:0.0)
> failed
> (XEN) [VT-D]iommu.c:1461: d0:PCI: unmap bdf = e:0.0
> (XEN) [VT-D]iommu.c:1332: d4:PCI: map bdf = e:0.0
> (XEN) domctl.c:853:d0 XEN_DOMCTL_assign_device: assign device (e:0.0)
> failed
>
> The full trace (xm dmes, xend.log) output and other relevant
> information from the system is attached.
>
> Things to note
> ===========
>
> 1) The problem still occurs if we try and hotplug the device using "xm
> pci-attach ..."
> 2) If the same hardware and software configuration is used with Xen
> 3.3.1 at Dom0 (SLES11) the problem is not seen.
> 3) We have compiled Xen 4.0.1 for SLES11 SP1 at Dom0 and still see the
> same problem.
> 4) We have tried passing back the PLX bridge device "0d:00.0"
> associated with the device as well as the device itself however we
> then see the error "Error: pci: PCI Backend and pci-stub don't own
> device 0000:0d:00.0" during DomU startup even though 0d:00.0 is shown
> in the assignable device list.
>
> Looking at recent posts this problems looks very similar to the one in
> subject "PCI Passthrough Problems/Questions" posted today.
>
> Any pointers at this stage would be great.
> Thanks,
> Adam
> ----------
> Adam Moss
>
> Adax Europe Ltd
> Email : Adam.Moss@adax.co.uk
> On 21/09/2010 20:05, Michael Maher wrote:
> > Hello,
> >
> > I was wondering if anybody had any insight they could lend us on this?
> >
> > Your help would be very much appreciated,
> >
> >
> > Many Thanks,
> > Michael
> >
> >
> >
> >
> > On Thu, 2010-09-16 at 18:44 +0100, Michael Maher wrote:
> > > Hello,
> > >
> > >
> > >
> > > Im hoping someone may be able to push us in the right direction.
Im
> > > trying to get one of our products to work with the current branch
of
> > > Xen (4.0/4.0.0) but Im hitting a problem.
> > >
> > >
> > >
> > > We are currently using Xen 3.3.2 on a Sun Blade 600 chassis (with
> > > x6270 blades) this works great.
> > >
> > >
> > >
> > > But when we upgrade to Xen 4.0/4.0.1 we get the following message
when
> > > starting the DomU:
> > >
> > > Device Cannot be passed to IOMMU
> > >
> > >
> > >
> > > I understand from the release note that there were some changes
made
> > > to the IOMMU/pciback pass-through in this release, does anyone
have
> > > any details as to what might be going wrong?
> > >
> > >
> > >
> > > Were using SLES11/sp1 and Ive tried with the Novell build and
> > > compiled it myself. I also have the latest firmware for the
chassis
> > > and blade from Sun, all of which show the same problem.
> > >
> > >
> > >
> > > Has anyone seen to same thing?
> > >
> > > Any help would be greatly appreciated,
> > >
> > >
> > >
> > >
> > >
> > > Many Thanks,
> > >
> > > Michael
> > >
> > >
> > > Michael Maher
> > >
> > > Adax Europe Ltd
> > >
> > > mmaher@adax.co.uk
> > >
> > >
--------
This e-mail may contain confidential and privileged material for the sole use of
the intended recipient. If this email is not intended for you, or you are not
responsible for the delivery of this message to the intended recipient, please
note that this message may contain SEAKR Engineering (SEAKR)
Privileged/Proprietary Information. In such a case, you are strictly prohibited
from downloading, photocopying, distributing or otherwise using this message,
its contents or attachments in any way. If you have received this message in
error, please notify us immediately by replying to this e-mail and delete the
message from your mailbox. Information contained in this message that does not
relate to the business of SEAKR is neither endorsed by nor attributable to
SEAKR.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
