Xen devel - Oct 2008 - iommu: mapping reserved region failed - Q35 - VT-D Issue

Xen 3.4 xen-unstable.hg from yesterday with debian etch on 64bit arch
Intel/Lenovo Q35 Mainboard with VT-d enabled

Bootoptions iommu=1 vtd=1
pci.backhide for a PCI-E nvidia graphiccard

xm dmesg Error messages includes:

[VT-D] iommu.c: 1694:d32767 iommu: mapping reserved region failed
[VT-D] iommu.c: 1542:d0 intel_iommu_add_device: context mapping failed

If i try to start my HVM by xm create win2k the system reboots

any ideas?

-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer schrieb:
> Bootoptions iommu=1 vtd=1
> pci.backhide for a PCI-E nvidia graphiccard
As mentioned in Documentation/Intel-IOMMU.txt boot option

intel_iommu=igfx_off did not change anything.

Regards

-- 
stefan.bauer@cubewerk.de		Cubewerk
Tel   +49 8621 996 02 37		IT-Beratung + Planung
Tel   +49  179 119 47 67		Verkauf von Hard und Software
Fax   +49 1212 511057903		Herzog-Otto-Straße 32
http://www.cubewerk.de			83308 Trostberg

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan,

So those errors happened during booting DOM0 and then you checked xm
dmesg, correct? Then you started a 2k VM and the system reset. Can you
rebuild xen with debug=y, you might get more information about why the
context mapping is failing. We could also put some more trace in there
to see exactly what happens. 

Can you attach a serial cable and send the output from xen and dom0 to
COM1 or something like that? There may be some relevant information
traced out right before the reset that you are not seeing (it is
probably a crash in xen that is resetting the system).

Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Wednesday, October 29, 2008 1:28 PM
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] iommu: mapping reserved region failed - Q35 - VT-D
Issue

Xen 3.4 xen-unstable.hg from yesterday with debian etch on 64bit arch
Intel/Lenovo Q35 Mainboard with VT-d enabled

Bootoptions iommu=1 vtd=1
pci.backhide for a PCI-E nvidia graphiccard

xm dmesg Error messages includes:

[VT-D] iommu.c: 1694:d32767 iommu: mapping reserved region failed
[VT-D] iommu.c: 1542:d0 intel_iommu_add_device: context mapping failed

If i try to start my HVM by xm create win2k the system reboots

any ideas?

-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> Stefan,
> 
> So those errors happened during booting DOM0 and then you checked xm
> dmesg, correct? Then you started a 2k VM and the system reset. Can you
> rebuild xen with debug=y, you might get more information about why the
> context mapping is failing. We could also put some more trace in there
> to see exactly what happens. 
I only found PCI Backend Debugging in the Xen menue which i
activacted, rebuild kernel and did again xm dmesg without further
informations. Do i have to activate it additionally?
> Can you attach a serial cable and send the output from xen and dom0 to
> COM1 or something like that? There may be some relevant information
> traced out right before the reset that you are not seeing (it is
> probably a crash in xen that is resetting the system).
I''ll try it if my laptop becomes available (women...)

-- 

Stefan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Can you put some printk''s in domain_context_mapping() to see at which
point it failed?

If it is a BIOS problem, it is possible this function cannot find the matching
vt-d engine for this device.

Allen
>-----Original Message-----
>From: xen-devel-bounces@lists.xensource.com 
>[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 
>Stefan Bauer
>Sent: Wednesday, October 29, 2008 10:28 AM
>To: xen-devel@lists.xensource.com
>Subject: [Xen-devel] iommu: mapping reserved region failed - 
>Q35 - VT-D Issue
>
>Xen 3.4 xen-unstable.hg from yesterday with debian etch on 64bit arch
>Intel/Lenovo Q35 Mainboard with VT-d enabled
>
>Bootoptions iommu=1 vtd=1
>pci.backhide for a PCI-E nvidia graphiccard
>
>xm dmesg Error messages includes:
>
>[VT-D] iommu.c: 1694:d32767 iommu: mapping reserved region failed
>[VT-D] iommu.c: 1542:d0 intel_iommu_add_device: context mapping failed
>
>If i try to start my HVM by xm create win2k the system reboots
>
>any ideas?
>
>-- 
>stefan
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I meant rebuild xen with debug=y. There is a bunch of tracing in the
routines that are trying to setup the context mapping that indicate what
is going on including tracing out what device it was dealing with when
things failed. Also (as mentioned in another email just sent), we can
enhance that logging in these routines to see if the hardware remapping
unit is missing for this device or something related.

Also again I was curious - it seems the failure trace you reported below
was during dom0 boot - is this true?

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Wednesday, October 29, 2008 3:16 PM
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35 -
VT-DIssue

Ross Philipson schrieb:
> Stefan,
> 
> So those errors happened during booting DOM0 and then you checked xm
> dmesg, correct? Then you started a 2k VM and the system reset. Can you
> rebuild xen with debug=y, you might get more information about why the
> context mapping is failing. We could also put some more trace in there
> to see exactly what happens. 
I only found PCI Backend Debugging in the Xen menue which i
activacted, rebuild kernel and did again xm dmesg without further
informations. Do i have to activate it additionally?
> Can you attach a serial cable and send the output from xen and dom0 to
> COM1 or something like that? There may be some relevant information
> traced out right before the reset that you are not seeing (it is
> probably a crash in xen that is resetting the system).
I''ll try it if my laptop becomes available (women...)

-- 

Stefan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Kay, Allen M schrieb:
> Can you put some printk''s in domain_context_mapping() to see at
which point it failed?
> 
> If it is a BIOS problem, it is possible this function cannot find the
matching vt-d engine for this device.
Allen,

unfortunately i''m not familiar with any higher programming language.
Please see attached a bit more verbose informations generated by PCI
Backend Debugging in the kernel:

For example:

pciback 0000:00:19.0: probing...
BUG: warning at
/usr/src/linux-2.6.18-xen.hg/drivers/xen/core/pci.c:31/pci_bus_probe_wrapper()

Call Trace:
 [<ffffffff8030cee1>] pci_bus_probe_wrapper+0x8f/0x9a

get''s me an unavailable network controller which has the 0000:00:19.0
address.

-- 
Stefan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> I meant rebuild xen with debug=y. There is a bunch of tracing in the
> routines that are trying to setup the context mapping that indicate what
> is going on including tracing out what device it was dealing with when
> things failed. Also (as mentioned in another email just sent), we can
> enhance that logging in these routines to see if the hardware remapping
> unit is missing for this device or something related.
debug=y can just be applied right after make linux-2.6-xen-build? or
where do i supply that debug=y ?
> Also again I was curious - it seems the failure trace you reported below
> was during dom0 boot - is this true?
Yes it''s all during dom0 booting.

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

For rebuilding the xen part - the messages are coming out of the
hypervisor. E.g:

xen-unstable.hg/xen/make install debug=y

Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Wednesday, October 29, 2008 3:41 PM
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35
-VT-DIssue

Ross Philipson schrieb:
> I meant rebuild xen with debug=y. There is a bunch of tracing in the
> routines that are trying to setup the context mapping that indicate
what
> is going on including tracing out what device it was dealing with when
> things failed. Also (as mentioned in another email just sent), we can
> enhance that logging in these routines to see if the hardware
remapping
> unit is missing for this device or something related.
debug=y can just be applied right after make linux-2.6-xen-build? or
where do i supply that debug=y ?
> Also again I was curious - it seems the failure trace you reported
below
> was during dom0 boot - is this true?
Yes it''s all during dom0 booting.

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> For rebuilding the xen part - the messages are coming out of the
> hypervisor. E.g:
> 
> xen-unstable.hg/xen/make install debug=y
Done, xenddebug.log attached.

HTH

Regards

-- 
Stefan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer wrote:
> Ross Philipson schrieb:
>> For rebuilding the xen part - the messages are coming out of the
>> hypervisor. E.g: 
>> 
>> xen-unstable.hg/xen/make install debug=y
> 
> Done, xenddebug.log attached.
> 
> HTH
> 
> Regards
Stefan,

xenddebug.log is not helpful. when you compile xen, pls use: make xen debug=y

or add "loglvl=all loglvl_guest=all" in grub kernel line like:
	kernel /boot/xen.gz dom0_mem=512m iommu=1 loglvl=all loglvl_guest=all

Per xm-dmesg in your another mail, I am very curious on why
domain_context_mapping failed (which also result in failure of reserved memory
mapping).

Per dom0 failure info (pci_bus_probe_wapper()), which is caused by
pci_remove_device(), I suspect the devices are not added to alldevs_list.

Pls use above method to catch more debug info, then we can analyse the info
easily.

Regards,
Weidong
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Han, Weidong schrieb:
> or add "loglvl=all loglvl_guest=all" in grub kernel line like:
> 	kernel /boot/xen.gz dom0_mem=512m iommu=1 loglvl=all loglvl_guest=all
Weidong,

please see attached a even more verbose debugging output.
Unfortunatel i was not able to redirect the console output to a serial
terminal, so i took a screenshot[1].

[1] www.plzk.de/IMG_2598.JPG

HTH

-- 
Stefan




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer schrieb:
> [1] www.plzk.de/IMG_2598.JPG
File updated, i uploaded the wrong one, sorry for that.

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer wrote:
> Han, Weidong schrieb:
>> or add "loglvl=all loglvl_guest=all" in grub kernel line
like:
>> 	kernel /boot/xen.gz dom0_mem=512m iommu=1 loglvl=all
>> loglvl_guest=all 
> 
> Weidong,
> 
> please see attached a even more verbose debugging output.
> Unfortunatel i was not able to redirect the console output to a serial
> terminal, so i took a screenshot[1].
Can you add some printk in domain_context_mapping()? Where does it exit?

You''d better make serial console work. Following is my grub including
serial console config.

title 3: xen on RHEL5 (x86_64)
        root (hd0,0)
        kernel /boot/xen.gz dom0_mem=512m console=com1 iommu=1 loglvl=all
loglvl_guest=all
        module /boot/vmlinuz-2.6.18.8-xen ro root=/dev/sda1 xencons=ttyS
console=tty0 console=ttyS0, pciback.hide=(01:00.0)
        module /boot/initrd-2.6.18.8-xen.img

Regards,
Weidong
> 
> [1] www.plzk.de/IMG_2598.JPG
> 
> HTH

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Han, Weidong schrieb:
> You''d better make serial console work. Following is my grub
including serial console config.
Weidong,

i get my hands on a usb-serial adapter, please see the output right here:

www.plzk.de/xendebug-serial.txt

Hope that helps.

Regards

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer wrote:
> Han, Weidong schrieb:
>> You''d better make serial console work. Following is my grub
>> including serial console config. 
> 
> Weidong,
> 
> i get my hands on a usb-serial adapter, please see the output right
> here: 
> 
> www.plzk.de/xendebug-serial.txt
Why it''s so short? Can post complete output?

And also that is a panic. Why it''s not consistent with the sighting
described in your previous mails?

I suspect your environment is not incorrect in somewhere. Pls list the following
info: platform, BIOS, dom0 changeset, Xen changeset, did you add-on extra
graphics card? did you follow steps listed in docs/misc/vtd.txt?

Actually, latest Xen VT-d is quite stable in our recent testing.

Regards,
Weidong
> 
> Hope that helps.
> 
> Regards

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Han, Weidong schrieb:
> Why it''s so short? Can post complete output?
I expected also an even bigger output, but it shows debug=n even
though i enabled it with loglvl=all loglvl_guest=all

Please see the new complete output here:

www.plzk.de/CAPTURE.TXT
> I suspect your environment is not incorrect in somewhere. Pls list the
following info: platform, BIOS, dom0 changeset, Xen changeset, did you add-on
extra graphics card? did you follow steps listed in docs/misc/vtd.txt?
64BIT Debian stable (etch)
Lenovo Thinkcentre M57 - Typ 6075-BQG
Bios Version 2RKT54AUS
Bios Date 08/20/2008
Dom0 is 2.6.18.8-xen Kernel with Xen 3.4 xen-unstable.hg, plain
installation without any modifications at all

One HVM guest for testing, windows 2000 with pci = [ ''
01:00.0'' ] line
in its configuration.

I add a pci graphic card for testing purpose to have vga output in
case i hide the pci-e card. that pci card is not used for passthrough.

I followed steps listed in
http://lxr.xensource.com/lxr/source/docs/misc/vtd.txt but came to the
conclusion that it might have something to do with Base Address Register?

> Actually, latest Xen VT-d is quite stable in our recent testing.
I wish here too.

Regards

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer wrote:
> Han, Weidong schrieb:
>> Why it''s so short? Can post complete output?
> 
> I expected also an even bigger output, but it shows debug=n even
> though i enabled it with loglvl=all loglvl_guest=all
> 
> Please see the new complete output here:
> 
> www.plzk.de/CAPTURE.TXT
Yes, this output is complete and helpful. I found the problem exists in
domain_context_mapping(), which never succeeds. Thus result in other problems.
Can you add some printk to locate where it fails?

In addition, I suggest you don''t add extra graphic card into your
machine, as I know it may introduce issues on Q35.

Regards,
Weidong
> 
>> I suspect your environment is not incorrect in somewhere. Pls list
>> the following info: platform, BIOS, dom0 changeset, Xen changeset,
>> did you add-on extra graphics card? did you follow steps listed in
>> docs/misc/vtd.txt?   
> 
> 64BIT Debian stable (etch)
> Lenovo Thinkcentre M57 - Typ 6075-BQG
> Bios Version 2RKT54AUS
> Bios Date 08/20/2008
> Dom0 is 2.6.18.8-xen Kernel with Xen 3.4 xen-unstable.hg, plain
> installation without any modifications at all
> 
> One HVM guest for testing, windows 2000 with pci = [ ''
01:00.0'' ] line
> in its configuration.
> 
> I add a pci graphic card for testing purpose to have vga output in
> case i hide the pci-e card. that pci card is not used for passthrough.
> 
> I followed steps listed in
> http://lxr.xensource.com/lxr/source/docs/misc/vtd.txt but came to the
> conclusion that it might have something to do with Base Address
> Register? 
> 
> 
>> Actually, latest Xen VT-d is quite stable in our recent testing. I
>> wish here too. 
> 
> Regards

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Han, Weidong schrieb:
> Yes, this output is complete and helpful. I found the problem exists in
domain_context_mapping(), which never succeeds. Thus result in other problems.
Can you add some printk to locate where it fails?
I will if you advice me how to do where exactly.
> In addition, I suggest you don''t add extra graphic card into your
machine, as I know it may introduce issues on Q35.
Problem is that i want to use the pci-e card for my hvm guest and the
onboard one for dom0. it seems that both the onboard and pci-e one
cant work alongside, so iam forced to add another one.

Regards


-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan,

Most of the paths through the function domain_context_mapping() (in
xen/drivers/passthrough/vtd/iommu.c) also trace extra information but I
don''t see those traces so it is reasonable to assume that the function
is returning an error when the call to acpi_find_matched_drhd_unit()
fails to find a DRHD for the particular device. I would start by putting
a trace there in the failure case and trace out the bus/dev info
(presumably it is device 00:03.2). If this is the case then we can look
at your DMAR ACPI table - maybe this is a faulty BIOS issue (e.g. there
is no DRHD reporting INCLUDE_ALL devices).

Or I may be wrong and the logging level is just not set for those
gdprintk() calls in trace in the switch statement (though I don''t think
this is the case). If it is the case put code after the various calls in
the switch statement to test the "ret" value and trace an error if it
fails.

Thanks
Ross


-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Friday, October 31, 2008 3:51 AM
Cc: ''xen-devel@lists.xensource.com''
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35
-VT-DIssue

Han, Weidong schrieb:
> Yes, this output is complete and helpful. I found the problem exists
in domain_context_mapping(), which never succeeds. Thus result in other
problems. Can you add some printk to locate where it fails?

I will if you advice me how to do where exactly.
> In addition, I suggest you don''t add extra graphic card into your
machine, as I know it may introduce issues on Q35.  

Problem is that i want to use the pci-e card for my hvm guest and the
onboard one for dom0. it seems that both the onboard and pci-e one
cant work alongside, so iam forced to add another one.

Regards


-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> Stefan,
> 
> Most of the paths through the function domain_context_mapping() (in
> xen/drivers/passthrough/vtd/iommu.c) also trace extra information but I
> don''t see those traces so it is reasonable to assume that the
function
> is returning an error when the call to acpi_find_matched_drhd_unit()
> fails to find a DRHD for the particular device. I would start by putting
> a trace there in the failure case and trace out the bus/dev info
> (presumably it is device 00:03.2). If this is the case then we can look
> at your DMAR ACPI table - maybe this is a faulty BIOS issue (e.g. there
> is no DRHD reporting INCLUDE_ALL devices).
Sorry but thats a way to technically for me. I''m not familiar with C
nor any other higher programming language. I will provide informations
and do testing.
> Or I may be wrong and the logging level is just not set for those
> gdprintk() calls in trace in the switch statement (though I don''t
think
> this is the case). If it is the case put code after the various calls in
> the switch statement to test the "ret" value and trace an error
if it
Regards

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer wrote:
> Han, Weidong schrieb:
>> Yes, this output is complete and helpful. I found the problem exists
>> in domain_context_mapping(), which never succeeds. Thus result in
>> other problems. Can you add some printk to locate where it fails?  
> 
> I will if you advice me how to do where exactly.
I think you can add printk after check of drhd:

     if ( !drhd )
+    {
+        printk("%s: can''t find drhd for device (bus=%x dev=%x
func=%x)\n",
+                __func__, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
         return -ENODEV;
+    }

and you also can change XENLOG_INFO to XENLOG_ERR in domain_context_mapping(). 
> 
>> In addition, I suggest you don''t add extra graphic card into
your
>> machine, as I know it may introduce issues on Q35. 
> 
> Problem is that i want to use the pci-e card for my hvm guest and the
> onboard one for dom0. it seems that both the onboard and pci-e one
> cant work alongside, so iam forced to add another one.
> 
A few months ago,  in xen mailing list I remember some guys reported: when using
PEG card (turning off integrated Gfx) on Q35, VT-d cannot work. It''s
the bios issue. I suspect it''s not fixed yet. So I suggest you to use
IGD instead of PEG.

Regards,
Weidong
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Han, Weidong schrieb:
>      if ( !drhd )
> +    {
> +        printk("%s: can''t find drhd for device (bus=%x
dev=%x func=%x)\n",
> +                __func__, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
>          return -ENODEV;
> +    }
> 
> and you also can change XENLOG_INFO to XENLOG_ERR in
domain_context_mapping().
Added and recompiled, output here:

www.plzk.de/CAPTURE2
> A few months ago,  in xen mailing list I remember some guys reported: when
using PEG card (turning off integrated Gfx) on Q35, VT-d cannot work.
It''s the bios issue. I suspect it''s not fixed yet. So I
suggest you to use IGD instead of PEG.
I would, but if i do not select PEG in bios, lspci did not show the
PEG card.

Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

OK, that is what I was going to give you anyway and that is sort of what
I expected. Can you get a dump of you ACPI DMAR table. You can use
pmtools to dump the ACPI tables:

http://kernel.org/pub/linux/kernel/people/lenb/acpi/utils/

You will have to build it and then run the dump tool. If you use the all
option it will dump everything. I then have a little tool that prints
out the DMAR in human readable form so we can see what the DRHD
reporting structures look like. 

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Friday, October 31, 2008 10:32 AM
Cc: ''xen-devel@lists.xensource.com''
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35
-VT-DIssue

Han, Weidong schrieb:
>      if ( !drhd )
> +    {
> +        printk("%s: can''t find drhd for device (bus=%x
dev=%x
func=%x)\n",
> +                __func__, bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
>          return -ENODEV;
> +    }
> 
> and you also can change XENLOG_INFO to XENLOG_ERR in
domain_context_mapping(). 

Added and recompiled, output here:

www.plzk.de/CAPTURE2
> A few months ago,  in xen mailing list I remember some guys reported:
when using PEG card (turning off integrated Gfx) on Q35, VT-d cannot
work. It''s the bios issue. I suspect it''s not fixed yet. So I
suggest
you to use IGD instead of PEG.

I would, but if i do not select PEG in bios, lspci did not show the
PEG card.

Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> You will have to build it and then run the dump tool. If you use the all
> option it will dump everything. I then have a little tool that prints
> out the DMAR in human readable form so we can see what the DRHD
> reporting structures look like. 
Here we go:

www.plzk.de/acpidump.txt

-- 
stefan.bauer@cubewerk.de		Cubewerk
Tel   +49 8621 996 02 37		IT-Beratung + Planung
Tel   +49  179 119 47 67		Verkauf von Hard und Software
Fax   +49 1212 511057903		Herzog-Otto-Straße 32
http://www.cubewerk.de			83308 Trostberg

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Attached is a printout of your DMAR tables. Focusing on the following failures
that occurred later in the boot process:

(XEN) domain_context_mapping: can''t find drhd for device (bus=0 dev=1f
func=2)
(XEN) [VT-D]iommu.c:1545:d0 intel_iommu_add_device: context mapping failed
(XEN) domain_context_mapping: can''t find drhd for device (bus=0 dev=1f
func=5)
(XEN) [VT-D]iommu.c:1545:d0 intel_iommu_add_device: context mapping failed

The DMAR only has two DRHD with 1 and 3 scope entries respectively for device
endpoints (covering devices 00:1b.0, 00:03.0, 00:03.2, 00:03.3). There are no
scope entries for entire bridges and there is not an INCLUDE_ALL DRHD. This is
what is causing the failure - no match. It seems odd to me that there are so few
entries.

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Friday, October 31, 2008 11:27 AM
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35 -VT-DIssue

Ross Philipson schrieb:
> You will have to build it and then run the dump tool. If you use the all
> option it will dump everything. I then have a little tool that prints
> out the DMAR in human readable form so we can see what the DRHD
> reporting structures look like. 
Here we go:

www.plzk.de/acpidump.txt

-- 
stefan.bauer@cubewerk.de		Cubewerk
Tel   +49 8621 996 02 37		IT-Beratung + Planung
Tel   +49  179 119 47 67		Verkauf von Hard und Software
Fax   +49 1212 511057903		Herzog-Otto-Straße 32
http://www.cubewerk.de			83308 Trostberg

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> Attached is a printout of your DMAR tables. Focusing on the following
failures that occurred later in the boot process:
> 
> (XEN) domain_context_mapping: can''t find drhd for device (bus=0
dev=1f func=2)
> (XEN) [VT-D]iommu.c:1545:d0 intel_iommu_add_device: context mapping failed
> (XEN) domain_context_mapping: can''t find drhd for device (bus=0
dev=1f func=5)
> (XEN) [VT-D]iommu.c:1545:d0 intel_iommu_add_device: context mapping failed
> 
> The DMAR only has two DRHD with 1 and 3 scope entries respectively for
device endpoints (covering devices 00:1b.0, 00:03.0, 00:03.2, 00:03.3). There
are no scope entries for entire bridges and there is not an INCLUDE_ALL DRHD.
This is what is causing the failure - no match. It seems odd to me that there
are so few entries.
Is there anything i can do to work on that problem further?

Regards

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Did you mention in an earlier email that you have the latest BIOS for
this system?

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Friday, October 31, 2008 12:20 PM
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35
-VT-DIssue

Ross Philipson schrieb:
> Attached is a printout of your DMAR tables. Focusing on the following
failures that occurred later in the boot process:
> 
> (XEN) domain_context_mapping: can''t find drhd for device (bus=0
dev=1f
func=2)
> (XEN) [VT-D]iommu.c:1545:d0 intel_iommu_add_device: context mapping
failed
> (XEN) domain_context_mapping: can''t find drhd for device (bus=0
dev=1f
func=5)
> (XEN) [VT-D]iommu.c:1545:d0 intel_iommu_add_device: context mapping
failed
> 
> The DMAR only has two DRHD with 1 and 3 scope entries respectively for
device endpoints (covering devices 00:1b.0, 00:03.0, 00:03.2, 00:03.3).
There are no scope entries for entire bridges and there is not an
INCLUDE_ALL DRHD. This is what is causing the failure - no match. It
seems odd to me that there are so few entries.

Is there anything i can do to work on that problem further?

Regards

-- 
Stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> Did you mention in an earlier email that you have the latest BIOS for
> this system?
The latest available from the lenovo webpage yes.

-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

That is a bummer. I am not really sure where to go with it. If there is
no DRHD for that device (or a number of other ones so it would seem)
then there is no way to setup DMA remapping for those devices. Maybe
someone else on the list has some ideas.

Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Friday, October 31, 2008 1:26 PM
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35
-VT-DIssue

Ross Philipson schrieb:
> Did you mention in an earlier email that you have the latest BIOS for
> this system?
The latest available from the lenovo webpage yes.

-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> That is a bummer. I am not really sure where to go with it. If there is
> no DRHD for that device (or a number of other ones so it would seem)
> then there is no way to setup DMA remapping for those devices. Maybe
> someone else on the list has some ideas.
So if i understand you correctly, there is some problem with the PEG
graphic card in my case? Another graphic card might solve that problem?

Regards

-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

The dump of the DMAR on your system showed the individual remapping
hardware units and which PCI devices they handled. Here is an excerpt
(my comment prefixed with a **):

DMA Remapping Hardware Unit Definition (DRHD) Structure #1
Type:           0x0000 (ACPI_DMAR_DRHD)
Length:         0x0018
Flags:          0x00  -- INCLUDE_ALL = no 
Reserved:       0x00
Segment Number: 0x0000
Register Base:  0xfed90000
    Device Scope Structure #1 ** This unit has one device scope entry
    =========================    Type:           0x01 (ACPI_DEV_ENDPOINT)
    Length:         0x08
    Reserved:       00 00
    Enumeration ID: 0x00 - Reserved
    Start Bus Num:  0x00
    Path Depth = 1, Path Entries:
       -- Device: 0x1b Function: 0x00 ** This unit support this
device/function on bus 0

DMA Remapping Hardware Unit Definition (DRHD) Structure #2
Type:           0x0000 (ACPI_DMAR_DRHD)
Length:         0x0028
Flags:          0x00  -- INCLUDE_ALL = no
Reserved:       0x00
Segment Number: 0x0000
Register Base:  0xfed92000
    Device Scope Structure #1 ** This unit has 3 device scope entries
each supporting one a different function on
device 3 bus 0
    =========================    Type:           0x01 (ACPI_DEV_ENDPOINT)
    Length:         0x08
    Reserved:       00 00
    Enumeration ID: 0x00 - Reserved
    Start Bus Num:  0x00
    Path Depth = 1, Path Entries:
       -- Device: 0x03 Function: 0x00 ** one function
    Device Scope Structure #2 ** another scope
    =========================    Type:           0x01 (ACPI_DEV_ENDPOINT)
    Length:         0x08
    Reserved:       00 00
    Enumeration ID: 0x00 - Reserved
    Start Bus Num:  0x00
    Path Depth = 1, Path Entries:
       -- Device: 0x03 Function: 0x02 ** another function
    Device Scope Structure #3 ** another scope
    =========================    Type:           0x01 (ACPI_DEV_ENDPOINT)
    Length:         0x08
    Reserved:       00 00
    Enumeration ID: 0x00 - Reserved
    Start Bus Num:  0x00
    Path Depth = 1, Path Entries:
       -- Device: 0x03 Function: 0x03 ** another function

Normally there is another DMA Remapping Hardware Unit Definition (DRHD)
that has the INCLUDE_ALL flag set and handles all devices that are not
specifically handled by DRHDs like the ones above (at least that has
been my experience though I haven''t seen all that many machines). Your
system does not seem to have that. So there is no real way to setup DMA
remapping for any of the devices on your system that is not explicitly
represented above. My gut feeling is that it is a BIOS problem - the
ACPI tables are incorrect/incomplete but I cannot be sure.

If you use lspci you can see the (small) set of PCI devices that the
VT-d hardware seems to have DRHD units for. You might also want to
contact Lenovo and see if this is a known issue.

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Stefan Bauer
Sent: Friday, October 31, 2008 3:43 PM
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] iommu: mapping reserved region failed - Q35
-VT-DIssue

Ross Philipson schrieb:
> That is a bummer. I am not really sure where to go with it. If there
is
> no DRHD for that device (or a number of other ones so it would seem)
> then there is no way to setup DMA remapping for those devices. Maybe
> someone else on the list has some ideas.
So if i understand you correctly, there is some problem with the PEG
graphic card in my case? Another graphic card might solve that problem?

Regards

-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson schrieb:
> If you use lspci you can see the (small) set of PCI devices that the
> VT-d hardware seems to have DRHD units for. You might also want to
> contact Lenovo and see if this is a known issue.
I''m still trying to get in contact with lenovo to work that problem
out. Will keep you up to date with news from that side.

Regards
-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Stefan Bauer wrote:
> Ross Philipson schrieb:
>> That is a bummer. I am not really sure where to go with it. If there
>> is no DRHD for that device (or a number of other ones so it would
>> seem) then there is no way to setup DMA remapping for those devices.
>> Maybe someone else on the list has some ideas.
> 
> So if i understand you correctly, there is some problem with the PEG
> graphic card in my case? Another graphic card might solve that
> problem? 
> 
You don''t use PEG, so it is not related to it. The dump of the DMAR on
your system shows definitely BIOS doesn''t setup ACPI correctly for
DMAR. Pls contact your vendor.

Regards,
Weidong


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Han, Weidong schrieb:
> Stefan Bauer wrote:
>> Ross Philipson schrieb:
>>> That is a bummer. I am not really sure where to go with it. If
there
>>> is no DRHD for that device (or a number of other ones so it would
>>> seem) then there is no way to setup DMA remapping for those
devices.
>>> Maybe someone else on the list has some ideas.
>> So if i understand you correctly, there is some problem with the PEG
>> graphic card in my case? Another graphic card might solve that
>> problem? 
>>
> 
> You don''t use PEG, so it is not related to it. The dump of the
DMAR on your system shows definitely BIOS doesn''t setup ACPI correctly
for DMAR. Pls contact your vendor.
My intention was to passthrough my PCI-E graphic card to the HVM domU,
so i definitely use a PEG-Card. I''m awaiting response from lenovo, but
as Rajesh mentioned, your ways to contact lenovo might be ways better
than mine.

Regards

-- 
stefan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel