thr3ads.net - Xen devel - [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU [Mar 2009]

If this information is useful, please help other people find it:
Share via:

Tim Moore

2009-Mar-30 13:27 UTC

[Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi

I have been testing xen-unstable.hg (29/03/09) over the past few days .. trying
to enable passthrough for VGA.

I am using Debian 5.0 (lenny) x64 and compiling from source each time. Tried
both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits ...

Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T Deluxe)
with Core i7 920 CPU.
(had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and remove
the DMAR bail because of bad RMRR tables.
I also have ATI X300 PCIe for Dom0 Console.

Using Xen parameter: iommu=passthrough,pv
Using Dom) parameter: pciback.hide=(''...'')

I compiled pciback as embedded in the kernel and I have tried many different
scenarios:

1) Nvidia Primary (hidden with pciback.hide= kernel param) (no ATI card
installed)
2) ATI Primary, with Nvidia hidden (with pciback.hide= kernel param)
3) all scenarios with pciback as module and using echo devn >
/sys/bus/pci/drivers/pciback/new_slot and bind

In all scenarios I can successfully see the correct PCI device in the DomU(s), I
have tried primarily Windows XP 32bit SP3 and also: linux-2.6.18.8 and
linux-2.6.27.5 PV and non PV kernels as DomU and (although I didn''t
load a driver) the lspci -vvv showed the devices.

Now, I progressed to installing drivers under the Win32 DomU against the Nvidia
card, windows correctly identified the device and installed the driver! (latest
from nvidia.com). As we all know - Windows wanted a reboot- beforehand a quick
check of the device and it has a Yellow exclamation indicating reboot required
and resources unavailable.
Reboot and XP starts up ... BSOD .. nv4 driver fails .. :(

Tried with the ATI X300 Primary (no NVidia connected) and went through the same
steps. Windows XP now boots without crash although the ATI device still shows a
yellow exclamation and insufficient resources.

To summarise, pciback functions as expected for PCIe in Dom0 and DomU sees the
device. All tools work fine (xm pci-attach / pci-detach / pci-list) and devid
show and are managed correctly.

I have read all that there is on this subject and have come to the conclusion
that this is a problem with qemu-dm and the memory mapping for the new Video
device, the Cirrus or stdvga card is always present which I believe may be
causing the problem. I have tried nographics=1 but qemu-dm still maps the Video
BIOS.

Then I noticed that qemu-dm has the new options for "-vga none" .. so
I built a wrapper script for qemu-dm to launch with this param. Unfortuntately
the DomU doesn''t startup and the qemu log does not show any mention of
the Video memory ! so I believe the switch works, but the DomU then fails to
boot :(

To solve this and make it usable I think some development of qemu with regards
to the Cirrus VGA and memory management when using more than 1 vga controller is
necessary in order to remap the Cirrus or secondary vga adapters in memory
correctly. (its also difficult to configure a Windows machine for a second vga
when the primary is disabled lol.)

Also, I have run strace using a qemu-dm wrapper (very verbose) and analysed the
results -nothing seems obvious, but i`m not an expert by far ....

Windows has been able to support multiple VGA cards for a long time; I assume
that MS is remapping these somehow?

I have read in 2 different places that this has worked for others before
(without the legacy patch for igfx) and with xen-unstable.hg .. why does it not
work for me ?

Any help / direction to further debug would be helpful .. I think many people
would like to get this one working once and for all!

Tim


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Mar-30 13:35 UTC

head link

[Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi

I have been testing xen-unstable.hg (29/03/09) over the past few days .. trying
to enable passthrough for VGA.

I am using Debian 5.0 (lenny) x64 and compiling from source each time. Tried
both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits ...

Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T Deluxe)
with Core i7 920 CPU.
(had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and remove
the DMAR bail because of bad RMRR tables.
I also have ATI X300 PCIe for Dom0 Console.

Using Xen parameter: iommu=passthrough,pv
Using Dom) parameter: pciback.hide=(''...'')

I compiled pciback as embedded in the kernel and I have tried many different
scenarios:

1) Nvidia Primary (hidden with pciback.hide= kernel param) (no ATI card
installed)
2) ATI Primary, with Nvidia hidden (with pciback.hide= kernel param)
3) all scenarios with pciback as module and using echo devn >
/sys/bus/pci/drivers/pciback/new_slot and bind

In all scenarios I can successfully see the correct PCI device in the DomU(s), I
have tried primarily Windows XP 32bit SP3 and also: linux-2.6.18.8 and
linux-2.6.27.5 PV and non PV kernels as DomU and (although I didn''t
load a driver) the lspci -vvv showed the devices.

Now, I progressed to installing drivers under the Win32 DomU against the Nvidia
card, windows correctly identified the device and installed the driver! (latest
from nvidia.com). As we all know - Windows wanted a reboot- beforehand a quick
check of the device and it has a Yellow exclamation indicating reboot required
and resources unavailable.
Reboot and XP starts up ... BSOD .. nv4 driver fails .. :(

Tried with the ATI X300 Primary (no NVidia connected) and went through the same
steps. Windows XP now boots without crash although the ATI device still shows a
yellow exclamation and insufficient resources.

To summarise, pciback functions as expected for PCIe in Dom0 and DomU sees the
device. All tools work fine (xm pci-attach / pci-detach / pci-list) and devid
show and are managed correctly.

I have read all that there is on this subject and have come to the conclusion
that this is a problem with qemu-dm and the memory mapping for the new Video
device, the Cirrus or stdvga card is always present which I believe may be
causing the problem. I have tried nographics=1 but qemu-dm still maps the Video
BIOS.

Then I noticed that qemu-dm has the new options for "-vga none" .. so
I built a wrapper script for qemu-dm to launch with this param. Unfortuntately
the DomU doesn''t startup and the qemu log does not show any mention of
the Video memory ! so I believe the switch works, but the DomU then fails to
boot :(

To solve this and make it usable I think some development of qemu with regards
to the Cirrus VGA and memory management when using more than 1 vga controller is
necessary in order to remap the Cirrus or secondary vga adapters in memory
correctly. (its also difficult to configure a Windows machine for a second vga
when the primary is disabled lol.)

Also, I have run strace using a qemu-dm wrapper (very verbose) and analysed the
results -nothing seems obvious, but i`m not an expert by far ....

Windows has been able to support multiple VGA cards for a long time; I assume
that MS is remapping these somehow?

I have read in 2 different places that this has worked for others before
(without the legacy patch for igfx) and with xen-unstable.hg .. why does it not
work for me ?

Any help / direction to further debug would be helpful .. I think many people
would like to get this one working once and for all!

Tim


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Alexia Benington

2009-Mar-30 14:03 UTC

head link

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi,

I''ve been trying to get this to work for some months, albeit not
full-time. I concur that many would like to see it working. Some have
posted that they managed to get it to work correctly, however, they
almost always never reply to queries on how they got it to work, or
their observations.

I did manage to get rid of the yellow exclamation mark. You''ll need to
disable the emulated VGA bios, pretty much as described here
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.

Even after doing so, some of the registers are not directly passed
through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg01131.html.
I''m not sure if that may affect anything.

I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290, nVidia
GF9500, ATI Radion HD 2600XT. The best results I get are when using
the onboard igfx as the primary, which is also being passed through.

There is also a writeup which you might want to take a look
http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.

I''m also trying to get DomU''s output to the display.

-Alex

2009/3/30 Tim Moore <timothy.moore@expidas.net>:> Hi
>
>
>
> I have been testing xen-unstable.hg (29/03/09) over the past few days ..
> trying to enable passthrough for VGA.
>
>
>
> I am using Debian 5.0 (lenny) x64 and compiling from source each time.
Tried
> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits …
>
>
>
> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T
> Deluxe) with Core i7 920 CPU.
>
> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and
> remove the DMAR bail because of bad RMRR tables.
>
> I also have ATI X300 PCIe for Dom0 Console.
>
>
>
> Using Xen parameter: iommu=passthrough,pv
>
> Using Dom) parameter: pciback.hide=(‘…’)
>
>
>
> I compiled pciback as embedded in the kernel and I have tried many
different
> scenarios:
>
>
>
> 1) Nvidia Primary (hidden with pciback.hide= kernel param) (no ATI card
> installed)
>
> 2) ATI Primary, with Nvidia hidden (with pciback.hide= kernel param)
>
> 3) all scenarios with pciback as module and using echo devn >
> /sys/bus/pci/drivers/pciback/new_slot and bind
>
>
>
> In all scenarios I can successfully see the correct PCI device in the
> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
> (although I didn’t load a driver) the lspci –vvv showed the devices.
>
>
>
> Now, I progressed to installing drivers under the Win32 DomU against the
> Nvidia card, windows correctly identified the device and installed the
> driver! (latest from nvidia.com). As we all know – Windows wanted a reboot-
> beforehand a quick check of the device and it has a Yellow exclamation
> indicating reboot required and resources unavailable.
>
> Reboot and XP starts up … BSOD .. nv4 driver fails .. L
>
>
>
> Tried with the ATI X300 Primary (no NVidia connected) and went through the
> same steps. Windows XP now boots without crash although the ATI device
still
> shows a yellow exclamation and insufficient resources.
>
>
>
> To summarise, pciback functions as expected for PCIe in Dom0 and DomU sees
> the device. All tools work fine (xm pci-attach / pci-detach / pci-list) and
> devid show and are managed correctly.
>
>
>
> I have read all that there is on this subject and have come to the
> conclusion that this is a problem with qemu-dm and the memory mapping for
> the new Video device, the Cirrus or stdvga card is always present which I
> believe may be causing the problem. I have tried nographics=1 but qemu-dm
> still maps the Video BIOS.
>
>
>
> Then I noticed that qemu-dm has the new options for “-vga none” .. so I
> built a wrapper script for qemu-dm to launch with this param.
Unfortuntately
> the DomU doesn’t startup and the qemu log does not show any mention of the
> Video memory ! so I believe the switch works, but the DomU then fails to
> boot L
>
>
>
> To solve this and make it usable I think some development of qemu with
> regards to the Cirrus VGA and memory management when using more than 1 vga
> controller is necessary in order to remap the Cirrus or secondary vga
> adapters in memory correctly. (its also difficult to configure a Windows
> machine for a second vga when the primary is disabled lol.)
>
>
>
> Also, I have run strace using a qemu-dm wrapper (very verbose) and analysed
> the results –nothing seems obvious, but i`m not an expert by far ….
>
>
>
> Windows has been able to support multiple VGA cards for a long time; I
> assume that MS is remapping these somehow?
>
>
>
> I have read in 2 different places that this has worked for others before
> (without the legacy patch for igfx) and with xen-unstable.hg .. why does it
> not work for me ?
>
>
>
> Any help / direction to further debug would be helpful .. I think many
> people would like to get this one working once and for all!
>
>
>
> Tim
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Mar-30 14:30 UTC

head link

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi Alex,

When you say you had success with the onboard igfx, how much success? Have you
ever had a passthrough VGA display from a DomU ?

I have also seen the write-up too although a lot of work has been done on the
VT-d support lately and I`m hoping to raise this again on the developer list and
start to document some of the problems/issues so as we can find a way do solve
it !

Thanks for the reply! It still leaves this issue unsolved though ;(

Tim

________________________________
Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
        <any> DomU
From: Alexia Benington <alexbenington@gmail.com>
To: Tim Moore <timothy.moore@expidas.net>
Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>

Hi,

I''ve been trying to get this to work for some months, albeit not
full-time. I concur that many would like to see it working. Some have
posted that they managed to get it to work correctly, however, they
almost always never reply to queries on how they got it to work, or
their observations.

I did manage to get rid of the yellow exclamation mark. You''ll need to
disable the emulated VGA bios, pretty much as described here
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.

Even after doing so, some of the registers are not directly passed
through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg01131.html.
I''m not sure if that may affect anything.

I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290, nVidia
GF9500, ATI Radion HD 2600XT. The best results I get are when using
the onboard igfx as the primary, which is also being passed through.

There is also a writeup which you might want to take a look
http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.

I''m also trying to get DomU''s output to the display.

-Alex

2009/3/30 Tim Moore <timothy.moore@expidas.net>:> Hi
>
>
>
> I have been testing xen-unstable.hg (29/03/09) over the past few days ..
> trying to enable passthrough for VGA.
>
>
>
> I am using Debian 5.0 (lenny) x64 and compiling from source each time.
Tried
> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>
>
>
> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T
> Deluxe) with Core i7 920 CPU.
>
> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and
> remove the DMAR bail because of bad RMRR tables.
>
> I also have ATI X300 PCIe for Dom0 Console.
>
>
>
> Using Xen parameter: iommu=3Dpassthrough,pv
>
> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>
>
>
> I compiled pciback as embedded in the kernel and I have tried many
different
> scenarios:
>
>
>
> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no ATI card
> installed)
>
> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel param)
>
> 3) all scenarios with pciback as module and using echo devn >
> /sys/bus/pci/drivers/pciback/new_slot and bind
>
>
>
> In all scenarios I can successfully see the correct PCI device in the
> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
> (although I didn=92t load a driver) the lspci =96vvv showed the devices.
>
>
>
> Now, I progressed to installing drivers under the Win32 DomU against the
> Nvidia card, windows correctly identified the device and installed the
> driver! (latest from nvidia.com). As we all know =96 Windows wanted a
reboot-
> beforehand a quick check of the device and it has a Yellow exclamation
> indicating reboot required and resources unavailable.
>
> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>
>
>
> Tried with the ATI X300 Primary (no NVidia connected) and went through the
> same steps. Windows XP now boots without crash although the ATI device
still
> shows a yellow exclamation and insufficient resources.
>
>
>
> To summarise, pciback functions as expected for PCIe in Dom0 and DomU sees
> the device. All tools work fine (xm pci-attach / pci-detach / pci-list) and
> devid show and are managed correctly.
>
>
>
> I have read all that there is on this subject and have come to the
> conclusion that this is a problem with qemu-dm and the memory mapping for
> the new Video device, the Cirrus or stdvga card is always present which I
> believe may be causing the problem. I have tried nographics=3D1 but qemu-dm
> still maps the Video BIOS.
>
>
>
> Then I noticed that qemu-dm has the new options for =93-vga none=94 .. so I
> built a wrapper script for qemu-dm to launch with this param.
Unfortuntately
> the DomU doesn=92t startup and the qemu log does not show any mention of
the
> Video memory ! so I believe the switch works, but the DomU then fails to
> boot L
>
>
>
> To solve this and make it usable I think some development of qemu with
> regards to the Cirrus VGA and memory management when using more than 1 vga
> controller is necessary in order to remap the Cirrus or secondary vga
> adapters in memory correctly. (its also difficult to configure a Windows
> machine for a second vga when the primary is disabled lol.)
>
>
>
> Also, I have run strace using a qemu-dm wrapper (very verbose) and analysed
> the results =96nothing seems obvious, but i`m not an expert by far =85.
>
>
>
> Windows has been able to support multiple VGA cards for a long time; I
> assume that MS is remapping these somehow?
>
>
>
> I have read in 2 different places that this has worked for others before
> (without the legacy patch for igfx) and with xen-unstable.hg .. why does it
> not work for me ?
>
>
>
> Any help / direction to further debug would be helpful .. I think many
> people would like to get this one working once and for all!
>
>
>
> Tim
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jean Guyader

2009-Mar-30 14:37 UTC

head link

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi Tim,

Most of the time the problam doesn''t come from the VT-d code but from
the graphic card device drivers. Most of them doesn''t really like PCI
Bar remapping and try to use the old one.
For instance the intel igfx has never been designed to be used as a
secondary display adapter or even with another Host bridge than the
one on the motherboard.

Jean

2009/3/30 Tim Moore <timothy.moore@expidas.net>:> Hi Alex,
>
>
>
> When you say you had success with the onboard igfx, how much success? Have
> you ever had a passthrough VGA display from a DomU ?
>
>
>
> I have also seen the write-up too although a lot of work has been done on
> the VT-d support lately and I`m hoping to raise this again on the developer
> list and start to document some of the problems/issues so as we can find a
> way do solve it !
>
>
>
> Thanks for the reply! It still leaves this issue unsolved though ;(
>
>
>
> Tim
>
>
>
> ________________________________
>
> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough
to
>
>         <any> DomU
>
> From: Alexia Benington <alexbenington@gmail.com>
>
> To: Tim Moore <timothy.moore@expidas.net>
>
> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>
>
>
> Hi,
>
>
>
> I''ve been trying to get this to work for some months, albeit not
>
> full-time. I concur that many would like to see it working. Some have
>
> posted that they managed to get it to work correctly, however, they
>
> almost always never reply to queries on how they got it to work, or
>
> their observations.
>
>
>
> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>
> disable the emulated VGA bios, pretty much as described here
>
> http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>
>
>
> Even after doing so, some of the registers are not directly passed
>
> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>
> 01131.html.
>
> I''m not sure if that may affect anything.
>
>
>
> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>
> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>
> the onboard igfx as the primary, which is also being passed through.
>
>
>
> There is also a writeup which you might want to take a look
>
> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>
>
>
> I''m also trying to get DomU''s output to the display.
>
>
>
> -Alex
>
>
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>
>> Hi
>
>>
>
>>
>
>>
>
>> I have been testing xen-unstable.hg (29/03/09) over the past few days
..
>
>> trying to enable passthrough for VGA.
>
>>
>
>>
>
>>
>
>> I am using Debian 5.0 (lenny) x64 and compiling from source each time.
Tr>
> ied
>
>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>
>>
>
>>
>
>>
>
>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T
>
>> Deluxe) with Core i7 920 CPU.
>
>>
>
>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and
>
>> remove the DMAR bail because of bad RMRR tables.
>
>>
>
>> I also have ATI X300 PCIe for Dom0 Console.
>
>>
>
>>
>
>>
>
>> Using Xen parameter: iommu=3Dpassthrough,pv
>
>>
>
>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>
>>
>
>>
>
>>
>
>> I compiled pciback as embedded in the kernel and I have tried many
differ>
> ent
>
>> scenarios:
>
>>
>
>>
>
>>
>
>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no ATI
card
>
>> installed)
>
>>
>
>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel param)
>
>>
>
>> 3) all scenarios with pciback as module and using echo devn >
>
>> /sys/bus/pci/drivers/pciback/new_slot and bind
>
>>
>
>>
>
>>
>
>> In all scenarios I can successfully see the correct PCI device in the
>
>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>
>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>
>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>
>>
>
>>
>
>>
>
>> Now, I progressed to installing drivers under the Win32 DomU against
the
>
>> Nvidia card, windows correctly identified the device and installed the
>
>> driver! (latest from nvidia.com). As we all know =96 Windows wanted a
reb>
> oot-
>
>> beforehand a quick check of the device and it has a Yellow exclamation
>
>> indicating reboot required and resources unavailable.
>
>>
>
>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>
>>
>
>>
>
>>
>
>> Tried with the ATI X300 Primary (no NVidia connected) and went through
th>
> e
>
>> same steps. Windows XP now boots without crash although the ATI device
st>
> ill
>
>> shows a yellow exclamation and insufficient resources.
>
>>
>
>>
>
>>
>
>> To summarise, pciback functions as expected for PCIe in Dom0 and DomU
see>
> s
>
>> the device. All tools work fine (xm pci-attach / pci-detach / pci-list)
a>
> nd
>
>> devid show and are managed correctly.
>
>>
>
>>
>
>>
>
>> I have read all that there is on this subject and have come to the
>
>> conclusion that this is a problem with qemu-dm and the memory mapping
for
>
>> the new Video device, the Cirrus or stdvga card is always present which
I
>
>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->
> dm
>
>> still maps the Video BIOS.
>
>>
>
>>
>
>>
>
>> Then I noticed that qemu-dm has the new options for =93-vga none=94 ..
so>
>  I
>
>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>
> ely
>
>> the DomU doesn=92t startup and the qemu log does not show any mention
of >
> the
>
>> Video memory ! so I believe the switch works, but the DomU then fails
to
>
>> boot L
>
>>
>
>>
>
>>
>
>> To solve this and make it usable I think some development of qemu with
>
>> regards to the Cirrus VGA and memory management when using more than 1
vg>
> a
>
>> controller is necessary in order to remap the Cirrus or secondary vga
>
>> adapters in memory correctly. (its also difficult to configure a
Windows
>
>> machine for a second vga when the primary is disabled lol.)
>
>>
>
>>
>
>>
>
>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>
> ed
>
>> the results =96nothing seems obvious, but i`m not an expert by far =85.
>
>>
>
>>
>
>>
>
>> Windows has been able to support multiple VGA cards for a long time; I
>
>> assume that MS is remapping these somehow?
>
>>
>
>>
>
>>
>
>> I have read in 2 different places that this has worked for others
before
>
>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >
> it
>
>> not work for me ?
>
>>
>
>>
>
>>
>
>> Any help / direction to further debug would be helpful .. I think many
>
>> people would like to get this one working once and for all!
>
>>
>
>>
>
>>
>
>> Tim
>
>>
>
>> _______________________________________________
>
>> Xen-devel mailing list
>
>> Xen-devel@lists.xensource.com
>
>> http://lists.xensource.com/xen-devel
>
>>
>
>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>


-- 
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Mar-30 14:42 UTC

head link

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi Jean,

I can understand the issue with the graphics card drivers, and I can see how
that could be able problem. I am using a very modern nvidia and ati card for
testing, Alex is also using a couple of other different varieties so we are able
to test with a few different scenarios...

How would this work under an MS kernel? these manufacturers do support dual
cards and I have also tested my ATI/Nvidia host with a Windows OS and both cards
are loaded and work fine. This says to me that the drivers can deal with PCI
remapping of some sorts?

Cheers,
Tim

-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@gmail.com] 
Sent: 30 March 2009 15:38
To: Tim Moore
Cc: xen-devel@lists.xensource.com
Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

Hi Tim,

Most of the time the problam doesn''t come from the VT-d code but from
the graphic card device drivers. Most of them doesn''t really like PCI
Bar remapping and try to use the old one.
For instance the intel igfx has never been designed to be used as a
secondary display adapter or even with another Host bridge than the
one on the motherboard.

Jean

2009/3/30 Tim Moore <timothy.moore@expidas.net>:> Hi Alex,
>
>
>
> When you say you had success with the onboard igfx, how much success? Have
> you ever had a passthrough VGA display from a DomU ?
>
>
>
> I have also seen the write-up too although a lot of work has been done on
> the VT-d support lately and I`m hoping to raise this again on the developer
> list and start to document some of the problems/issues so as we can find a
> way do solve it !
>
>
>
> Thanks for the reply! It still leaves this issue unsolved though ;(
>
>
>
> Tim
>
>
>
> ________________________________
>
> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough
to
>
>         <any> DomU
>
> From: Alexia Benington <alexbenington@gmail.com>
>
> To: Tim Moore <timothy.moore@expidas.net>
>
> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>
>
>
> Hi,
>
>
>
> I''ve been trying to get this to work for some months, albeit not
>
> full-time. I concur that many would like to see it working. Some have
>
> posted that they managed to get it to work correctly, however, they
>
> almost always never reply to queries on how they got it to work, or
>
> their observations.
>
>
>
> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>
> disable the emulated VGA bios, pretty much as described here
>
> http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>
>
>
> Even after doing so, some of the registers are not directly passed
>
> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>
> 01131.html.
>
> I''m not sure if that may affect anything.
>
>
>
> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>
> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>
> the onboard igfx as the primary, which is also being passed through.
>
>
>
> There is also a writeup which you might want to take a look
>
> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>
>
>
> I''m also trying to get DomU''s output to the display.
>
>
>
> -Alex
>
>
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>
>> Hi
>
>>
>
>>
>
>>
>
>> I have been testing xen-unstable.hg (29/03/09) over the past few days
..
>
>> trying to enable passthrough for VGA.
>
>>
>
>>
>
>>
>
>> I am using Debian 5.0 (lenny) x64 and compiling from source each time.
Tr>
> ied
>
>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>
>>
>
>>
>
>>
>
>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T
>
>> Deluxe) with Core i7 920 CPU.
>
>>
>
>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and
>
>> remove the DMAR bail because of bad RMRR tables.
>
>>
>
>> I also have ATI X300 PCIe for Dom0 Console.
>
>>
>
>>
>
>>
>
>> Using Xen parameter: iommu=3Dpassthrough,pv
>
>>
>
>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>
>>
>
>>
>
>>
>
>> I compiled pciback as embedded in the kernel and I have tried many
differ>
> ent
>
>> scenarios:
>
>>
>
>>
>
>>
>
>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no ATI
card
>
>> installed)
>
>>
>
>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel param)
>
>>
>
>> 3) all scenarios with pciback as module and using echo devn >
>
>> /sys/bus/pci/drivers/pciback/new_slot and bind
>
>>
>
>>
>
>>
>
>> In all scenarios I can successfully see the correct PCI device in the
>
>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>
>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>
>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>
>>
>
>>
>
>>
>
>> Now, I progressed to installing drivers under the Win32 DomU against
the
>
>> Nvidia card, windows correctly identified the device and installed the
>
>> driver! (latest from nvidia.com). As we all know =96 Windows wanted a
reb>
> oot-
>
>> beforehand a quick check of the device and it has a Yellow exclamation
>
>> indicating reboot required and resources unavailable.
>
>>
>
>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>
>>
>
>>
>
>>
>
>> Tried with the ATI X300 Primary (no NVidia connected) and went through
th>
> e
>
>> same steps. Windows XP now boots without crash although the ATI device
st>
> ill
>
>> shows a yellow exclamation and insufficient resources.
>
>>
>
>>
>
>>
>
>> To summarise, pciback functions as expected for PCIe in Dom0 and DomU
see>
> s
>
>> the device. All tools work fine (xm pci-attach / pci-detach / pci-list)
a>
> nd
>
>> devid show and are managed correctly.
>
>>
>
>>
>
>>
>
>> I have read all that there is on this subject and have come to the
>
>> conclusion that this is a problem with qemu-dm and the memory mapping
for
>
>> the new Video device, the Cirrus or stdvga card is always present which
I
>
>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->
> dm
>
>> still maps the Video BIOS.
>
>>
>
>>
>
>>
>
>> Then I noticed that qemu-dm has the new options for =93-vga none=94 ..
so>
>  I
>
>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>
> ely
>
>> the DomU doesn=92t startup and the qemu log does not show any mention
of >
> the
>
>> Video memory ! so I believe the switch works, but the DomU then fails
to
>
>> boot L
>
>>
>
>>
>
>>
>
>> To solve this and make it usable I think some development of qemu with
>
>> regards to the Cirrus VGA and memory management when using more than 1
vg>
> a
>
>> controller is necessary in order to remap the Cirrus or secondary vga
>
>> adapters in memory correctly. (its also difficult to configure a
Windows
>
>> machine for a second vga when the primary is disabled lol.)
>
>>
>
>>
>
>>
>
>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>
> ed
>
>> the results =96nothing seems obvious, but i`m not an expert by far =85.
>
>>
>
>>
>
>>
>
>> Windows has been able to support multiple VGA cards for a long time; I
>
>> assume that MS is remapping these somehow?
>
>>
>
>>
>
>>
>
>> I have read in 2 different places that this has worked for others
before
>
>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >
> it
>
>> not work for me ?
>
>>
>
>>
>
>>
>
>> Any help / direction to further debug would be helpful .. I think many
>
>> people would like to get this one working once and for all!
>
>>
>
>>
>
>>
>
>> Tim
>
>>
>
>> _______________________________________________
>
>> Xen-devel mailing list
>
>> Xen-devel@lists.xensource.com
>
>> http://lists.xensource.com/xen-devel
>
>>
>
>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>


-- 
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Jean Guyader

2009-Mar-30 14:56 UTC

head link

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi Tim,

In the native case there is only one mapping, there isn''t any
remapping ones the bios has been executed. It is different in the hvm
scenario.

Anyway, the nvidia graphic card should works fine as a secondary, side
by side with the cirrus. I don''t know if this card support
FLR''s but
that would be good to have a look.

Jean

2009/3/30 Tim Moore <timothy.moore@expidas.net>:>
> Hi Jean,
>
> I can understand the issue with the graphics card drivers, and I can see
how that could be able problem. I am using a very modern nvidia and ati card for
testing, Alex is also using a couple of other different varieties so we are able
to test with a few different scenarios...
>
> How would this work under an MS kernel? these manufacturers do support dual
cards and I have also tested my ATI/Nvidia host with a Windows OS and both cards
are loaded and work fine. This says to me that the drivers can deal with PCI
remapping of some sorts?
>
> Cheers,
> Tim
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@gmail.com]
> Sent: 30 March 2009 15:38
> To: Tim Moore
> Cc: xen-devel@lists.xensource.com
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>
> Hi Tim,
>
> Most of the time the problam doesn''t come from the VT-d code but
from
> the graphic card device drivers. Most of them doesn''t really like
PCI
> Bar remapping and try to use the old one.
> For instance the intel igfx has never been designed to be used as a
> secondary display adapter or even with another Host bridge than the
> one on the motherboard.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>> Hi Alex,
>>
>>
>>
>> When you say you had success with the onboard igfx, how much success?
Have
>> you ever had a passthrough VGA display from a DomU ?
>>
>>
>>
>> I have also seen the write-up too although a lot of work has been done
on
>> the VT-d support lately and I`m hoping to raise this again on the
developer
>> list and start to document some of the problems/issues so as we can
find a
>> way do solve it !
>>
>>
>>
>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>
>>
>>
>> Tim
>>
>>
>>
>> ________________________________
>>
>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>
>>         <any> DomU
>>
>> From: Alexia Benington <alexbenington@gmail.com>
>>
>> To: Tim Moore <timothy.moore@expidas.net>
>>
>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>
>>
>>
>> Hi,
>>
>>
>>
>> I''ve been trying to get this to work for some months, albeit
not
>>
>> full-time. I concur that many would like to see it working. Some have
>>
>> posted that they managed to get it to work correctly, however, they
>>
>> almost always never reply to queries on how they got it to work, or
>>
>> their observations.
>>
>>
>>
>> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>>
>> disable the emulated VGA bios, pretty much as described here
>>
>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>
>>
>>
>> Even after doing so, some of the registers are not directly passed
>>
>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>
>> 01131.html.
>>
>> I''m not sure if that may affect anything.
>>
>>
>>
>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>>
>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>
>> the onboard igfx as the primary, which is also being passed through.
>>
>>
>>
>> There is also a writeup which you might want to take a look
>>
>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>
>>
>>
>> I''m also trying to get DomU''s output to the display.
>>
>>
>>
>> -Alex
>>
>>
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>>> Hi
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have been testing xen-unstable.hg (29/03/09) over the past few
days ..
>>
>>> trying to enable passthrough for VGA.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I am using Debian 5.0 (lenny) x64 and compiling from source each
time. Tr>>
>> ied
>>
>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>
>>>
>>
>>>
>>
>>>
>>
>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS
P6T
>>
>>> Deluxe) with Core i7 920 CPU.
>>
>>>
>>
>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388)
and
>>
>>> remove the DMAR bail because of bad RMRR tables.
>>
>>>
>>
>>> I also have ATI X300 PCIe for Dom0 Console.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>
>>>
>>
>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>
>>>
>>
>>>
>>
>>>
>>
>>> I compiled pciback as embedded in the kernel and I have tried many
differ>>
>> ent
>>
>>> scenarios:
>>
>>>
>>
>>>
>>
>>>
>>
>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no
ATI card
>>
>>> installed)
>>
>>>
>>
>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>
>>>
>>
>>> 3) all scenarios with pciback as module and using echo devn >
>>
>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>
>>>
>>
>>>
>>
>>>
>>
>>> In all scenarios I can successfully see the correct PCI device in
the
>>
>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>
>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>>
>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>
>>> Nvidia card, windows correctly identified the device and installed
the
>>
>>> driver! (latest from nvidia.com). As we all know =96 Windows wanted
a reb>>
>> oot-
>>
>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>
>>> indicating reboot required and resources unavailable.
>>
>>>
>>
>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>
>> e
>>
>>> same steps. Windows XP now boots without crash although the ATI
device st>>
>> ill
>>
>>> shows a yellow exclamation and insufficient resources.
>>
>>>
>>
>>>
>>
>>>
>>
>>> To summarise, pciback functions as expected for PCIe in Dom0 and
DomU see>>
>> s
>>
>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>
>> nd
>>
>>> devid show and are managed correctly.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read all that there is on this subject and have come to the
>>
>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>
>>> the new Video device, the Cirrus or stdvga card is always present
which I
>>
>>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->>
>> dm
>>
>>> still maps the Video BIOS.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Then I noticed that qemu-dm has the new options for =93-vga none=94
.. so>>
>>  I
>>
>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>
>> ely
>>
>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>
>> the
>>
>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>
>>> boot L
>>
>>>
>>
>>>
>>
>>>
>>
>>> To solve this and make it usable I think some development of qemu
with
>>
>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>
>> a
>>
>>> controller is necessary in order to remap the Cirrus or secondary
vga
>>
>>> adapters in memory correctly. (its also difficult to configure a
Windows
>>
>>> machine for a second vga when the primary is disabled lol.)
>>
>>>
>>
>>>
>>
>>>
>>
>>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>>
>> ed
>>
>>> the results =96nothing seems obvious, but i`m not an expert by far
=85.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Windows has been able to support multiple VGA cards for a long
time; I
>>
>>> assume that MS is remapping these somehow?
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read in 2 different places that this has worked for others
before
>>
>>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >>
>> it
>>
>>> not work for me ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> Any help / direction to further debug would be helpful .. I think
many
>>
>>> people would like to get this one working once and for all!
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tim
>>
>>>
>>
>>> _______________________________________________
>>
>>> Xen-devel mailing list
>>
>>> Xen-devel@lists.xensource.com
>>
>>> http://lists.xensource.com/xen-devel
>>
>>>
>>
>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Mar-30 15:33 UTC

head link

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

I agree, if any I was expecting the Nvidia card to work the most ..

One interesting thing is that there is a fan on the Nvidia that spins down when
the bios is loaded, with Windows it is evident when the bios is reset/loaded -
just before the GUI is displayed.
When booting Xen the fan stays high / fast as the pciback is loaded to prevent
Dom0 taking ownership.. boot the DomU (WinXP) and the fan spins down the same as
booting a native Win32 host .. aka I assume Bios/Card is loaded/Reset here ..

Pls can you explain FLR''s ?


-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@gmail.com] 
Sent: 30 March 2009 15:57
To: Tim Moore
Cc: xen-devel@lists.xensource.com
Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

Hi Tim,

In the native case there is only one mapping, there isn''t any
remapping ones the bios has been executed. It is different in the hvm
scenario.

Anyway, the nvidia graphic card should works fine as a secondary, side
by side with the cirrus. I don''t know if this card support
FLR''s but
that would be good to have a look.

Jean

2009/3/30 Tim Moore <timothy.moore@expidas.net>:>
> Hi Jean,
>
> I can understand the issue with the graphics card drivers, and I can see
how that could be able problem. I am using a very modern nvidia and ati card for
testing, Alex is also using a couple of other different varieties so we are able
to test with a few different scenarios...
>
> How would this work under an MS kernel? these manufacturers do support dual
cards and I have also tested my ATI/Nvidia host with a Windows OS and both cards
are loaded and work fine. This says to me that the drivers can deal with PCI
remapping of some sorts?
>
> Cheers,
> Tim
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@gmail.com]
> Sent: 30 March 2009 15:38
> To: Tim Moore
> Cc: xen-devel@lists.xensource.com
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>
> Hi Tim,
>
> Most of the time the problam doesn''t come from the VT-d code but
from
> the graphic card device drivers. Most of them doesn''t really like
PCI
> Bar remapping and try to use the old one.
> For instance the intel igfx has never been designed to be used as a
> secondary display adapter or even with another Host bridge than the
> one on the motherboard.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>> Hi Alex,
>>
>>
>>
>> When you say you had success with the onboard igfx, how much success?
Have
>> you ever had a passthrough VGA display from a DomU ?
>>
>>
>>
>> I have also seen the write-up too although a lot of work has been done
on
>> the VT-d support lately and I`m hoping to raise this again on the
developer
>> list and start to document some of the problems/issues so as we can
find a
>> way do solve it !
>>
>>
>>
>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>
>>
>>
>> Tim
>>
>>
>>
>> ________________________________
>>
>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>
>>         <any> DomU
>>
>> From: Alexia Benington <alexbenington@gmail.com>
>>
>> To: Tim Moore <timothy.moore@expidas.net>
>>
>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>
>>
>>
>> Hi,
>>
>>
>>
>> I''ve been trying to get this to work for some months, albeit
not
>>
>> full-time. I concur that many would like to see it working. Some have
>>
>> posted that they managed to get it to work correctly, however, they
>>
>> almost always never reply to queries on how they got it to work, or
>>
>> their observations.
>>
>>
>>
>> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>>
>> disable the emulated VGA bios, pretty much as described here
>>
>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>
>>
>>
>> Even after doing so, some of the registers are not directly passed
>>
>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>
>> 01131.html.
>>
>> I''m not sure if that may affect anything.
>>
>>
>>
>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>>
>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>
>> the onboard igfx as the primary, which is also being passed through.
>>
>>
>>
>> There is also a writeup which you might want to take a look
>>
>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>
>>
>>
>> I''m also trying to get DomU''s output to the display.
>>
>>
>>
>> -Alex
>>
>>
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>>> Hi
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have been testing xen-unstable.hg (29/03/09) over the past few
days ..
>>
>>> trying to enable passthrough for VGA.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I am using Debian 5.0 (lenny) x64 and compiling from source each
time. Tr>>
>> ied
>>
>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>
>>>
>>
>>>
>>
>>>
>>
>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS
P6T
>>
>>> Deluxe) with Core i7 920 CPU.
>>
>>>
>>
>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388)
and
>>
>>> remove the DMAR bail because of bad RMRR tables.
>>
>>>
>>
>>> I also have ATI X300 PCIe for Dom0 Console.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>
>>>
>>
>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>
>>>
>>
>>>
>>
>>>
>>
>>> I compiled pciback as embedded in the kernel and I have tried many
differ>>
>> ent
>>
>>> scenarios:
>>
>>>
>>
>>>
>>
>>>
>>
>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no
ATI card
>>
>>> installed)
>>
>>>
>>
>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>
>>>
>>
>>> 3) all scenarios with pciback as module and using echo devn >
>>
>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>
>>>
>>
>>>
>>
>>>
>>
>>> In all scenarios I can successfully see the correct PCI device in
the
>>
>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>
>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>>
>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>
>>> Nvidia card, windows correctly identified the device and installed
the
>>
>>> driver! (latest from nvidia.com). As we all know =96 Windows wanted
a reb>>
>> oot-
>>
>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>
>>> indicating reboot required and resources unavailable.
>>
>>>
>>
>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>
>> e
>>
>>> same steps. Windows XP now boots without crash although the ATI
device st>>
>> ill
>>
>>> shows a yellow exclamation and insufficient resources.
>>
>>>
>>
>>>
>>
>>>
>>
>>> To summarise, pciback functions as expected for PCIe in Dom0 and
DomU see>>
>> s
>>
>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>
>> nd
>>
>>> devid show and are managed correctly.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read all that there is on this subject and have come to the
>>
>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>
>>> the new Video device, the Cirrus or stdvga card is always present
which I
>>
>>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->>
>> dm
>>
>>> still maps the Video BIOS.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Then I noticed that qemu-dm has the new options for =93-vga none=94
.. so>>
>>  I
>>
>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>
>> ely
>>
>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>
>> the
>>
>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>
>>> boot L
>>
>>>
>>
>>>
>>
>>>
>>
>>> To solve this and make it usable I think some development of qemu
with
>>
>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>
>> a
>>
>>> controller is necessary in order to remap the Cirrus or secondary
vga
>>
>>> adapters in memory correctly. (its also difficult to configure a
Windows
>>
>>> machine for a second vga when the primary is disabled lol.)
>>
>>>
>>
>>>
>>
>>>
>>
>>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>>
>> ed
>>
>>> the results =96nothing seems obvious, but i`m not an expert by far
=85.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Windows has been able to support multiple VGA cards for a long
time; I
>>
>>> assume that MS is remapping these somehow?
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read in 2 different places that this has worked for others
before
>>
>>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >>
>> it
>>
>>> not work for me ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> Any help / direction to further debug would be helpful .. I think
many
>>
>>> people would like to get this one working once and for all!
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tim
>>
>>>
>>
>>> _______________________________________________
>>
>>> Xen-devel mailing list
>>
>>> Xen-devel@lists.xensource.com
>>
>>> http://lists.xensource.com/xen-devel
>>
>>>
>>
>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Alexia Benington

2009-Mar-30 15:47 UTC

head link

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

[This email is either empty or too large to be displayed at this time]

Alexia Benington

2009-Mar-30 15:54 UTC

head link

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi Tim,

Apologies for the double post, I think I didn''t get the screenshot
through due to its size.

It depends on your definition of success. If it helps, I''ve attached a
screenshot of DomU''s device manager. Dom0 will, obviously, lose access
to the display, when the gfx is handed to pciback. I''ll be glad to
describe further if you require.

With nVidia Quadro, the driver sits in an idling loop. According to
the bugcheck error code, it''s waiting for some kind of response from
the gfx. I suppose this reflects Jean''s point that some drivers are
not very compatible.

I was mostly trying Jean''s patch, which he released sometime last
year, into Xen unstable. I''m also curious how some have mentioned that
they were able to get vga passthrough to work correctly without
applying the patch. It might mostly work, but you''ll still get the
emulated VGA in DomU.

-Alex

On Mon, Mar 30, 2009 at 11:33 AM, Tim Moore <timothy.moore@expidas.net>
wrote:>
> I agree, if any I was expecting the Nvidia card to work the most ..
>
> One interesting thing is that there is a fan on the Nvidia that spins down
when the bios is loaded, with Windows it is evident when the bios is
reset/loaded - just before the GUI is displayed.
> When booting Xen the fan stays high / fast as the pciback is loaded to
prevent Dom0 taking ownership.. boot the DomU (WinXP) and the fan spins down the
same as booting a native Win32 host .. aka I assume Bios/Card is loaded/Reset
here ..
>
> Pls can you explain FLR''s ?
>
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@gmail.com]
> Sent: 30 March 2009 15:57
> To: Tim Moore
> Cc: xen-devel@lists.xensource.com
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>
> Hi Tim,
>
> In the native case there is only one mapping, there isn''t any
> remapping ones the bios has been executed. It is different in the hvm
> scenario.
>
> Anyway, the nvidia graphic card should works fine as a secondary, side
> by side with the cirrus. I don''t know if this card support
FLR''s but
> that would be good to have a look.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>> Hi Jean,
>>
>> I can understand the issue with the graphics card drivers, and I can
see how that could be able problem. I am using a very modern nvidia and ati card
for testing, Alex is also using a couple of other different varieties so we are
able to test with a few different scenarios...
>>
>> How would this work under an MS kernel? these manufacturers do support
dual cards and I have also tested my ATI/Nvidia host with a Windows OS and both
cards are loaded and work fine. This says to me that the drivers can deal with
PCI remapping of some sorts?
>>
>> Cheers,
>> Tim
>>
>> -----Original Message-----
>> From: Jean Guyader [mailto:jean.guyader@gmail.com]
>> Sent: 30 March 2009 15:38
>> To: Tim Moore
>> Cc: xen-devel@lists.xensource.com
>> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>>
>> Hi Tim,
>>
>> Most of the time the problam doesn''t come from the VT-d code
but from
>> the graphic card device drivers. Most of them doesn''t really
like PCI
>> Bar remapping and try to use the old one.
>> For instance the intel igfx has never been designed to be used as a
>> secondary display adapter or even with another Host bridge than the
>> one on the motherboard.
>>
>> Jean
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>> Hi Alex,
>>>
>>>
>>>
>>> When you say you had success with the onboard igfx, how much
success? Have
>>> you ever had a passthrough VGA display from a DomU ?
>>>
>>>
>>>
>>> I have also seen the write-up too although a lot of work has been
done on
>>> the VT-d support lately and I`m hoping to raise this again on the
developer
>>> list and start to document some of the problems/issues so as we can
find a
>>> way do solve it !
>>>
>>>
>>>
>>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>>
>>>
>>>
>>> Tim
>>>
>>>
>>>
>>> ________________________________
>>>
>>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>>
>>>         <any> DomU
>>>
>>> From: Alexia Benington <alexbenington@gmail.com>
>>>
>>> To: Tim Moore <timothy.moore@expidas.net>
>>>
>>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I''ve been trying to get this to work for some months,
albeit not
>>>
>>> full-time. I concur that many would like to see it working. Some
have
>>>
>>> posted that they managed to get it to work correctly, however, they
>>>
>>> almost always never reply to queries on how they got it to work, or
>>>
>>> their observations.
>>>
>>>
>>>
>>> I did manage to get rid of the yellow exclamation mark.
You''ll need to
>>>
>>> disable the emulated VGA bios, pretty much as described here
>>>
>>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>>
>>>
>>>
>>> Even after doing so, some of the registers are not directly passed
>>>
>>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>>
>>> 01131.html.
>>>
>>> I''m not sure if that may affect anything.
>>>
>>>
>>>
>>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS
290, nVidia
>>>
>>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>>
>>> the onboard igfx as the primary, which is also being passed
through.
>>>
>>>
>>>
>>> There is also a writeup which you might want to take a look
>>>
>>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>>
>>>
>>>
>>> I''m also trying to get DomU''s output to the
display.
>>>
>>>
>>>
>>> -Alex
>>>
>>>
>>>
>>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>>
>>>> Hi
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have been testing xen-unstable.hg (29/03/09) over the past
few days ..
>>>
>>>> trying to enable passthrough for VGA.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I am using Debian 5.0 (lenny) x64 and compiling from source
each time. Tr>>>
>>> ied
>>>
>>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58
(ASUS P6T
>>>
>>>> Deluxe) with Core i7 920 CPU.
>>>
>>>>
>>>
>>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line
388) and
>>>
>>>> remove the DMAR bail because of bad RMRR tables.
>>>
>>>>
>>>
>>>> I also have ATI X300 PCIe for Dom0 Console.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>>
>>>>
>>>
>>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I compiled pciback as embedded in the kernel and I have tried
many differ>>>
>>> ent
>>>
>>>> scenarios:
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param)
(no ATI card
>>>
>>>> installed)
>>>
>>>>
>>>
>>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>>
>>>>
>>>
>>>> 3) all scenarios with pciback as module and using echo devn
>
>>>
>>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> In all scenarios I can successfully see the correct PCI device
in the
>>>
>>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>>
>>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU
and
>>>
>>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>>
>>>> Nvidia card, windows correctly identified the device and
installed the
>>>
>>>> driver! (latest from nvidia.com). As we all know =96 Windows
wanted a reb>>>
>>> oot-
>>>
>>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>>
>>>> indicating reboot required and resources unavailable.
>>>
>>>>
>>>
>>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>>
>>> e
>>>
>>>> same steps. Windows XP now boots without crash although the ATI
device st>>>
>>> ill
>>>
>>>> shows a yellow exclamation and insufficient resources.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> To summarise, pciback functions as expected for PCIe in Dom0
and DomU see>>>
>>> s
>>>
>>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>>
>>> nd
>>>
>>>> devid show and are managed correctly.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have read all that there is on this subject and have come to
the
>>>
>>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>>
>>>> the new Video device, the Cirrus or stdvga card is always
present which I
>>>
>>>> believe may be causing the problem. I have tried nographics=3D1
but qemu->>>
>>> dm
>>>
>>>> still maps the Video BIOS.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Then I noticed that qemu-dm has the new options for =93-vga
none=94 .. so>>>
>>>  I
>>>
>>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>>
>>> ely
>>>
>>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>>
>>> the
>>>
>>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>>
>>>> boot L
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> To solve this and make it usable I think some development of
qemu with
>>>
>>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>>
>>> a
>>>
>>>> controller is necessary in order to remap the Cirrus or
secondary vga
>>>
>>>> adapters in memory correctly. (its also difficult to configure
a Windows
>>>
>>>> machine for a second vga when the primary is disabled lol.)
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Also, I have run strace using a qemu-dm wrapper (very verbose)
and analys>>>
>>> ed
>>>
>>>> the results =96nothing seems obvious, but i`m not an expert by
far =85.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Windows has been able to support multiple VGA cards for a long
time; I
>>>
>>>> assume that MS is remapping these somehow?
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have read in 2 different places that this has worked for
others before
>>>
>>>> (without the legacy patch for igfx) and with xen-unstable.hg ..
why does >>>
>>> it
>>>
>>>> not work for me ?
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Any help / direction to further debug would be helpful .. I
think many
>>>
>>>> people would like to get this one working once and for all!
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Tim
>>>
>>>>
>>>
>>>> _______________________________________________
>>>
>>>> Xen-devel mailing list
>>>
>>>> Xen-devel@lists.xensource.com
>>>
>>>> http://lists.xensource.com/xen-devel
>>>
>>>>
>>>
>>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>
>>>
>>
>>
>>
>> --
>> Jean Guyader
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Mar-30 16:11 UTC

head link

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

That looks great! (I wish I had a Q45 with embedded Gfx)

I`m not using Jean`s patch in this case and therefore i`m not getting as far as
you, maybe I need to hack some of the patch into my xen-unstable? i`ve not even
looked into what it does yet.

Are the other cards you tested with PCI Express or just PCI?

It''s the switchover from the Cirrus/vnc-vga to the pciback card that is
failing, or should they be able to co-exist?
My assumption is the pciback card bios needs to be reloaded to remap the memory
adjacent to the Cirrus, but this is not happening.


-----Original Message-----
From: Alexia Benington [mailto:alexbenington@gmail.com] 
Sent: 30 March 2009 16:54
To: Tim Moore
Cc: xen-devel@lists.xensource.com
Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

Hi Tim,

Apologies for the double post, I think I didn''t get the screenshot
through due to its size.

It depends on your definition of success. If it helps, I''ve attached a
screenshot of DomU''s device manager. Dom0 will, obviously, lose access
to the display, when the gfx is handed to pciback. I''ll be glad to
describe further if you require.

With nVidia Quadro, the driver sits in an idling loop. According to
the bugcheck error code, it''s waiting for some kind of response from
the gfx. I suppose this reflects Jean''s point that some drivers are
not very compatible.

I was mostly trying Jean''s patch, which he released sometime last
year, into Xen unstable. I''m also curious how some have mentioned that
they were able to get vga passthrough to work correctly without
applying the patch. It might mostly work, but you''ll still get the
emulated VGA in DomU.

-Alex

On Mon, Mar 30, 2009 at 11:33 AM, Tim Moore <timothy.moore@expidas.net>
wrote:>
> I agree, if any I was expecting the Nvidia card to work the most ..
>
> One interesting thing is that there is a fan on the Nvidia that spins down
when the bios is loaded, with Windows it is evident when the bios is
reset/loaded - just before the GUI is displayed.
> When booting Xen the fan stays high / fast as the pciback is loaded to
prevent Dom0 taking ownership.. boot the DomU (WinXP) and the fan spins down the
same as booting a native Win32 host .. aka I assume Bios/Card is loaded/Reset
here ..
>
> Pls can you explain FLR''s ?
>
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@gmail.com]
> Sent: 30 March 2009 15:57
> To: Tim Moore
> Cc: xen-devel@lists.xensource.com
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>
> Hi Tim,
>
> In the native case there is only one mapping, there isn''t any
> remapping ones the bios has been executed. It is different in the hvm
> scenario.
>
> Anyway, the nvidia graphic card should works fine as a secondary, side
> by side with the cirrus. I don''t know if this card support
FLR''s but
> that would be good to have a look.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>> Hi Jean,
>>
>> I can understand the issue with the graphics card drivers, and I can
see how that could be able problem. I am using a very modern nvidia and ati card
for testing, Alex is also using a couple of other different varieties so we are
able to test with a few different scenarios...
>>
>> How would this work under an MS kernel? these manufacturers do support
dual cards and I have also tested my ATI/Nvidia host with a Windows OS and both
cards are loaded and work fine. This says to me that the drivers can deal with
PCI remapping of some sorts?
>>
>> Cheers,
>> Tim
>>
>> -----Original Message-----
>> From: Jean Guyader [mailto:jean.guyader@gmail.com]
>> Sent: 30 March 2009 15:38
>> To: Tim Moore
>> Cc: xen-devel@lists.xensource.com
>> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>>
>> Hi Tim,
>>
>> Most of the time the problam doesn''t come from the VT-d code
but from
>> the graphic card device drivers. Most of them doesn''t really
like PCI
>> Bar remapping and try to use the old one.
>> For instance the intel igfx has never been designed to be used as a
>> secondary display adapter or even with another Host bridge than the
>> one on the motherboard.
>>
>> Jean
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>> Hi Alex,
>>>
>>>
>>>
>>> When you say you had success with the onboard igfx, how much
success? Have
>>> you ever had a passthrough VGA display from a DomU ?
>>>
>>>
>>>
>>> I have also seen the write-up too although a lot of work has been
done on
>>> the VT-d support lately and I`m hoping to raise this again on the
developer
>>> list and start to document some of the problems/issues so as we can
find a
>>> way do solve it !
>>>
>>>
>>>
>>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>>
>>>
>>>
>>> Tim
>>>
>>>
>>>
>>> ________________________________
>>>
>>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>>
>>>         <any> DomU
>>>
>>> From: Alexia Benington <alexbenington@gmail.com>
>>>
>>> To: Tim Moore <timothy.moore@expidas.net>
>>>
>>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I''ve been trying to get this to work for some months,
albeit not
>>>
>>> full-time. I concur that many would like to see it working. Some
have
>>>
>>> posted that they managed to get it to work correctly, however, they
>>>
>>> almost always never reply to queries on how they got it to work, or
>>>
>>> their observations.
>>>
>>>
>>>
>>> I did manage to get rid of the yellow exclamation mark.
You''ll need to
>>>
>>> disable the emulated VGA bios, pretty much as described here
>>>
>>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>>
>>>
>>>
>>> Even after doing so, some of the registers are not directly passed
>>>
>>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>>
>>> 01131.html.
>>>
>>> I''m not sure if that may affect anything.
>>>
>>>
>>>
>>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS
290, nVidia
>>>
>>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>>
>>> the onboard igfx as the primary, which is also being passed
through.
>>>
>>>
>>>
>>> There is also a writeup which you might want to take a look
>>>
>>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>>
>>>
>>>
>>> I''m also trying to get DomU''s output to the
display.
>>>
>>>
>>>
>>> -Alex
>>>
>>>
>>>
>>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>>
>>>> Hi
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have been testing xen-unstable.hg (29/03/09) over the past
few days ..
>>>
>>>> trying to enable passthrough for VGA.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I am using Debian 5.0 (lenny) x64 and compiling from source
each time. Tr>>>
>>> ied
>>>
>>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58
(ASUS P6T
>>>
>>>> Deluxe) with Core i7 920 CPU.
>>>
>>>>
>>>
>>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line
388) and
>>>
>>>> remove the DMAR bail because of bad RMRR tables.
>>>
>>>>
>>>
>>>> I also have ATI X300 PCIe for Dom0 Console.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>>
>>>>
>>>
>>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I compiled pciback as embedded in the kernel and I have tried
many differ>>>
>>> ent
>>>
>>>> scenarios:
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param)
(no ATI card
>>>
>>>> installed)
>>>
>>>>
>>>
>>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>>
>>>>
>>>
>>>> 3) all scenarios with pciback as module and using echo devn
>
>>>
>>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> In all scenarios I can successfully see the correct PCI device
in the
>>>
>>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>>
>>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU
and
>>>
>>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>>
>>>> Nvidia card, windows correctly identified the device and
installed the
>>>
>>>> driver! (latest from nvidia.com). As we all know =96 Windows
wanted a reb>>>
>>> oot-
>>>
>>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>>
>>>> indicating reboot required and resources unavailable.
>>>
>>>>
>>>
>>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>>
>>> e
>>>
>>>> same steps. Windows XP now boots without crash although the ATI
device st>>>
>>> ill
>>>
>>>> shows a yellow exclamation and insufficient resources.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> To summarise, pciback functions as expected for PCIe in Dom0
and DomU see>>>
>>> s
>>>
>>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>>
>>> nd
>>>
>>>> devid show and are managed correctly.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have read all that there is on this subject and have come to
the
>>>
>>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>>
>>>> the new Video device, the Cirrus or stdvga card is always
present which I
>>>
>>>> believe may be causing the problem. I have tried nographics=3D1
but qemu->>>
>>> dm
>>>
>>>> still maps the Video BIOS.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Then I noticed that qemu-dm has the new options for =93-vga
none=94 .. so>>>
>>>  I
>>>
>>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>>
>>> ely
>>>
>>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>>
>>> the
>>>
>>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>>
>>>> boot L
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> To solve this and make it usable I think some development of
qemu with
>>>
>>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>>
>>> a
>>>
>>>> controller is necessary in order to remap the Cirrus or
secondary vga
>>>
>>>> adapters in memory correctly. (its also difficult to configure
a Windows
>>>
>>>> machine for a second vga when the primary is disabled lol.)
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Also, I have run strace using a qemu-dm wrapper (very verbose)
and analys>>>
>>> ed
>>>
>>>> the results =96nothing seems obvious, but i`m not an expert by
far =85.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Windows has been able to support multiple VGA cards for a long
time; I
>>>
>>>> assume that MS is remapping these somehow?
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have read in 2 different places that this has worked for
others before
>>>
>>>> (without the legacy patch for igfx) and with xen-unstable.hg ..
why does >>>
>>> it
>>>
>>>> not work for me ?
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Any help / direction to further debug would be helpful .. I
think many
>>>
>>>> people would like to get this one working once and for all!
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Tim
>>>
>>>>
>>>
>>>> _______________________________________________
>>>
>>>> Xen-devel mailing list
>>>
>>>> Xen-devel@lists.xensource.com
>>>
>>>> http://lists.xensource.com/xen-devel
>>>
>>>>
>>>
>>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>
>>>
>>
>>
>>
>> --
>> Jean Guyader
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ross Philipson

2009-Mar-30 16:43 UTC

head link

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

FLRs allow you to do a full reset of the device at the slot/function level
putting the device is an uninitialized and quiescent state. For PCI devices it
should show up as PCI capability 0x13. PCIe support for it is indicated in the
PCI Express Capabilities -> Device Capabilities Register (bit 28). For both
types you write the appropriate FLR command bit to do the reset. This is more or
less a requirement for passing devices between domains (from what I have been
told). The code to perform the actions is in the xend python tools if you want
more details on it.

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tim Moore
Sent: Monday, March 30, 2009 11:34 AM
To: xen-devel@lists.xensource.com
Subject: RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU


I agree, if any I was expecting the Nvidia card to work the most ..

One interesting thing is that there is a fan on the Nvidia that spins down when
the bios is loaded, with Windows it is evident when the bios is reset/loaded -
just before the GUI is displayed.
When booting Xen the fan stays high / fast as the pciback is loaded to prevent
Dom0 taking ownership.. boot the DomU (WinXP) and the fan spins down the same as
booting a native Win32 host .. aka I assume Bios/Card is loaded/Reset here ..

Pls can you explain FLR''s ?


-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@gmail.com] 
Sent: 30 March 2009 15:57
To: Tim Moore
Cc: xen-devel@lists.xensource.com
Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

Hi Tim,

In the native case there is only one mapping, there isn''t any
remapping ones the bios has been executed. It is different in the hvm
scenario.

Anyway, the nvidia graphic card should works fine as a secondary, side
by side with the cirrus. I don''t know if this card support
FLR''s but
that would be good to have a look.

Jean

2009/3/30 Tim Moore <timothy.moore@expidas.net>:>
> Hi Jean,
>
> I can understand the issue with the graphics card drivers, and I can see
how that could be able problem. I am using a very modern nvidia and ati card for
testing, Alex is also using a couple of other different varieties so we are able
to test with a few different scenarios...
>
> How would this work under an MS kernel? these manufacturers do support dual
cards and I have also tested my ATI/Nvidia host with a Windows OS and both cards
are loaded and work fine. This says to me that the drivers can deal with PCI
remapping of some sorts?
>
> Cheers,
> Tim
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@gmail.com]
> Sent: 30 March 2009 15:38
> To: Tim Moore
> Cc: xen-devel@lists.xensource.com
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>
> Hi Tim,
>
> Most of the time the problam doesn''t come from the VT-d code but
from
> the graphic card device drivers. Most of them doesn''t really like
PCI
> Bar remapping and try to use the old one.
> For instance the intel igfx has never been designed to be used as a
> secondary display adapter or even with another Host bridge than the
> one on the motherboard.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>> Hi Alex,
>>
>>
>>
>> When you say you had success with the onboard igfx, how much success?
Have
>> you ever had a passthrough VGA display from a DomU ?
>>
>>
>>
>> I have also seen the write-up too although a lot of work has been done
on
>> the VT-d support lately and I`m hoping to raise this again on the
developer
>> list and start to document some of the problems/issues so as we can
find a
>> way do solve it !
>>
>>
>>
>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>
>>
>>
>> Tim
>>
>>
>>
>> ________________________________
>>
>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>
>>         <any> DomU
>>
>> From: Alexia Benington <alexbenington@gmail.com>
>>
>> To: Tim Moore <timothy.moore@expidas.net>
>>
>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>
>>
>>
>> Hi,
>>
>>
>>
>> I''ve been trying to get this to work for some months, albeit
not
>>
>> full-time. I concur that many would like to see it working. Some have
>>
>> posted that they managed to get it to work correctly, however, they
>>
>> almost always never reply to queries on how they got it to work, or
>>
>> their observations.
>>
>>
>>
>> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>>
>> disable the emulated VGA bios, pretty much as described here
>>
>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>
>>
>>
>> Even after doing so, some of the registers are not directly passed
>>
>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>
>> 01131.html.
>>
>> I''m not sure if that may affect anything.
>>
>>
>>
>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>>
>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>
>> the onboard igfx as the primary, which is also being passed through.
>>
>>
>>
>> There is also a writeup which you might want to take a look
>>
>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>
>>
>>
>> I''m also trying to get DomU''s output to the display.
>>
>>
>>
>> -Alex
>>
>>
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>>> Hi
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have been testing xen-unstable.hg (29/03/09) over the past few
days ..
>>
>>> trying to enable passthrough for VGA.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I am using Debian 5.0 (lenny) x64 and compiling from source each
time. Tr>>
>> ied
>>
>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>
>>>
>>
>>>
>>
>>>
>>
>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS
P6T
>>
>>> Deluxe) with Core i7 920 CPU.
>>
>>>
>>
>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388)
and
>>
>>> remove the DMAR bail because of bad RMRR tables.
>>
>>>
>>
>>> I also have ATI X300 PCIe for Dom0 Console.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>
>>>
>>
>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>
>>>
>>
>>>
>>
>>>
>>
>>> I compiled pciback as embedded in the kernel and I have tried many
differ>>
>> ent
>>
>>> scenarios:
>>
>>>
>>
>>>
>>
>>>
>>
>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no
ATI card
>>
>>> installed)
>>
>>>
>>
>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>
>>>
>>
>>> 3) all scenarios with pciback as module and using echo devn >
>>
>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>
>>>
>>
>>>
>>
>>>
>>
>>> In all scenarios I can successfully see the correct PCI device in
the
>>
>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>
>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>>
>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>
>>> Nvidia card, windows correctly identified the device and installed
the
>>
>>> driver! (latest from nvidia.com). As we all know =96 Windows wanted
a reb>>
>> oot-
>>
>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>
>>> indicating reboot required and resources unavailable.
>>
>>>
>>
>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>
>> e
>>
>>> same steps. Windows XP now boots without crash although the ATI
device st>>
>> ill
>>
>>> shows a yellow exclamation and insufficient resources.
>>
>>>
>>
>>>
>>
>>>
>>
>>> To summarise, pciback functions as expected for PCIe in Dom0 and
DomU see>>
>> s
>>
>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>
>> nd
>>
>>> devid show and are managed correctly.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read all that there is on this subject and have come to the
>>
>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>
>>> the new Video device, the Cirrus or stdvga card is always present
which I
>>
>>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->>
>> dm
>>
>>> still maps the Video BIOS.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Then I noticed that qemu-dm has the new options for =93-vga none=94
.. so>>
>>  I
>>
>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>
>> ely
>>
>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>
>> the
>>
>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>
>>> boot L
>>
>>>
>>
>>>
>>
>>>
>>
>>> To solve this and make it usable I think some development of qemu
with
>>
>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>
>> a
>>
>>> controller is necessary in order to remap the Cirrus or secondary
vga
>>
>>> adapters in memory correctly. (its also difficult to configure a
Windows
>>
>>> machine for a second vga when the primary is disabled lol.)
>>
>>>
>>
>>>
>>
>>>
>>
>>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>>
>> ed
>>
>>> the results =96nothing seems obvious, but i`m not an expert by far
=85.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Windows has been able to support multiple VGA cards for a long
time; I
>>
>>> assume that MS is remapping these somehow?
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read in 2 different places that this has worked for others
before
>>
>>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >>
>> it
>>
>>> not work for me ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> Any help / direction to further debug would be helpful .. I think
many
>>
>>> people would like to get this one working once and for all!
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tim
>>
>>>
>>
>>> _______________________________________________
>>
>>> Xen-devel mailing list
>>
>>> Xen-devel@lists.xensource.com
>>
>>> http://lists.xensource.com/xen-devel
>>
>>>
>>
>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Apr-07 00:04 UTC

head link

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to and DomU

Some further developments .. (sorry for the long post)

After more wasted days - sigh - not much further ..

Still trying with xen-unstable-hg modified as mentioned earlier in the thread to
workaround bad RMRR/DMAR addresses.

Have also successfully compiled and booted XenClient (XCI), which I have tried
booting DomU with vga and no success :(

I believe the problem is now down to getting the correct FLR to prepare the VGA
card for DomU. I am starting the DomU with  pci = [''03:00.0''] 
in the config file so as qemu starts with the device attached. I have also tried
xm pci-attach and detach (which work great by the way).

I can see the device in WinXP DomU and installed latest NVidia driver, driver
can`t load due to lack of resources (Code 10).

Also tried with nographics=1 and stdvga=0/1.
Also created qemu-dm wrapper to pass "-vga none" ...

Still Code 10 and no response from the Nvidia card/monitor in Windows DomU.

Here is my lspci listing for the Nvidia 8800 GTS 512 (I see FLReset- so is card
supported?)

This is from Dom0 (2.6.27.5) with pciback.hide=(03:00.0) and iommu=on,pv (DomU
is loaded with card attached)
Memory Region 0,1,3,5 all show [disabled] when DomU is down btw.

03:00.0 VGA compatible controller: nVidia Corporation GeForce 8800 GTS 512 (rev
a2) (prog-if 00 [VGA controller])
        Subsystem: Device 19f1:0719
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 30
        Region 0: Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
        Region 1: Memory at d0000000 (64-bit, prefetchable) [size=256M]
        Region 3: Memory at f8000000 (64-bit, non-prefetchable) [size=32M]
        Region 5: I/O ports at ac00 [size=128]
        Expansion ROM at fbae0000 [disabled] [size=128K]
        Capabilities: [60] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [68] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0
Enable+
                Address: 00000000fee004d8  Data: 0000
        Capabilities: [78] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
<512ns, L1 <4us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr-
TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0
<256ns, L1 <1us
                        ClockPM- Suprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive-
BWMgmt- ABWMgmt-
        Capabilities: [100] Virtual Channel <?>
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [600] Vendor Specific Information <?>
        Kernel driver in use: pciback
        Kernel modules: nvidiafb


qemu-dm output (pertinent information):

[2009-04-07 00:31:21 3873] DEBUG (XendDomainInfo:2444)
_initDomain:shadow_memory=0x0, memory_static_max=0x40000000,
memory_static_min=0x0.
[2009-04-07 00:31:21 3873] DEBUG (balloon:166) Balloon: 5130492 KiB free; need
1065984; done.
[2009-04-07 00:31:21 3873] INFO (image:168) buildDomain os=hvm dom=22 vcpus=1
[2009-04-07 00:31:21 3873] DEBUG (image:843) domid          = 22
[2009-04-07 00:31:21 3873] DEBUG (image:844) image          =
/usr/lib/xen/boot/hvmloader
[2009-04-07 00:31:21 3873] DEBUG (image:845) store_evtchn   = 2
[2009-04-07 00:31:21 3873] DEBUG (image:846) memsize        = 1024
[2009-04-07 00:31:21 3873] DEBUG (image:847) target         = 1024
[2009-04-07 00:31:21 3873] DEBUG (image:848) vcpus          = 1
[2009-04-07 00:31:21 3873] DEBUG (image:849) acpi           = 1
[2009-04-07 00:31:21 3873] DEBUG (image:850) apic           = 1
....
...
[2009-04-07 00:31:21 3873] INFO (pciquirk:90) NO quirks found for PCI device
[10de:0600:19f1:0719]
[2009-04-07 00:31:21 3873] DEBUG (pciquirk:130) Permissive mode enabled for PCI
device [10de:0600:19f1:0719]
[2009-04-07 00:31:21 3873] DEBUG (pciquirk:139) Unconstrained device:
0000:03:00.0
[2009-04-07 00:31:21 3873] DEBUG (pciif:335) pci: enabling ioport 0xac00/0x80
[2009-04-07 00:31:21 3873] DEBUG (pciif:349) pci: enabling iomem
0xfa000000/0x1000000 pfn 0xfa000/0x1000
[2009-04-07 00:31:21 3873] DEBUG (pciif:349) pci: enabling iomem
0xd0000000/0x10000000 pfn 0xd0000/0x10000
[2009-04-07 00:31:21 3873] DEBUG (pciif:349) pci: enabling iomem
0xf8000000/0x2000000 pfn 0xf8000/0x2000
[2009-04-07 00:31:21 3873] DEBUG (pciif:349) pci: enabling iomem
0xfbae0000/0x20000 pfn 0xfbae0/0x20
[2009-04-07 00:31:22 3873] DEBUG (pciif:378) pci: enabling irq 30
[2009-04-07 00:31:22 3873] DEBUG (pciif:457) pci: register aer watch
/local/domain/0/backend/pci/22/0/aerState

.....
.....
xm dmesg output ....
(XEN) HVM22: HVM Loader
(XEN) HVM22: Detected Xen v3.4-unstable
(XEN) HVM22: CPU speed is 2673 MHz
(XEN) irq.c:235: Dom22 PCI link 0 changed 0 -> 5
(XEN) HVM22: PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:235: Dom22 PCI link 1 changed 0 -> 10
(XEN) HVM22: PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:235: Dom22 PCI link 2 changed 0 -> 11
(XEN) HVM22: PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:235: Dom22 PCI link 3 changed 0 -> 5
(XEN) HVM22: PCI-ISA link 3 routed to IRQ5
(XEN) HVM22: pci dev 01:3 INTA->IRQ10
(XEN) HVM22: pci dev 03:0 INTA->IRQ5
(XEN) HVM22: pci dev 04:0 INTA->IRQ5
(XEN) HVM22: pci dev 05:0 INTA->IRQ10
(XEN) HVM22: pci dev 03:0 bar 14 size 10000000: e000000c
(XEN) domctl.c:822:d0 memory_map:add: gfn=e0000 mfn=d0000 nr_mfns=10000
(XEN) HVM22: pci dev 03:0 bar 1c size 02000000: f0000004
(XEN) domctl.c:822:d0 memory_map:add: gfn=f0000 mfn=f8000 nr_mfns=2000
(XEN) HVM22: pci dev 03:0 bar 10 size 01000000: f2000000
(XEN) domctl.c:822:d0 memory_map:add: gfn=f2000 mfn=fa000 nr_mfns=1000
(XEN) HVM22: pci dev 04:0 bar 14 size 01000000: f3000008
(XEN) HVM22: pci dev 02:0 bar 10 size 00400000: f4000008
(XEN) HVM22: pci dev 03:0 bar 30 size 00020000: f4400000
(XEN) domctl.c:822:d0 memory_map:add: gfn=f4400 mfn=fbae0 nr_mfns=20
(XEN) HVM22: pci dev 04:0 bar 10 size 00000100: 0000c001
(XEN) HVM22: pci dev 05:0 bar 10 size 00000100: 0000c101
(XEN) HVM22: pci dev 05:0 bar 14 size 00000100: f4420000
(XEN) HVM22: pci dev 03:0 bar 24 size 00000080: 0000c201
(XEN) domctl.c:873:d0 ioport_map:add f_gport=c200 f_mport=ac00 np=80
(XEN) HVM22: pci dev 01:1 bar 20 size 00000010: 0000c281
(XEN) HVM22: Multiprocessor initialisation:
(XEN) HVM22:  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [3/8] ...
done.
(XEN) HVM22: Testing HVM environment:
(XEN) HVM22:  - REP INSB across page boundaries ... passed
(XEN) HVM22: Passed 1/1 tests
(XEN) HVM22: Writing SMBIOS tables ...
(XEN) HVM22: Loading ROMBIOS ...
(XEN) HVM22: 9788 bytes of ROMBIOS high-memory extensions:
(XEN) HVM22:   Relocating to 0xfc000000-0xfc00263c ... done
(XEN) HVM22: Creating MP tables ...
(XEN) HVM22: Loading Standard VGABIOS ...
(XEN) HVM22: Loading PCI Option ROM ...
(XEN) HVM22:  - Manufacturer: http://etherboot.org
(XEN) HVM22:  - Product name: gPXE
(XEN) HVM22: Loading ACPI ...
(XEN) HVM22:  - Lo data: 000ea020-000ea04f
(XEN) HVM22:  - Hi data: fc002800-fc005caf
(XEN) HVM22: vm86 TSS at fc006000
(XEN) HVM22: BIOS map:
(XEN) HVM22:  c0000-c9fff: VGA BIOS
(XEN) HVM22:  ca000-d67ff: Etherboot ROM
(XEN) HVM22:  eb000-eb14e: SMBIOS tables
(XEN) HVM22:  f0000-fffff: Main BIOS
(XEN) HVM22: Invoking ROMBIOS ...
(XEN) HVM22: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) stdvga.c:147:d22 entering stdvga and caching modes
(XEN) HVM22: VGABios $Id: vgabios.c,v 1.67 2008/01/27 09:44:12 vruppert Exp $
(XEN) HVM22: VBE Bios $Id: vbe.c,v 1.60 2008/03/02 07:47:21 vruppert Exp $
(XEN) HVM22: Bochs BIOS - build: 06/23/99
(XEN) HVM22: $Revision: 1.221 $ $Date: 2008/12/07 17:32:29 $
(XEN) HVM22: Options: apmbios pcibios eltorito PMM
(XEN) HVM22:
(XEN) HVM22: ata0-0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
(XEN) HVM22: ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (38912 MBytes)
(XEN) HVM22: IDE time out
(XEN) HVM22: ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
(XEN) HVM22: IDE time out
(XEN) HVM22:
(XEN) HVM22:
(XEN) HVM22:
(XEN) HVM22: Press F12 for boot menu.
(XEN) HVM22:
(XEN) HVM22: Booting from Hard Disk...
(XEN) HVM22: Booting from 0000:7c00
(XEN) HVM22: int13_harddisk: function 15, unmapped device for ELDL=81
(XEN) HVM22: *** int 15h function AX=e980, BX=0066 not yet supported!
(XEN) irq.c:235: Dom22 PCI link 0 changed 5 -> 0
(XEN) irq.c:235: Dom22 PCI link 1 changed 10 -> 0
(XEN) irq.c:235: Dom22 PCI link 2 changed 11 -> 0
(XEN) irq.c:235: Dom22 PCI link 3 changed 5 -> 0
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f2000 mfn=fa000 nr_mfns=1000
(XEN) domctl.c:832:d0 memory_map:remove: gfn=e0000 mfn=d0000 nr_mfns=10000
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f0000 mfn=f8000 nr_mfns=2000
(XEN) domctl.c:897:d0 ioport_map:remove f_gport=c200 f_mport=ac00 np=80
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f4400 mfn=fbae0 nr_mfns=20
(XEN) domctl.c:822:d0 memory_map:add: gfn=f2000 mfn=fa000 nr_mfns=1000
(XEN) domctl.c:822:d0 memory_map:add: gfn=e0000 mfn=d0000 nr_mfns=10000
(XEN) domctl.c:822:d0 memory_map:add: gfn=f0000 mfn=f8000 nr_mfns=2000
(XEN) domctl.c:873:d0 ioport_map:add f_gport=c200 f_mport=ac00 np=80
(XEN) domctl.c:822:d0 memory_map:add: gfn=f4400 mfn=fbae0 nr_mfns=20
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f2000 mfn=fa000 nr_mfns=1000
(XEN) domctl.c:832:d0 memory_map:remove: gfn=e0000 mfn=d0000 nr_mfns=10000
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f0000 mfn=f8000 nr_mfns=2000
(XEN) domctl.c:897:d0 ioport_map:remove f_gport=c200 f_mport=ac00 np=80
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f4400 mfn=fbae0 nr_mfns=20
(XEN) domctl.c:822:d0 memory_map:add: gfn=f2000 mfn=fa000 nr_mfns=1000
(XEN) domctl.c:822:d0 memory_map:add: gfn=e0000 mfn=d0000 nr_mfns=10000
(XEN) domctl.c:822:d0 memory_map:add: gfn=f0000 mfn=f8000 nr_mfns=2000
(XEN) domctl.c:873:d0 ioport_map:add f_gport=c200 f_mport=ac00 np=80
(XEN) domctl.c:822:d0 memory_map:add: gfn=f4400 mfn=fbae0 nr_mfns=20
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f2000 mfn=fa000 nr_mfns=1000
(XEN) domctl.c:832:d0 memory_map:remove: gfn=e0000 mfn=d0000 nr_mfns=10000
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f0000 mfn=f8000 nr_mfns=2000
(XEN) domctl.c:897:d0 ioport_map:remove f_gport=c200 f_mport=ac00 np=80
(XEN) domctl.c:832:d0 memory_map:remove: gfn=f4400 mfn=fbae0 nr_mfns=20
(XEN) domctl.c:822:d0 memory_map:add: gfn=f2000 mfn=fa000 nr_mfns=1000
(XEN) domctl.c:822:d0 memory_map:add: gfn=e0000 mfn=d0000 nr_mfns=10000
(XEN) domctl.c:822:d0 memory_map:add: gfn=f0000 mfn=f8000 nr_mfns=2000
(XEN) domctl.c:873:d0 ioport_map:add f_gport=c200 f_mport=ac00 np=80
(XEN) domctl.c:822:d0 memory_map:add: gfn=f4400 mfn=fbae0 nr_mfns=20



I see no errors ... why is the nvidia card not being initialised ? FLR ?


Thanks,
Tim



-----Original Message-----
From: Ross Philipson [mailto:Ross.Philipson@citrix.com]
Sent: 30 March 2009 17:43
To: Tim Moore; xen-devel@lists.xensource.com
Subject: RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

FLRs allow you to do a full reset of the device at the slot/function level
putting the device is an uninitialized and quiescent state. For PCI devices it
should show up as PCI capability 0x13. PCIe support for it is indicated in the
PCI Express Capabilities -> Device Capabilities Register (bit 28). For both
types you write the appropriate FLR command bit to do the reset. This is more or
less a requirement for passing devices between domains (from what I have been
told). The code to perform the actions is in the xend python tools if you want
more details on it.

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tim Moore
Sent: Monday, March 30, 2009 11:34 AM
To: xen-devel@lists.xensource.com
Subject: RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU


I agree, if any I was expecting the Nvidia card to work the most ..

One interesting thing is that there is a fan on the Nvidia that spins down when
the bios is loaded, with Windows it is evident when the bios is reset/loaded -
just before the GUI is displayed.
When booting Xen the fan stays high / fast as the pciback is loaded to prevent
Dom0 taking ownership.. boot the DomU (WinXP) and the fan spins down the same as
booting a native Win32 host .. aka I assume Bios/Card is loaded/Reset here ..

Pls can you explain FLR''s ?


-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@gmail.com]
Sent: 30 March 2009 15:57
To: Tim Moore
Cc: xen-devel@lists.xensource.com
Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

Hi Tim,

In the native case there is only one mapping, there isn''t any
remapping ones the bios has been executed. It is different in the hvm
scenario.

Anyway, the nvidia graphic card should works fine as a secondary, side
by side with the cirrus. I don''t know if this card support
FLR''s but
that would be good to have a look.

Jean

2009/3/30 Tim Moore <timothy.moore@expidas.net>:>
> Hi Jean,
>
> I can understand the issue with the graphics card drivers, and I can see
how that could be able problem. I am using a very modern nvidia and ati card for
testing, Alex is also using a couple of other different varieties so we are able
to test with a few different scenarios...
>
> How would this work under an MS kernel? these manufacturers do support dual
cards and I have also tested my ATI/Nvidia host with a Windows OS and both cards
are loaded and work fine. This says to me that the drivers can deal with PCI
remapping of some sorts?
>
> Cheers,
> Tim
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@gmail.com]
> Sent: 30 March 2009 15:38
> To: Tim Moore
> Cc: xen-devel@lists.xensource.com
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>
> Hi Tim,
>
> Most of the time the problam doesn''t come from the VT-d code but
from
> the graphic card device drivers. Most of them doesn''t really like
PCI
> Bar remapping and try to use the old one.
> For instance the intel igfx has never been designed to be used as a
> secondary display adapter or even with another Host bridge than the
> one on the motherboard.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>> Hi Alex,
>>
>>
>>
>> When you say you had success with the onboard igfx, how much success?
Have
>> you ever had a passthrough VGA display from a DomU ?
>>
>>
>>
>> I have also seen the write-up too although a lot of work has been done
on
>> the VT-d support lately and I`m hoping to raise this again on the
developer
>> list and start to document some of the problems/issues so as we can
find a
>> way do solve it !
>>
>>
>>
>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>
>>
>>
>> Tim
>>
>>
>>
>> ________________________________
>>
>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>
>>         <any> DomU
>>
>> From: Alexia Benington <alexbenington@gmail.com>
>>
>> To: Tim Moore <timothy.moore@expidas.net>
>>
>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>
>>
>>
>> Hi,
>>
>>
>>
>> I''ve been trying to get this to work for some months, albeit
not
>>
>> full-time. I concur that many would like to see it working. Some have
>>
>> posted that they managed to get it to work correctly, however, they
>>
>> almost always never reply to queries on how they got it to work, or
>>
>> their observations.
>>
>>
>>
>> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>>
>> disable the emulated VGA bios, pretty much as described here
>>
>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>
>>
>>
>> Even after doing so, some of the registers are not directly passed
>>
>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>
>> 01131.html.
>>
>> I''m not sure if that may affect anything.
>>
>>
>>
>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>>
>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>
>> the onboard igfx as the primary, which is also being passed through.
>>
>>
>>
>> There is also a writeup which you might want to take a look
>>
>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>
>>
>>
>> I''m also trying to get DomU''s output to the display.
>>
>>
>>
>> -Alex
>>
>>
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>>> Hi
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have been testing xen-unstable.hg (29/03/09) over the past few
days ..
>>
>>> trying to enable passthrough for VGA.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I am using Debian 5.0 (lenny) x64 and compiling from source each
time. Tr>>
>> ied
>>
>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>
>>>
>>
>>>
>>
>>>
>>
>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS
P6T
>>
>>> Deluxe) with Core i7 920 CPU.
>>
>>>
>>
>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388)
and
>>
>>> remove the DMAR bail because of bad RMRR tables.
>>
>>>
>>
>>> I also have ATI X300 PCIe for Dom0 Console.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>
>>>
>>
>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>
>>>
>>
>>>
>>
>>>
>>
>>> I compiled pciback as embedded in the kernel and I have tried many
differ>>
>> ent
>>
>>> scenarios:
>>
>>>
>>
>>>
>>
>>>
>>
>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no
ATI card
>>
>>> installed)
>>
>>>
>>
>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>
>>>
>>
>>> 3) all scenarios with pciback as module and using echo devn >
>>
>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>
>>>
>>
>>>
>>
>>>
>>
>>> In all scenarios I can successfully see the correct PCI device in
the
>>
>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>
>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>>
>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>
>>> Nvidia card, windows correctly identified the device and installed
the
>>
>>> driver! (latest from nvidia.com). As we all know =96 Windows wanted
a reb>>
>> oot-
>>
>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>
>>> indicating reboot required and resources unavailable.
>>
>>>
>>
>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>
>> e
>>
>>> same steps. Windows XP now boots without crash although the ATI
device st>>
>> ill
>>
>>> shows a yellow exclamation and insufficient resources.
>>
>>>
>>
>>>
>>
>>>
>>
>>> To summarise, pciback functions as expected for PCIe in Dom0 and
DomU see>>
>> s
>>
>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>
>> nd
>>
>>> devid show and are managed correctly.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read all that there is on this subject and have come to the
>>
>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>
>>> the new Video device, the Cirrus or stdvga card is always present
which I
>>
>>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->>
>> dm
>>
>>> still maps the Video BIOS.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Then I noticed that qemu-dm has the new options for =93-vga none=94
.. so>>
>>  I
>>
>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>
>> ely
>>
>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>
>> the
>>
>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>
>>> boot L
>>
>>>
>>
>>>
>>
>>>
>>
>>> To solve this and make it usable I think some development of qemu
with
>>
>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>
>> a
>>
>>> controller is necessary in order to remap the Cirrus or secondary
vga
>>
>>> adapters in memory correctly. (its also difficult to configure a
Windows
>>
>>> machine for a second vga when the primary is disabled lol.)
>>
>>>
>>
>>>
>>
>>>
>>
>>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>>
>> ed
>>
>>> the results =96nothing seems obvious, but i`m not an expert by far
=85.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Windows has been able to support multiple VGA cards for a long
time; I
>>
>>> assume that MS is remapping these somehow?
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read in 2 different places that this has worked for others
before
>>
>>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >>
>> it
>>
>>> not work for me ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> Any help / direction to further debug would be helpful .. I think
many
>>
>>> people would like to get this one working once and for all!
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tim
>>
>>>
>>
>>> _______________________________________________
>>
>>> Xen-devel mailing list
>>
>>> Xen-devel@lists.xensource.com
>>
>>> http://lists.xensource.com/xen-devel
>>
>>>
>>
>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


--
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Apr-07 11:04 UTC

head link

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Hi Jean / Ross,

I have been spending a lot of time trying to make this work.

At the moment I am booting with the XenClientInitiative (2.6.18) all downloaded
and compiled from source. (took some time to understand the buildroot and
toolchain due to limited documentation) but I''m there now !

Scenarios tried:

Under XCI:
echo -n "sbr=0000:03:00.0" >/sys/bus/pci/drivers/pciback/resets
echo -n "d3r=0000:03:00.0" >/sys/bus/pci/drivers/pciback/resets
echo -n "noflr=0000:03:00.0" >/sys/bus/pci/drivers/pciback/resets

The output (on tty3) each time shows "PCI FLR not performed for device
03:00.0" when I try:

echo -n "0000:03:00.0" >/sys/bus/pci/drivers/pciback/do_flr

Also get "PCI FLR not performed for device 03:00.0"

pciback and qemu all seems to map all i/o memory and irq through correctly.

Please see the lspci at earlier post:
http://lists.xensource.com/archives/html/xen-devel/2009-04/msg00218.html

(same scenario in both XCI and xen-unstable)

I have also read through the pci.py and pciiy.py files under xen-unstable. The
function do_FLR() seems to attempt all the FLR methods when the device is
disconnected, so I tried attach/detach under xen-unstable but there is no
indication in DEBUG messages about the FLR being performed or failing at all !?

Do you know how I can dump out the pci dev structure to determine if the FLR for
this card is being detected/enumerated on the PCIe capabilties ? Maybe this
would be a good addition to the DEBUG output for the future anyway, to help
diagnose FLR capabilities...

Thanks,
Tim

P.S I have now also purchased a Nvidia Geforce 9500GT and attempted to setup the
configuration from here:
http://www.nabble.com/Successful-PCIe-Graphics-VT-d-Passthrough-to-Win32-DomU,-Q35-chipset-td21671745.html

I think the 8800 GTS has a better chance of working, although they both show
FLReset- in the lspci devcap.



-----Original Message-----
From: Ross Philipson [mailto:Ross.Philipson@citrix.com] 
Sent: 30 March 2009 17:43
To: Tim Moore; xen-devel@lists.xensource.com
Subject: RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

FLRs allow you to do a full reset of the device at the slot/function level
putting the device is an uninitialized and quiescent state. For PCI devices it
should show up as PCI capability 0x13. PCIe support for it is indicated in the
PCI Express Capabilities -> Device Capabilities Register (bit 28). For both
types you write the appropriate FLR command bit to do the reset. This is more or
less a requirement for passing devices between domains (from what I have been
told). The code to perform the actions is in the xend python tools if you want
more details on it.

Thanks
Ross

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tim Moore
Sent: Monday, March 30, 2009 11:34 AM
To: xen-devel@lists.xensource.com
Subject: RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU


I agree, if any I was expecting the Nvidia card to work the most ..

One interesting thing is that there is a fan on the Nvidia that spins down when
the bios is loaded, with Windows it is evident when the bios is reset/loaded -
just before the GUI is displayed.
When booting Xen the fan stays high / fast as the pciback is loaded to prevent
Dom0 taking ownership.. boot the DomU (WinXP) and the fan spins down the same as
booting a native Win32 host .. aka I assume Bios/Card is loaded/Reset here ..

Pls can you explain FLR''s ?


-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@gmail.com] 
Sent: 30 March 2009 15:57
To: Tim Moore
Cc: xen-devel@lists.xensource.com
Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

Hi Tim,

In the native case there is only one mapping, there isn''t any
remapping ones the bios has been executed. It is different in the hvm
scenario.

Anyway, the nvidia graphic card should works fine as a secondary, side
by side with the cirrus. I don''t know if this card support
FLR''s but
that would be good to have a look.

Jean

2009/3/30 Tim Moore <timothy.moore@expidas.net>:>
> Hi Jean,
>
> I can understand the issue with the graphics card drivers, and I can see
how that could be able problem. I am using a very modern nvidia and ati card for
testing, Alex is also using a couple of other different varieties so we are able
to test with a few different scenarios...
>
> How would this work under an MS kernel? these manufacturers do support dual
cards and I have also tested my ATI/Nvidia host with a Windows OS and both cards
are loaded and work fine. This says to me that the drivers can deal with PCI
remapping of some sorts?
>
> Cheers,
> Tim
>
> -----Original Message-----
> From: Jean Guyader [mailto:jean.guyader@gmail.com]
> Sent: 30 March 2009 15:38
> To: Tim Moore
> Cc: xen-devel@lists.xensource.com
> Subject: Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to <any> DomU
>
> Hi Tim,
>
> Most of the time the problam doesn''t come from the VT-d code but
from
> the graphic card device drivers. Most of them doesn''t really like
PCI
> Bar remapping and try to use the old one.
> For instance the intel igfx has never been designed to be used as a
> secondary display adapter or even with another Host bridge than the
> one on the motherboard.
>
> Jean
>
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>> Hi Alex,
>>
>>
>>
>> When you say you had success with the onboard igfx, how much success?
Have
>> you ever had a passthrough VGA display from a DomU ?
>>
>>
>>
>> I have also seen the write-up too although a lot of work has been done
on
>> the VT-d support lately and I`m hoping to raise this again on the
developer
>> list and start to document some of the problems/issues so as we can
find a
>> way do solve it !
>>
>>
>>
>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>
>>
>>
>> Tim
>>
>>
>>
>> ________________________________
>>
>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>
>>         <any> DomU
>>
>> From: Alexia Benington <alexbenington@gmail.com>
>>
>> To: Tim Moore <timothy.moore@expidas.net>
>>
>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>
>>
>>
>> Hi,
>>
>>
>>
>> I''ve been trying to get this to work for some months, albeit
not
>>
>> full-time. I concur that many would like to see it working. Some have
>>
>> posted that they managed to get it to work correctly, however, they
>>
>> almost always never reply to queries on how they got it to work, or
>>
>> their observations.
>>
>>
>>
>> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>>
>> disable the emulated VGA bios, pretty much as described here
>>
>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>
>>
>>
>> Even after doing so, some of the registers are not directly passed
>>
>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>
>> 01131.html.
>>
>> I''m not sure if that may affect anything.
>>
>>
>>
>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>>
>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>
>> the onboard igfx as the primary, which is also being passed through.
>>
>>
>>
>> There is also a writeup which you might want to take a look
>>
>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>
>>
>>
>> I''m also trying to get DomU''s output to the display.
>>
>>
>>
>> -Alex
>>
>>
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>>> Hi
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have been testing xen-unstable.hg (29/03/09) over the past few
days ..
>>
>>> trying to enable passthrough for VGA.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I am using Debian 5.0 (lenny) x64 and compiling from source each
time. Tr>>
>> ied
>>
>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>
>>>
>>
>>>
>>
>>>
>>
>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS
P6T
>>
>>> Deluxe) with Core i7 920 CPU.
>>
>>>
>>
>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388)
and
>>
>>> remove the DMAR bail because of bad RMRR tables.
>>
>>>
>>
>>> I also have ATI X300 PCIe for Dom0 Console.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>
>>>
>>
>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>
>>>
>>
>>>
>>
>>>
>>
>>> I compiled pciback as embedded in the kernel and I have tried many
differ>>
>> ent
>>
>>> scenarios:
>>
>>>
>>
>>>
>>
>>>
>>
>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no
ATI card
>>
>>> installed)
>>
>>>
>>
>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>
>>>
>>
>>> 3) all scenarios with pciback as module and using echo devn >
>>
>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>
>>>
>>
>>>
>>
>>>
>>
>>> In all scenarios I can successfully see the correct PCI device in
the
>>
>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>
>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>>
>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>
>>> Nvidia card, windows correctly identified the device and installed
the
>>
>>> driver! (latest from nvidia.com). As we all know =96 Windows wanted
a reb>>
>> oot-
>>
>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>
>>> indicating reboot required and resources unavailable.
>>
>>>
>>
>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>
>> e
>>
>>> same steps. Windows XP now boots without crash although the ATI
device st>>
>> ill
>>
>>> shows a yellow exclamation and insufficient resources.
>>
>>>
>>
>>>
>>
>>>
>>
>>> To summarise, pciback functions as expected for PCIe in Dom0 and
DomU see>>
>> s
>>
>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>
>> nd
>>
>>> devid show and are managed correctly.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read all that there is on this subject and have come to the
>>
>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>
>>> the new Video device, the Cirrus or stdvga card is always present
which I
>>
>>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->>
>> dm
>>
>>> still maps the Video BIOS.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Then I noticed that qemu-dm has the new options for =93-vga none=94
.. so>>
>>  I
>>
>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>
>> ely
>>
>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>
>> the
>>
>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>
>>> boot L
>>
>>>
>>
>>>
>>
>>>
>>
>>> To solve this and make it usable I think some development of qemu
with
>>
>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>
>> a
>>
>>> controller is necessary in order to remap the Cirrus or secondary
vga
>>
>>> adapters in memory correctly. (its also difficult to configure a
Windows
>>
>>> machine for a second vga when the primary is disabled lol.)
>>
>>>
>>
>>>
>>
>>>
>>
>>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>>
>> ed
>>
>>> the results =96nothing seems obvious, but i`m not an expert by far
=85.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Windows has been able to support multiple VGA cards for a long
time; I
>>
>>> assume that MS is remapping these somehow?
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read in 2 different places that this has worked for others
before
>>
>>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >>
>> it
>>
>>> not work for me ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> Any help / direction to further debug would be helpful .. I think
many
>>
>>> people would like to get this one working once and for all!
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tim
>>
>>>
>>
>>> _______________________________________________
>>
>>> Xen-devel mailing list
>>
>>> Xen-devel@lists.xensource.com
>>
>>> http://lists.xensource.com/xen-devel
>>
>>>
>>
>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>
>
>
>
> --
> Jean Guyader
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
Jean Guyader

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Tim Moore

2009-Apr-13 19:43 UTC

head link

[Xen-devel] RE: VGA Passthrough

Hi Ben,

I have tried XCI and built it successfully (it''s not a big thing and I
don''t like the Buildroot). Anyway, I don''t think it is going
to solve the problem. Xen client is focused on passing through the primary vga
card, and is still very specific to Intel integrated gfx.

Most of the functionality is already in Xen-3.4-rc1/2 - have you seen my posts
to the xen-devel list recently?

http://markmail.org/message/ndznz4rd6fip32s7

http://markmail.org/message/s56ningx6lrtnxb2

http://markmail.org/message/cfm43qydktvkbijj

As you can see, I''m making some progress but not completely there yet!

I think the next investigation is qemu-dm or the hvmloader, but I
didn''t write this stuff and I''m no expert by far!
I''ve already emailed Ian Pratt to ask for his assistance...

Keep me posted on you progress!

Cheers,
Tim

-----Original Message-----
From: Beng Heng Ng [mailto:bengheng@eecs.umich.edu] 
Sent: 13 April 2009 19:56
To: Tim Moore
Subject: VGA Passthrough

Hi Tim,

I''ve been trying to port the vga passthrough code from XCI to Xen
unstable, unfortunately I''m getting error signal 7.

I think I''ll try to compile XCI and give that a shot. I tried that
earlier, but had a hard time getting it to even build properly. Thus,
can I ask you for a summary of the steps you took to build it, or
perhaps the resources to do so?

What I did was to download all the sources (build, xen, kernel,
toolstack and ioemu) into one folder, and tried to build each component.
Unfortunately, the build is really unforgiving.

I''ll appreciate your help and suggestions. If I''m able to
build and test
it successfully, then at least we have one more test case.

Thanks.

Best Rgds,
Beng Heng, Ng

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sander Eikelenboom

2009-May-01 20:01 UTC

head link

Re[3]: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

For what i understand of it, the video drivers (linux or windows alike)
expect the videobios to be in a hardcoded region, but cannot access it,
even with vt-d/iommu.

Is there a way to map regions from dom0 to domU (same or different address
?)

I haven''t been able to seriously try it because of, guess what .. bad
RMRR
tables. Currently waiting for a response of ASUS and AMI on this issue.


There is also a small thread on LKML about vga passthrough issues
http://lkml.indiana.edu/hypermail/linux/kernel/0904.3/01787.html

--
Sander



Hello Alexia,

Monday, March 30, 2009, 5:47:05 PM, you wrote:
> Hi Tim,
> It depends on your definition of success. If it helps, I''ve
attached a
> screenshot of DomU''s device manager. Dom0 will, obviously, lose
access
> to the display, when the gfx is handed to pciback. I''ll be glad to
> describe further if you require.
> With nVidia Quadro, the driver sits in an idling loop. According to
> the bugcheck error code, it''s waiting for some kind of response
from
> the gfx. I suppose this reflects Jean''s point that some drivers
are
> not very compatible.
> I was mostly trying Jean''s patch, which he released sometime last
> year, into Xen unstable. I''m also curious how some have mentioned
that
> they were able to get vga passthrough to work correctly without
> applying the patch. It might mostly work, but you''ll still get the
> emulated VGA in DomU.
> -Alex
> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>> Hi Alex,
>>
>>
>>
>> When you say you had success with the onboard igfx, how much success?
Have
>> you ever had a passthrough VGA display from a DomU ?
>>
>>
>>
>> I have also seen the write-up too although a lot of work has been done
on
>> the VT-d support lately and I`m hoping to raise this again on the
developer
>> list and start to document some of the problems/issues so as we can
find a
>> way do solve it !
>>
>>
>>
>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>
>>
>>
>> Tim
>>
>>
>>
>> ________________________________
>>
>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>
>>         <any> DomU
>>
>> From: Alexia Benington <alexbenington@gmail.com>
>>
>> To: Tim Moore <timothy.moore@expidas.net>
>>
>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>
>>
>>
>> Hi,
>>
>>
>>
>> I''ve been trying to get this to work for some months, albeit
not
>>
>> full-time. I concur that many would like to see it working. Some have
>>
>> posted that they managed to get it to work correctly, however, they
>>
>> almost always never reply to queries on how they got it to work, or
>>
>> their observations.
>>
>>
>>
>> I did manage to get rid of the yellow exclamation mark. You''ll
need to
>>
>> disable the emulated VGA bios, pretty much as described here
>>
>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>
>>
>>
>> Even after doing so, some of the registers are not directly passed
>>
>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>
>> 01131.html.
>>
>> I''m not sure if that may affect anything.
>>
>>
>>
>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS 290,
nVidia
>>
>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>
>> the onboard igfx as the primary, which is also being passed through.
>>
>>
>>
>> There is also a writeup which you might want to take a look
>>
>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>
>>
>>
>> I''m also trying to get DomU''s output to the display.
>>
>>
>>
>> -Alex
>>
>>
>>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>
>>> Hi
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have been testing xen-unstable.hg (29/03/09) over the past few
days ..
>>
>>> trying to enable passthrough for VGA.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I am using Debian 5.0 (lenny) x64 and compiling from source each
time. Tr>>
>> ied
>>
>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>
>>>
>>
>>>
>>
>>>
>>
>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS
P6T
>>
>>> Deluxe) with Core i7 920 CPU.
>>
>>>
>>
>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388)
and
>>
>>> remove the DMAR bail because of bad RMRR tables.
>>
>>>
>>
>>> I also have ATI X300 PCIe for Dom0 Console.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>
>>>
>>
>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>
>>>
>>
>>>
>>
>>>
>>
>>> I compiled pciback as embedded in the kernel and I have tried many
differ>>
>> ent
>>
>>> scenarios:
>>
>>>
>>
>>>
>>
>>>
>>
>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param) (no
ATI card
>>
>>> installed)
>>
>>>
>>
>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>
>>>
>>
>>> 3) all scenarios with pciback as module and using echo devn >
>>
>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>
>>>
>>
>>>
>>
>>>
>>
>>> In all scenarios I can successfully see the correct PCI device in
the
>>
>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>
>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
>>
>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>
>>> Nvidia card, windows correctly identified the device and installed
the
>>
>>> driver! (latest from nvidia.com). As we all know =96 Windows wanted
a reb>>
>> oot-
>>
>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>
>>> indicating reboot required and resources unavailable.
>>
>>>
>>
>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>
>> e
>>
>>> same steps. Windows XP now boots without crash although the ATI
device st>>
>> ill
>>
>>> shows a yellow exclamation and insufficient resources.
>>
>>>
>>
>>>
>>
>>>
>>
>>> To summarise, pciback functions as expected for PCIe in Dom0 and
DomU see>>
>> s
>>
>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>
>> nd
>>
>>> devid show and are managed correctly.
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read all that there is on this subject and have come to the
>>
>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>
>>> the new Video device, the Cirrus or stdvga card is always present
which I
>>
>>> believe may be causing the problem. I have tried nographics=3D1 but
qemu->>
>> dm
>>
>>> still maps the Video BIOS.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Then I noticed that qemu-dm has the new options for =93-vga none=94
.. so>>
>>  I
>>
>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>
>> ely
>>
>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>
>> the
>>
>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>
>>> boot L
>>
>>>
>>
>>>
>>
>>>
>>
>>> To solve this and make it usable I think some development of qemu
with
>>
>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>
>> a
>>
>>> controller is necessary in order to remap the Cirrus or secondary
vga
>>
>>> adapters in memory correctly. (its also difficult to configure a
Windows
>>
>>> machine for a second vga when the primary is disabled lol.)
>>
>>>
>>
>>>
>>
>>>
>>
>>> Also, I have run strace using a qemu-dm wrapper (very verbose) and
analys>>
>> ed
>>
>>> the results =96nothing seems obvious, but i`m not an expert by far
=85.
>>
>>>
>>
>>>
>>
>>>
>>
>>> Windows has been able to support multiple VGA cards for a long
time; I
>>
>>> assume that MS is remapping these somehow?
>>
>>>
>>
>>>
>>
>>>
>>
>>> I have read in 2 different places that this has worked for others
before
>>
>>> (without the legacy patch for igfx) and with xen-unstable.hg .. why
does >>
>> it
>>
>>> not work for me ?
>>
>>>
>>
>>>
>>
>>>
>>
>>> Any help / direction to further debug would be helpful .. I think
many
>>
>>> people would like to get this one working once and for all!
>>
>>>
>>
>>>
>>
>>>
>>
>>> Tim
>>
>>>
>>
>>> _______________________________________________
>>
>>> Xen-devel mailing list
>>
>>> Xen-devel@lists.xensource.com
>>
>>> http://lists.xensource.com/xen-devel
>>
>>>
>>
>>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>>
>>


-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Alexia Benington

2009-May-13 23:45 UTC

head link

Re: Re[3]: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Most video drivers should not need to access the bios. Once the system
is up and running, the video bios has done its job. I''m not sure if
there are exceptions to this.

I believe you are referring more to the framebuffers, or video RAM. I
guess that depends on whether addresses are being hardcoded in the
driver. Will appreciate if there''s anyone who can explain if
there''s
any address translation being done in the case of Vtd.

On another note, Jean''s patch for the bios is only for the range from
C0000 - CAFFF. Typical video BIOS range is up to C7FFF. Is there any
reason for the smaller range? Is this range only specific to Intel?

- Alex

On Fri, May 1, 2009 at 4:01 PM, Sander Eikelenboom <linux@eikelenboom.it>
wrote:> For what i understand of it, the video drivers (linux or windows alike)
> expect the videobios to be in a hardcoded region, but cannot access it,
> even with vt-d/iommu.
>
> Is there a way to map regions from dom0 to domU (same or different address
> ?)
>
> I haven''t been able to seriously try it because of, guess what ..
bad RMRR
> tables. Currently waiting for a response of ASUS and AMI on this issue.
>
>
> There is also a small thread on LKML about vga passthrough issues
> http://lkml.indiana.edu/hypermail/linux/kernel/0904.3/01787.html
>
> --
> Sander
>
>
>
> Hello Alexia,
>
> Monday, March 30, 2009, 5:47:05 PM, you wrote:
>
>> Hi Tim,
>
>> It depends on your definition of success. If it helps, I''ve
attached a
>> screenshot of DomU''s device manager. Dom0 will, obviously,
lose access
>> to the display, when the gfx is handed to pciback. I''ll be
glad to
>> describe further if you require.
>
>> With nVidia Quadro, the driver sits in an idling loop. According to
>> the bugcheck error code, it''s waiting for some kind of
response from
>> the gfx. I suppose this reflects Jean''s point that some
drivers are
>> not very compatible.
>
>> I was mostly trying Jean''s patch, which he released sometime
last
>> year, into Xen unstable. I''m also curious how some have
mentioned that
>> they were able to get vga passthrough to work correctly without
>> applying the patch. It might mostly work, but you''ll still get
the
>> emulated VGA in DomU.
>
>> -Alex
>
>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>> Hi Alex,
>>>
>>>
>>>
>>> When you say you had success with the onboard igfx, how much
success? Have
>>> you ever had a passthrough VGA display from a DomU ?
>>>
>>>
>>>
>>> I have also seen the write-up too although a lot of work has been
done on
>>> the VT-d support lately and I`m hoping to raise this again on the
developer
>>> list and start to document some of the problems/issues so as we can
find a
>>> way do solve it !
>>>
>>>
>>>
>>> Thanks for the reply! It still leaves this issue unsolved though ;(
>>>
>>>
>>>
>>> Tim
>>>
>>>
>>>
>>> ________________________________
>>>
>>> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics
Passthrough to
>>>
>>>         <any> DomU
>>>
>>> From: Alexia Benington <alexbenington@gmail.com>
>>>
>>> To: Tim Moore <timothy.moore@expidas.net>
>>>
>>> Cc: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I''ve been trying to get this to work for some months,
albeit not
>>>
>>> full-time. I concur that many would like to see it working. Some
have
>>>
>>> posted that they managed to get it to work correctly, however, they
>>>
>>> almost always never reply to queries on how they got it to work, or
>>>
>>> their observations.
>>>
>>>
>>>
>>> I did manage to get rid of the yellow exclamation mark.
You''ll need to
>>>
>>> disable the emulated VGA bios, pretty much as described here
>>>
>>>
http://lists.xensource.com/archives/html/xen-devel/2008-12/msg00474.html.
>>>
>>>
>>>
>>> Even after doing so, some of the registers are not directly passed
>>>
>>> through. See
http://lists.xensource.com/archives/html/xen-devel/2009-02/msg>>>
>>> 01131.html.
>>>
>>> I''m not sure if that may affect anything.
>>>
>>>
>>>
>>> I''ve experimented with an Intel DQ45CB, nVidia Quadro NVS
290, nVidia
>>>
>>> GF9500, ATI Radion HD 2600XT. The best results I get are when using
>>>
>>> the onboard igfx as the primary, which is also being passed
through.
>>>
>>>
>>>
>>> There is also a writeup which you might want to take a look
>>>
>>> http://staff.science.uva.nl/~delaat/sne-2008-2009/p22/report.pdf.
>>>
>>>
>>>
>>> I''m also trying to get DomU''s output to the
display.
>>>
>>>
>>>
>>> -Alex
>>>
>>>
>>>
>>> 2009/3/30 Tim Moore <timothy.moore@expidas.net>:
>>>
>>>> Hi
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have been testing xen-unstable.hg (29/03/09) over the past
few days ..
>>>
>>>> trying to enable passthrough for VGA.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I am using Debian 5.0 (lenny) x64 and compiling from source
each time. Tr>>>
>>> ied
>>>
>>>> both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits =85
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58
(ASUS P6T
>>>
>>>> Deluxe) with Core i7 920 CPU.
>>>
>>>>
>>>
>>>> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line
388) and
>>>
>>>> remove the DMAR bail because of bad RMRR tables.
>>>
>>>>
>>>
>>>> I also have ATI X300 PCIe for Dom0 Console.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Using Xen parameter: iommu=3Dpassthrough,pv
>>>
>>>>
>>>
>>>> Using Dom) parameter: pciback.hide=3D(=91=85=92)
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I compiled pciback as embedded in the kernel and I have tried
many differ>>>
>>> ent
>>>
>>>> scenarios:
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> 1) Nvidia Primary (hidden with pciback.hide=3D kernel param)
(no ATI card
>>>
>>>> installed)
>>>
>>>>
>>>
>>>> 2) ATI Primary, with Nvidia hidden (with pciback.hide=3D kernel
param)
>>>
>>>>
>>>
>>>> 3) all scenarios with pciback as module and using echo devn
>
>>>
>>>> /sys/bus/pci/drivers/pciback/new_slot and bind
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> In all scenarios I can successfully see the correct PCI device
in the
>>>
>>>> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
>>>
>>>> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU
and
>>>
>>>> (although I didn=92t load a driver) the lspci =96vvv showed the
devices.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Now, I progressed to installing drivers under the Win32 DomU
against the
>>>
>>>> Nvidia card, windows correctly identified the device and
installed the
>>>
>>>> driver! (latest from nvidia.com). As we all know =96 Windows
wanted a reb>>>
>>> oot-
>>>
>>>> beforehand a quick check of the device and it has a Yellow
exclamation
>>>
>>>> indicating reboot required and resources unavailable.
>>>
>>>>
>>>
>>>> Reboot and XP starts up =85 BSOD .. nv4 driver fails .. L
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Tried with the ATI X300 Primary (no NVidia connected) and went
through th>>>
>>> e
>>>
>>>> same steps. Windows XP now boots without crash although the ATI
device st>>>
>>> ill
>>>
>>>> shows a yellow exclamation and insufficient resources.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> To summarise, pciback functions as expected for PCIe in Dom0
and DomU see>>>
>>> s
>>>
>>>> the device. All tools work fine (xm pci-attach / pci-detach /
pci-list) a>>>
>>> nd
>>>
>>>> devid show and are managed correctly.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have read all that there is on this subject and have come to
the
>>>
>>>> conclusion that this is a problem with qemu-dm and the memory
mapping for
>>>
>>>> the new Video device, the Cirrus or stdvga card is always
present which I
>>>
>>>> believe may be causing the problem. I have tried nographics=3D1
but qemu->>>
>>> dm
>>>
>>>> still maps the Video BIOS.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Then I noticed that qemu-dm has the new options for =93-vga
none=94 .. so>>>
>>>  I
>>>
>>>> built a wrapper script for qemu-dm to launch with this param.
Unfortuntat>>>
>>> ely
>>>
>>>> the DomU doesn=92t startup and the qemu log does not show any
mention of >>>
>>> the
>>>
>>>> Video memory ! so I believe the switch works, but the DomU then
fails to
>>>
>>>> boot L
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> To solve this and make it usable I think some development of
qemu with
>>>
>>>> regards to the Cirrus VGA and memory management when using more
than 1 vg>>>
>>> a
>>>
>>>> controller is necessary in order to remap the Cirrus or
secondary vga
>>>
>>>> adapters in memory correctly. (its also difficult to configure
a Windows
>>>
>>>> machine for a second vga when the primary is disabled lol.)
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Also, I have run strace using a qemu-dm wrapper (very verbose)
and analys>>>
>>> ed
>>>
>>>> the results =96nothing seems obvious, but i`m not an expert by
far =85.
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Windows has been able to support multiple VGA cards for a long
time; I
>>>
>>>> assume that MS is remapping these somehow?
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> I have read in 2 different places that this has worked for
others before
>>>
>>>> (without the legacy patch for igfx) and with xen-unstable.hg ..
why does >>>
>>> it
>>>
>>>> not work for me ?
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Any help / direction to further debug would be helpful .. I
think many
>>>
>>>> people would like to get this one working once and for all!
>>>
>>>>
>>>
>>>>
>>>
>>>>
>>>
>>>> Tim
>>>
>>>>
>>>
>>>> _______________________________________________
>>>
>>>> Xen-devel mailing list
>>>
>>>> Xen-devel@lists.xensource.com
>>>
>>>> http://lists.xensource.com/xen-devel
>>>
>>>>
>>>
>>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xensource.com
>>> http://lists.xensource.com/xen-devel
>>>
>>>
>
>
>
> --
> Best regards,
>  Sander                            mailto:linux@eikelenboom.it
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Fantu

2010-Jan-05 14:43 UTC

head link

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

I have asus p6t se (bios 0608 with vt-d enable) with core i7 920, nvidia 9800
gt and nvidia 8600 gt for passthrough.
I have installed lenny 64 bit with xen 4 from hg unstable and kernel
2.6.31.6 pv_ops from git but i was not been able to make vt-d working.
can you help me to resolve, please?

Tim Moore-13 wrote:> 
> Hi
> 
> I have been testing xen-unstable.hg (29/03/09) over the past few days ..
> trying to enable passthrough for VGA.
> 
> I am using Debian 5.0 (lenny) x64 and compiling from source each time.
> Tried both linux-2.6.18.8 and linux-2.6.27.5 from Xenbits ...
> 
> Hardware is Nvidia Geforce 8800 512mb GTS PCIe on Intel X58 (ASUS P6T
> Deluxe) with Core i7 920 CPU.
> (had to hack the xen/drviers/pci/passthrough/vtd/dmar.c (line 388) and
> remove the DMAR bail because of bad RMRR tables.
> I also have ATI X300 PCIe for Dom0 Console.
> 
> Using Xen parameter: iommu=passthrough,pv
> Using Dom) parameter: pciback.hide=(''...'')
> 
> I compiled pciback as embedded in the kernel and I have tried many
> different scenarios:
> 
> 1) Nvidia Primary (hidden with pciback.hide= kernel param) (no ATI card
> installed)
> 2) ATI Primary, with Nvidia hidden (with pciback.hide= kernel param)
> 3) all scenarios with pciback as module and using echo devn >
> /sys/bus/pci/drivers/pciback/new_slot and bind
> 
> In all scenarios I can successfully see the correct PCI device in the
> DomU(s), I have tried primarily Windows XP 32bit SP3 and also:
> linux-2.6.18.8 and linux-2.6.27.5 PV and non PV kernels as DomU and
> (although I didn''t load a driver) the lspci -vvv showed the
devices.
> 
> Now, I progressed to installing drivers under the Win32 DomU against the
> Nvidia card, windows correctly identified the device and installed the
> driver! (latest from nvidia.com). As we all know - Windows wanted a
> reboot- beforehand a quick check of the device and it has a Yellow
> exclamation indicating reboot required and resources unavailable.
> Reboot and XP starts up ... BSOD .. nv4 driver fails .. :(
> 
> Tried with the ATI X300 Primary (no NVidia connected) and went through the
> same steps. Windows XP now boots without crash although the ATI device
> still shows a yellow exclamation and insufficient resources.
> 
> To summarise, pciback functions as expected for PCIe in Dom0 and DomU sees
> the device. All tools work fine (xm pci-attach / pci-detach / pci-list)
> and devid show and are managed correctly.
> 
> I have read all that there is on this subject and have come to the
> conclusion that this is a problem with qemu-dm and the memory mapping for
> the new Video device, the Cirrus or stdvga card is always present which I
> believe may be causing the problem. I have tried nographics=1 but qemu-dm
> still maps the Video BIOS.
> 
> Then I noticed that qemu-dm has the new options for "-vga none"
.. so I
> built a wrapper script for qemu-dm to launch with this param.
> Unfortuntately the DomU doesn''t startup and the qemu log does not
show any
> mention of the Video memory ! so I believe the switch works, but the DomU
> then fails to boot :(
> 
> To solve this and make it usable I think some development of qemu with
> regards to the Cirrus VGA and memory management when using more than 1 vga
> controller is necessary in order to remap the Cirrus or secondary vga
> adapters in memory correctly. (its also difficult to configure a Windows
> machine for a second vga when the primary is disabled lol.)
> 
> Also, I have run strace using a qemu-dm wrapper (very verbose) and
> analysed the results -nothing seems obvious, but i`m not an expert by far
> ....
> 
> Windows has been able to support multiple VGA cards for a long time; I
> assume that MS is remapping these somehow?
> 
> I have read in 2 different places that this has worked for others before
> (without the legacy patch for igfx) and with xen-unstable.hg .. why does
> it not work for me ?
> 
> Any help / direction to further debug would be helpful .. I think many
> people would like to get this one working once and for all!
> 
> Tim
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 
-- 
View this message in context:
http://old.nabble.com/Memory-mapping-for-PEG-PCIe-Graphics-Passthrough-to-%3Cany%3E-DomU-tp22783371p27026626.html
Sent from the Xen - Dev mailing list archive at Nabble.com.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Pasi Kärkkäinen

2010-Jan-07 15:05 UTC

head link

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

On Tue, Jan 05, 2010 at 06:43:05AM -0800, Fantu wrote:> 
> I have asus p6t se (bios 0608 with vt-d enable) with core i7 920, nvidia
9800
> gt and nvidia 8600 gt for passthrough.
> I have installed lenny 64 bit with xen 4 from hg unstable and kernel
> 2.6.31.6 pv_ops from git but i was not been able to make vt-d working.
> can you help me to resolve, please?
> 
Please post your "xm dmesg" log.. let''s start with that.

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Fantu

2010-Jan-07 16:14 UTC

head link

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

i have grub2 also here and i have add dummy=foo before root...

xm dmesg
])
(XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000
(XEN) [VT-D]dmar.c:548: Host address width 39
(XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:374:   dmaru->address = fbfff000
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:374:   dmaru->address = fbffe000
(XEN) [VT-D]dmar.c:331:   IOAPIC: f0:1f.7
(XEN) [VT-D]dmar.c:331:   IOAPIC: 0:13.0
(XEN) [VT-D]dmar.c:386:   flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:446:   RMRR region: base_addr ec000 end_address effff
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:417: RMRR error: base_addr bf7dc000 end_address bf7dbfff
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
(XEN) PCI: Not using MMCONFIG.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2672.800 MHz processor.
(XEN) Initing memory sharing.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: ASIDs enabled. 
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM1)
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 1/2 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM1)
(XEN) CPU1: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 2/4 eip 8c000
(XEN) Initializing CPU#2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 2
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#2.
(XEN) CPU2: Thermal monitoring enabled (TM1)
(XEN) CPU2: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 3/6 eip 8c000
(XEN) Initializing CPU#3
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 3
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#3.
(XEN) CPU3: Thermal monitoring enabled (TM1)
(XEN) CPU3: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 4/1 eip 8c000
(XEN) Initializing CPU#4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#4.
(XEN) CPU4: Thermal monitoring enabled (TM1)
(XEN) CPU4: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 5/3 eip 8c000
(XEN) Initializing CPU#5
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#5.
(XEN) CPU5: Thermal monitoring enabled (TM1)
(XEN) CPU5: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 6/5 eip 8c000
(XEN) Initializing CPU#6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 2
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#6.
(XEN) CPU6: Thermal monitoring enabled (TM1)
(XEN) CPU6: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 7/7 eip 8c000
(XEN) Initializing CPU#7
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 3
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#7.
(XEN) CPU7: Thermal monitoring enabled (TM1)
(XEN) CPU7: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Total of 8 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 8 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) microcode.c:73:d32767 microcode: CPU6 resumed
(XEN) microcode.c:73:d32767 microcode: CPU4 resumed
(XEN) microcode.c:73:d32767 microcode: CPU3 resumed
(XEN) microcode.c:73:d32767 microcode: CPU5 resumed
(XEN) microcode.c:73:d32767 microcode: CPU7 resumed
(XEN) microcode.c:73:d32767 microcode: CPU2 resumed
(XEN) Brought up 8 CPUs
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x82a000
(XEN) elf_parse_binary: phdr: paddr=0x182a000 memsz=0xe5108
(XEN) elf_parse_binary: phdr: paddr=0x1910000 memsz=0x888
(XEN) elf_parse_binary: phdr: paddr=0x1911000 memsz=0x15ca0
(XEN) elf_parse_binary: phdr: paddr=0x1926ca0 memsz=0x1a9360
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x1ad0000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81926ea0
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81009000
(XEN) elf_xen_parse_note: FEATURES
"!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0x0
(XEN)     virt_offset      = 0xffffffff80000000
(XEN)     virt_kstart      = 0xffffffff81000000
(XEN)     virt_kend        = 0xffffffff81ad0000
(XEN)     virt_entry       = 0xffffffff81926ea0
(XEN)     p2m_base         = 0xffffffffffffffff
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1ad0000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   00000001b4000000->00000001b8000000 (114688 pages to be
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81ad0000
(XEN)  Init. ramdisk: ffffffff81ad0000->ffffffff81ed7400
(XEN)  Phys-Mach map: ffffffff81ed8000->ffffffff81fd8000
(XEN)  Start info:    ffffffff81fd8000->ffffffff81fd84b4
(XEN)  Page tables:   ffffffff81fd9000->ffffffff81fee000
(XEN)  Boot stack:    ffffffff81fee000->ffffffff81fef000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82400000
(XEN)  ENTRY ADDRESS: ffffffff81926ea0
(XEN) Dom0 has maximum 8 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff8182a000
(XEN) elf_load_binary: phdr 1 at 0xffffffff8182a000 -> 0xffffffff8190f108
(XEN) elf_load_binary: phdr 2 at 0xffffffff81910000 -> 0xffffffff81910888
(XEN) elf_load_binary: phdr 3 at 0xffffffff81911000 -> 0xffffffff81926ca0
(XEN) elf_load_binary: phdr 4 at 0xffffffff81926ca0 -> 0xffffffff819a4000
(XEN) Scrubbing Free RAM:
.......................................................done.
(XEN) Xen trace buffers: disabled
(XEN) tmem: initialized comp=0 global-lock=0
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to switch
input to Xen)
(XEN) Freed 156kB init memory.
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=0, irq=0
(XEN) ioapic_guest_write: new_entry=00010900
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=2, irq=0
(XEN) ioapic_guest_write: new_entry=00010900
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) irq.c:1431: dom0: pirq 0 or irq 3 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 4 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 5 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 6 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 7 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 8 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 9 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 10 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 11 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 12 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 13 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 14 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 15 already mapped
(XEN) allocated vector for irq:16
(XEN) irq.c:1431: dom0: pirq 0 or irq 16 already mapped
(XEN) allocated vector for irq:17
(XEN) irq.c:1431: dom0: pirq 0 or irq 17 already mapped
(XEN) allocated vector for irq:18
(XEN) irq.c:1431: dom0: pirq 0 or irq 18 already mapped
(XEN) allocated vector for irq:20
(XEN) irq.c:1431: dom0: pirq 0 or irq 20 already mapped
(XEN) allocated vector for irq:21
(XEN) irq.c:1431: dom0: pirq 0 or irq 21 already mapped
(XEN) allocated vector for irq:22
(XEN) irq.c:1431: dom0: pirq 0 or irq 22 already mapped
(XEN) allocated vector for irq:23
(XEN) irq.c:1431: dom0: pirq 0 or irq 23 already mapped
(XEN) allocated vector for irq:24
(XEN) irq.c:1431: dom0: pirq 0 or irq 24 already mapped
(XEN) allocated vector for irq:25
(XEN) irq.c:1431: dom0: pirq 0 or irq 25 already mapped
(XEN) allocated vector for irq:26
(XEN) irq.c:1431: dom0: pirq 0 or irq 26 already mapped
(XEN) allocated vector for irq:27
(XEN) irq.c:1431: dom0: pirq 0 or irq 27 already mapped
(XEN) allocated vector for irq:28
(XEN) irq.c:1431: dom0: pirq 0 or irq 28 already mapped
(XEN) allocated vector for irq:29
(XEN) irq.c:1431: dom0: pirq 0 or irq 29 already mapped
(XEN) allocated vector for irq:30
(XEN) irq.c:1431: dom0: pirq 0 or irq 30 already mapped
(XEN) allocated vector for irq:31
(XEN) irq.c:1431: dom0: pirq 0 or irq 31 already mapped
(XEN) allocated vector for irq:32
(XEN) irq.c:1431: dom0: pirq 0 or irq 32 already mapped
(XEN) allocated vector for irq:33
(XEN) irq.c:1431: dom0: pirq 0 or irq 33 already mapped
(XEN) allocated vector for irq:34
(XEN) irq.c:1431: dom0: pirq 0 or irq 34 already mapped
(XEN) allocated vector for irq:35
(XEN) irq.c:1431: dom0: pirq 0 or irq 35 already mapped
(XEN) allocated vector for irq:36
(XEN) irq.c:1431: dom0: pirq 0 or irq 36 already mapped
(XEN) allocated vector for irq:37
(XEN) irq.c:1431: dom0: pirq 0 or irq 37 already mapped
(XEN) allocated vector for irq:38
(XEN) irq.c:1431: dom0: pirq 0 or irq 38 already mapped
(XEN) allocated vector for irq:39
(XEN) irq.c:1431: dom0: pirq 0 or irq 39 already mapped
(XEN) allocated vector for irq:40
(XEN) irq.c:1431: dom0: pirq 0 or irq 40 already mapped
(XEN) allocated vector for irq:41
(XEN) irq.c:1431: dom0: pirq 0 or irq 41 already mapped
(XEN) allocated vector for irq:42
(XEN) irq.c:1431: dom0: pirq 0 or irq 42 already mapped
(XEN) allocated vector for irq:43
(XEN) irq.c:1431: dom0: pirq 0 or irq 43 already mapped
(XEN) allocated vector for irq:44
(XEN) irq.c:1431: dom0: pirq 0 or irq 44 already mapped
(XEN) allocated vector for irq:45
(XEN) irq.c:1431: dom0: pirq 0 or irq 45 already mapped
(XEN) allocated vector for irq:46
(XEN) irq.c:1431: dom0: pirq 0 or irq 46 already mapped
(XEN) allocated vector for irq:47
(XEN) irq.c:1431: dom0: pirq 0 or irq 47 already mapped
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=6
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) tmem_relinquish_page: failing order=3
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=0, irq=0
(XEN) ioapic_guest_write: new_entry=00000900
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) PCI add device 00:00.0
(XEN) PCI add device 00:01.0
(XEN) PCI add device 00:03.0
(XEN) PCI add device 00:07.0
(XEN) PCI add device 00:14.0
(XEN) PCI add device 00:14.1
(XEN) PCI add device 00:14.2
(XEN) PCI add device 00:14.3
(XEN) PCI add device 00:1a.0
(XEN) PCI add device 00:1a.1
(XEN) PCI add device 00:1a.2
(XEN) PCI add device 00:1a.7
(XEN) PCI add device 00:1b.0
(XEN) PCI add device 00:1c.0
(XEN) PCI add device 00:1c.2
(XEN) PCI add device 00:1c.3
(XEN) PCI add device 00:1c.4
(XEN) PCI add device 00:1d.0
(XEN) PCI add device 00:1d.1
(XEN) PCI add device 00:1d.2
(XEN) PCI add device 00:1d.7
(XEN) PCI add device 00:1e.0
(XEN) PCI add device 00:1f.0
(XEN) PCI add device 00:1f.2
(XEN) PCI add device 00:1f.3
(XEN) PCI add device 02:00.0
(XEN) PCI add device 06:00.0
(XEN) PCI add device 05:00.0
(XEN) PCI add device 04:00.0
(XEN) PCI add device 04:00.1
(XEN) allocated vector for irq:19
(XEN) Set CPU acpi_id(1) cpuid(0) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(2) cpuid(1) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(3) cpuid(2) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(4) cpuid(3) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(5) cpuid(4) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(6) cpuid(5) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(7) cpuid(6) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(8) cpuid(7) Px State info:
(XEN)   _PPC: 0
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=16, irq=16
(XEN) ioapic_guest_write: new_entry=0001a910
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=23, irq=23
(XEN) ioapic_guest_write: new_entry=0001a917
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=19, irq=19
(XEN) ioapic_guest_write: new_entry=0001a913
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=18, irq=18
(XEN) ioapic_guest_write: new_entry=0001a912
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=18, irq=18
(XEN) ioapic_guest_write: new_entry=0001a912
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!

Pasi Kärkkäinen wrote:> 
> On Tue, Jan 05, 2010 at 06:43:05AM -0800, Fantu wrote:
>> 
>> I have asus p6t se (bios 0608 with vt-d enable) with core i7 920,
nvidia
>> 9800
>> gt and nvidia 8600 gt for passthrough.
>> I have installed lenny 64 bit with xen 4 from hg unstable and kernel
>> 2.6.31.6 pv_ops from git but i was not been able to make vt-d working.
>> can you help me to resolve, please?
>> 
> 
> Please post your "xm dmesg" log.. let''s start with that.
> 
> -- Pasi
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 
-- 
View this message in context:
http://old.nabble.com/Memory-mapping-for-PEG-PCIe-Graphics-Passthrough-to-%3Cany%3E-DomU-tp22783371p27062451.html
Sent from the Xen - Dev mailing list archive at Nabble.com.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Pasi Kärkkäinen

2010-Jan-07 21:07 UTC

head link

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

On Thu, Jan 07, 2010 at 08:14:47AM -0800, Fantu wrote:> 
> i have grub2 also here and i have add dummy=foo before root...
> 
> xm dmesg
> ])
> (XEN) [VT-D]dmar.c:548: Host address width 39
> (XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
> (XEN) [VT-D]dmar.c:374:   dmaru->address = fbfff000
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1b.0
> (XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
> (XEN) [VT-D]dmar.c:374:   dmaru->address = fbffe000
> (XEN) [VT-D]dmar.c:331:   IOAPIC: f0:1f.7
> (XEN) [VT-D]dmar.c:331:   IOAPIC: 0:13.0
> (XEN) [VT-D]dmar.c:386:   flags: INCLUDE_ALL
> (XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
> (XEN) [VT-D]dmar.c:446:   RMRR region: base_addr ec000 end_address effff
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.0
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.1
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.2
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.7
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.0
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.1
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.2
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.7
> (XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
> (XEN) [VT-D]dmar.c:417: RMRR error: base_addr bf7dc000 end_address bf7dbfff
> (XEN) Failed to parse ACPI DMAR.  Disabling VT-d.
>
This looks like you have a buggy BIOS, it doesn''t have correct DMAR
RMRR table..

When RMRR address range is not in reserved memory (BIOS issue), can use
''iommu_inclusive_mapping=1'' to work around it. 

So try that options for xen.gz in grub config.

http://wiki.xensource.com/xenwiki/VTdHowTo

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Weidong Han

2010-Jan-08 02:45 UTC

head link

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Pasi Kärkkäinen wrote:

On Thu, Jan 07, 2010 at 08:14:47AM -0800, Fantu wrote:

i have grub2 also here and i have add dummy=foo before root...

xm dmesg
])
(XEN) [VT-D]dmar.c:548: Host address width 39
(XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:374:   dmaru-&gt;address = fbfff000
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:374:   dmaru-&gt;address = fbffe000
(XEN) [VT-D]dmar.c:331:   IOAPIC: f0:1f.7
(XEN) [VT-D]dmar.c:331:   IOAPIC: 0:13.0
(XEN) [VT-D]dmar.c:386:   flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:446:   RMRR region: base_addr ec000 end_address effff
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:417: RMRR error: base_addr bf7dc000 end_address bf7dbfff
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.

This looks like you have a buggy BIOS, it doesn''t have correct DMAR
RMRR table..

When RMRR address range is not in reserved memory (BIOS issue), can use
''iommu_inclusive_mapping=1'' to work around it. 

So try that options for xen.gz in grub config.

http://wiki.xensource.com/xenwiki/VTdHowTo

yes, obviously the RMRR is setting incorrectly in BIOS. The base
address shouldn''t be greater than end address.
"iommu_inclusive_mapping=1" cannot help here. You should report this
issue to the vendor and ask for a correct BIOS.

Regards,

Weidong

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Han, Weidong

2010-Jan-08 09:18 UTC

head link

RE: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Fantu,

Can you try with the attached patch? and pls post the output of "lspci
-v"?

Regards,
Weidong

From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Weidong Han
Sent: Friday, January 08, 2010 10:46 AM
To: Pasi Kärkkäinen
Cc: xen-devel@lists.xensource.com; Fantu
Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to
<any> DomU

Pasi Kärkkäinen wrote:

On Thu, Jan 07, 2010 at 08:14:47AM -0800, Fantu wrote:



i have grub2 also here and i have add dummy=foo before root...



xm dmesg

])

(XEN) [VT-D]dmar.c:548: Host address width 39

(XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:

(XEN) [VT-D]dmar.c:374:   dmaru->address = fbfff000

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1b.0

(XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:

(XEN) [VT-D]dmar.c:374:   dmaru->address = fbffe000

(XEN) [VT-D]dmar.c:331:   IOAPIC: f0:1f.7

(XEN) [VT-D]dmar.c:331:   IOAPIC: 0:13.0

(XEN) [VT-D]dmar.c:386:   flags: INCLUDE_ALL

(XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:

(XEN) [VT-D]dmar.c:446:   RMRR region: base_addr ec000 end_address effff

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.0

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.1

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.2

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.7

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.0

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.1

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.2

(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.7

(XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:

(XEN) [VT-D]dmar.c:417: RMRR error: base_addr bf7dc000 end_address bf7dbfff

(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.







This looks like you have a buggy BIOS, it doesn''t have correct DMAR
RMRR table..



When RMRR address range is not in reserved memory (BIOS issue), can use

''iommu_inclusive_mapping=1'' to work around it.



So try that options for xen.gz in grub config.



http://wiki.xensource.com/xenwiki/VTdHowTo






yes, obviously the RMRR is setting incorrectly in BIOS. The base address
shouldn''t be greater than end address.
"iommu_inclusive_mapping=1" cannot help here. You should report this
issue to the vendor and ask for a correct BIOS.

Regards,
Weidong






_______________________________________________

Xen-devel mailing list

Xen-devel@lists.xensource.com<mailto:Xen-devel@lists.xensource.com>

http://lists.xensource.com/xen-devel






_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Fantu

2010-Jan-11 14:15 UTC

head link

RE: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

I have try with patch, the problem persist, under the output of xm info, xm
dmesg and lspci -v:

lspci -v
00:00.0 Host bridge: Intel Corporation QuickPath Architecture I/O Hub to ESI
Port (rev 13)
        Subsystem: ASUSTeK Computer Inc. Device 836b
        Flags: fast devsel
        Capabilities: [60] Message Signalled Interrupts: Mask+ 64bit-
Queue=0/1 Enable-
        Capabilities: [90] Express Root Port (Slot-), MSI 00
        Capabilities: [e0] Power Management version 3
        Capabilities: [100] Advanced Error Reporting <?>
        Capabilities: [150] Access Controls <?>
        Capabilities: [160] Vendor Specific Information <?>

00:01.0 PCI bridge: Intel Corporation QuickPath Architecture I/O Hub PCI
Express Root Port 1 (rev 13) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        Capabilities: [40] Subsystem: ASUSTeK Computer Inc. Device 836b
        Capabilities: [60] Message Signalled Interrupts: Mask+ 64bit-
Queue=0/1 Enable+
        Capabilities: [90] Express Root Port (Slot+), MSI 00
        Capabilities: [e0] Power Management version 3
        Capabilities: [100] Advanced Error Reporting <?>
        Capabilities: [150] Access Controls <?>
        Capabilities: [160] Vendor Specific Information <?>
        Kernel driver in use: pcieport-driver

00:03.0 PCI bridge: Intel Corporation QuickPath Architecture I/O Hub PCI
Express Root Port 3 (rev 13) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
        I/O behind bridge: 0000b000-0000bfff
        Memory behind bridge: f8000000-fbbfffff
        Prefetchable memory behind bridge: 00000000d0000000-00000000dfffffff
        Capabilities: [40] Subsystem: ASUSTeK Computer Inc. Device 836b
        Capabilities: [60] Message Signalled Interrupts: Mask+ 64bit-
Queue=0/1 Enable+
        Capabilities: [90] Express Root Port (Slot+), MSI 00
        Capabilities: [e0] Power Management version 3
        Capabilities: [100] Advanced Error Reporting <?>
        Capabilities: [150] Access Controls <?>
        Capabilities: [160] Vendor Specific Information <?>
        Kernel driver in use: pcieport-driver

00:07.0 PCI bridge: Intel Corporation QuickPath Architecture I/O Hub PCI
Express Root Port 7 (rev 13) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
        Capabilities: [40] Subsystem: ASUSTeK Computer Inc. Device 836b
        Capabilities: [60] Message Signalled Interrupts: Mask+ 64bit-
Queue=0/1 Enable+
        Capabilities: [90] Express Root Port (Slot+), MSI 00
        Capabilities: [e0] Power Management version 3
        Capabilities: [100] Advanced Error Reporting <?>
        Capabilities: [150] Access Controls <?>
        Capabilities: [160] Vendor Specific Information <?>
        Kernel driver in use: pcieport-driver

00:14.0 PIC: Intel Corporation QuickPath Architecture I/O Hub System
Management Registers (rev 13) (prog-if 00 [8259])
        Flags: fast devsel
        Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:14.1 PIC: Intel Corporation QuickPath Architecture I/O Hub GPIO and
Scratch Pad Registers (rev 13) (prog-if 00 [8259])
        Flags: fast devsel
        Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:14.2 PIC: Intel Corporation QuickPath Architecture I/O Hub Control Status
and RAS Registers (rev 13) (prog-if 00 [8259])
        Flags: fast devsel
        Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00

00:14.3 PIC: Intel Corporation QuickPath Architecture I/O Hub Throttle
Registers (rev 13) (prog-if 00 [8259])
        Flags: fast devsel

00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI
Controller #4 (prog-if 00 [UHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 16
        I/O ports at a800 [size=32]
        Capabilities: [50] PCIe advanced features <?>
        Kernel driver in use: uhci_hcd

00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI
Controller #5 (prog-if 00 [UHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 21
        I/O ports at a880 [size=32]
        Capabilities: [50] PCIe advanced features <?>
        Kernel driver in use: uhci_hcd

00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI
Controller #6 (prog-if 00 [UHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 19
        I/O ports at ac00 [size=32]
        Capabilities: [50] PCIe advanced features <?>
        Kernel driver in use: uhci_hcd

00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI
Controller #2 (prog-if 20 [EHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 18
        Memory at f7fff000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [50] Power Management version 2
        Capabilities: [58] Debug port: BAR=1 offset=00a0
        Capabilities: [98] PCIe advanced features <?>
        Kernel driver in use: ehci_hcd

00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio
Controller
        Subsystem: ASUSTeK Computer Inc. Device 8357
        Flags: bus master, fast devsel, latency 0, IRQ 22
        Memory at f7ff8000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 2
        Capabilities: [60] Message Signalled Interrupts: Mask- 64bit+
Queue=0/0 Enable-
        Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
        Capabilities: [100] Virtual Channel <?>
        Capabilities: [130] Root Complex Link <?>
        Kernel driver in use: HDA Intel

00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express
Port 1 (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=07, subordinate=07, sec-latency=0
        Prefetchable memory behind bridge: 00000000f6f00000-00000000f6ffffff
        Capabilities: [40] Express Root Port (Slot+), MSI 00
        Capabilities: [80] Message Signalled Interrupts: Mask- 64bit-
Queue=0/0 Enable+
        Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8357
        Capabilities: [a0] Power Management version 2
        Capabilities: [100] Virtual Channel <?>
        Capabilities: [180] Root Complex Link <?>
        Kernel driver in use: pcieport-driver

00:1c.2 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express
Port 3 (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=06, subordinate=06, sec-latency=0
        I/O behind bridge: 0000e000-0000efff
        Memory behind bridge: fbe00000-fbefffff
        Prefetchable memory behind bridge: 00000000f6e00000-00000000f6efffff
        Capabilities: [40] Express Root Port (Slot+), MSI 00
        Capabilities: [80] Message Signalled Interrupts: Mask- 64bit-
Queue=0/0 Enable+
        Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8357
        Capabilities: [a0] Power Management version 2
        Capabilities: [100] Virtual Channel <?>
        Capabilities: [180] Root Complex Link <?>
        Kernel driver in use: pcieport-driver

00:1c.3 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express
Port 4 (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
        I/O behind bridge: 0000d000-0000dfff
        Memory behind bridge: fbd00000-fbdfffff
        Capabilities: [40] Express Root Port (Slot+), MSI 00
        Capabilities: [80] Message Signalled Interrupts: Mask- 64bit-
Queue=0/0 Enable+
        Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8357
        Capabilities: [a0] Power Management version 2
        Capabilities: [100] Virtual Channel <?>
        Capabilities: [180] Root Complex Link <?>
        Kernel driver in use: pcieport-driver

00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express
Port 5 (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
        I/O behind bridge: 0000c000-0000cfff
        Memory behind bridge: fbc00000-fbcfffff
        Capabilities: [40] Express Root Port (Slot+), MSI 00
        Capabilities: [80] Message Signalled Interrupts: Mask- 64bit-
Queue=0/0 Enable+
        Capabilities: [90] Subsystem: ASUSTeK Computer Inc. Device 8357
        Capabilities: [a0] Power Management version 2
        Capabilities: [100] Virtual Channel <?>
        Capabilities: [180] Root Complex Link <?>
        Kernel driver in use: pcieport-driver

00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI
Controller #1 (prog-if 00 [UHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 23
        I/O ports at a080 [size=32]
        Capabilities: [50] PCIe advanced features <?>
        Kernel driver in use: uhci_hcd

00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI
Controller #2 (prog-if 00 [UHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 19
        I/O ports at a400 [size=32]
        Capabilities: [50] PCIe advanced features <?>
        Kernel driver in use: uhci_hcd

00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI
Controller #3 (prog-if 00 [UHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 18
        I/O ports at a480 [size=32]
        Capabilities: [50] PCIe advanced features <?>
        Kernel driver in use: uhci_hcd

00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI
Controller #1 (prog-if 20 [EHCI])
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0, IRQ 23
        Memory at f7ffe000 (32-bit, non-prefetchable) [size=1K]
        Capabilities: [50] Power Management version 2
        Capabilities: [58] Debug port: BAR=1 offset=00a0
        Capabilities: [98] PCIe advanced features <?>
        Kernel driver in use: ehci_hcd

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90) (prog-if 01
[Subtractive decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=08, subordinate=08, sec-latency=32
        Capabilities: [50] Subsystem: ASUSTeK Computer Inc. Device 82d4

00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface
Controller
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, medium devsel, latency 0
        Capabilities: [e0] Vendor Specific Information <?>

00:1f.2 RAID bus controller: Intel Corporation 82801 SATA RAID Controller
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 828
        I/O ports at 9c00 [size=8]
        I/O ports at 9880 [size=4]
        I/O ports at 9800 [size=8]
        I/O ports at 9480 [size=4]
        I/O ports at 9400 [size=32]
        Memory at f7ffc000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [80] Message Signalled Interrupts: Mask- 64bit-
Queue=0/4 Enable+
        Capabilities: [70] Power Management version 3
        Capabilities: [a8] SATA HBA <?>
        Capabilities: [b0] PCIe advanced features <?>
        Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
        Subsystem: ASUSTeK Computer Inc. Device 82d4
        Flags: medium devsel, IRQ 18
        Memory at f7ffd000 (64-bit, non-prefetchable) [size=256]
        I/O ports at 0400 [size=32]

02:00.0 VGA compatible controller: nVidia Corporation Device 0640 (rev a1)
(prog-if 00 [VGA controller])
        Flags: bus master, fast devsel, latency 0, IRQ 15
        Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
        Memory at d0000000 (64-bit, prefetchable) [size=256M]
        Memory at f8000000 (64-bit, non-prefetchable) [size=32M]
        I/O ports at bc00 [size=128]
        Expansion ROM at fbb80000 [disabled] [size=512K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] Message Signalled Interrupts: Mask- 64bit+
Queue=0/0 Enable-
        Capabilities: [78] Express Endpoint, MSI 00
        Capabilities: [b4] Vendor Specific Information <?>
        Capabilities: [100] Virtual Channel <?>
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [600] Vendor Specific Information <?>

04:00.0 SATA controller: JMicron Technologies, Inc. JMicron 20360/20363 AHCI
Controller (rev 03) (prog-if 01 [AHCI 1.0])
        Subsystem: ASUSTeK Computer Inc. Device 824f
        Flags: bus master, fast devsel, latency 0, IRQ 16
        Memory at fbcfe000 (32-bit, non-prefetchable) [size=8K]
        Expansion ROM at fbce0000 [disabled] [size=64K]
        Capabilities: [68] Power Management version 2
        Capabilities: [50] Express Legacy Endpoint, MSI 01
        Kernel driver in use: ahci

04:00.1 IDE interface: JMicron Technologies, Inc. JMicron 20360/20363 AHCI
Controller (rev 03) (prog-if 85 [Master SecO PriO])
        Subsystem: ASUSTeK Computer Inc. Device 824f
        Flags: bus master, fast devsel, latency 0
        I/O ports at cc00 [size=8]
        I/O ports at c880 [size=4]
        I/O ports at c800 [size=8]
        I/O ports at c480 [size=4]
        I/O ports at c400 [size=16]
        Capabilities: [68] Power Management version 2

05:00.0 FireWire (IEEE 1394): VIA Technologies, Inc. Device 3403 (prog-if 10
[OHCI])
        Subsystem: ASUSTeK Computer Inc. Device 8384
        Flags: bus master, fast devsel, latency 0, IRQ 10
        Memory at fbdff000 (64-bit, non-prefetchable) [size=2K]
        I/O ports at d800 [size=256]
        Capabilities: [50] Power Management version 3
        Capabilities: [80] Message Signalled Interrupts: Mask+ 64bit+
Queue=0/0 Enable-
        Capabilities: [98] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting <?>
        Capabilities: [130] Device Serial Number 12-90-bd-ff-ff-8c-1e-00

06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B
PCI Express Gigabit Ethernet controller (rev 02)
        Subsystem: ASUSTeK Computer Inc. Device 82c6
        Flags: bus master, fast devsel, latency 0, IRQ 827
        I/O ports at e800 [size=256]
        Memory at fbeff000 (64-bit, non-prefetchable) [size=4K]
        Memory at f6ef0000 (64-bit, prefetchable) [size=64K]
        Expansion ROM at fbec0000 [disabled] [size=128K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] Message Signalled Interrupts: Mask- 64bit+
Queue=0/0 Enable+
        Capabilities: [70] Express Endpoint, MSI 01
        Capabilities: [b0] MSI-X: Enable- Mask- TabSize=2
        Capabilities: [d0] Vital Product Data <?>
        Capabilities: [100] Advanced Error Reporting <?>
        Capabilities: [140] Virtual Channel <?>
        Capabilities: [160] Device Serial Number 00-e0-4c-68-00-00-00-03
        Kernel driver in use: r8169

xm info
host                   : vfarm
release                : 2.6.31.6-xen
version                : #2 SMP Mon Jan 11 14:33:17 CET 2010
machine                : x86_64
nr_cpus                : 8
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 2
cpu_mhz                : 2672
hw_caps                :
bfebfbff:28100800:00000000:00001b40:0098e3bd:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 6108
free_memory            : 5546
node_to_cpu            : node0:0-7
node_to_memory         : node0:5546
node_to_dma32_mem      : node0:2996
xen_major              : 4
xen_minor              : 0
xen_extra              : .0-rc1-pre
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : Sat Jan 09 08:14:44 2010 +0000 20779:596f7893b5ac
xen_commandline        : dom0_mem=512M iommu=1 iommu_inclusive_mapping=1
cc_compiler            : gcc version 4.3.2 (Debian 4.3.2-1.1) 
cc_compile_by          : root
cc_compile_domain      : 
cc_compile_date        : Mon Jan 11 14:25:59 CET 2010
xend_config_format     : 4
vfarm:~# xm dmesg
0x08] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x09] address[0xfec8a000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec8a000, GSI 24-47
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) ACPI: HPET id: 0x8086a301 base: 0xfed00000
(XEN) [VT-D]dmar.c:579: Host address width 39
(XEN) [VT-D]dmar.c:588: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:374:   dmaru->address = fbfff000
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:588: found ACPI_DMAR_DRHD:
(XEN) [VT-D]dmar.c:374:   dmaru->address = fbffe000
(XEN) [VT-D]dmar.c:331:   IOAPIC: f0:1f.7
(XEN) [VT-D]dmar.c:331:   IOAPIC: 0:13.0
(XEN) [VT-D]dmar.c:386:   flags: INCLUDE_ALL
(XEN) [VT-D]dmar.c:592: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:438:   RMRR region: base_addr ec000 end_address effff
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:592: found ACPI_DMAR_RMRR:
(XEN) [VT-D]dmar.c:425:   RMRR address range not in reserved memory base
bf7dc000 end = bf7dbfff; iommu_inclusive_mapping=1 parameter may be needed.
(XEN) [VT-D]dmar.c:438:   RMRR region: base_addr bf7dc000 end_address
bf7dbfff
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:479:   The RMRR (bf7dc000, bf7dbfff) is incorrect!
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
(XEN) PCI: Not using MMCONFIG.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2672.822 MHz processor.
(XEN) Initing memory sharing.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: ASIDs enabled. 
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging detected.
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM1)
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 1/2 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM1)
(XEN) CPU1: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 2/4 eip 8c000
(XEN) Initializing CPU#2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 2
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#2.
(XEN) CPU2: Thermal monitoring enabled (TM1)
(XEN) CPU2: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 3/6 eip 8c000
(XEN) Initializing CPU#3
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 3
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#3.
(XEN) CPU3: Thermal monitoring enabled (TM1)
(XEN) CPU3: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 4/1 eip 8c000
(XEN) Initializing CPU#4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#4.
(XEN) CPU4: Thermal monitoring enabled (TM1)
(XEN) CPU4: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 5/3 eip 8c000
(XEN) Initializing CPU#5
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#5.
(XEN) CPU5: Thermal monitoring enabled (TM1)
(XEN) CPU5: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 6/5 eip 8c000
(XEN) Initializing CPU#6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 2
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#6.
(XEN) CPU6: Thermal monitoring enabled (TM1)
(XEN) CPU6: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Booting processor 7/7 eip 8c000
(XEN) Initializing CPU#7
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 8192K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 3
(XEN) HVM: ASIDs enabled. 
(XEN) Intel machine check reporting enabled on CPU#7.
(XEN) CPU7: Thermal monitoring enabled (TM1)
(XEN) CPU7: Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz stepping 05
(XEN) Total of 8 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 8 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) microcode.c:73:d32767 microcode: CPU6 resumed
(XEN) Brought up 8 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU2 resumed
(XEN) microcode.c:73:d32767 microcode: CPU7 resumed
(XEN) microcode.c:73:d32767 microcode: CPU5 resumed
(XEN) microcode.c:73:d32767 microcode: CPU4 resumed
(XEN) microcode.c:73:d32767 microcode: CPU3 resumed
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x829000
(XEN) elf_parse_binary: phdr: paddr=0x1829000 memsz=0xe6108
(XEN) elf_parse_binary: phdr: paddr=0x1910000 memsz=0x888
(XEN) elf_parse_binary: phdr: paddr=0x1911000 memsz=0x15ca0
(XEN) elf_parse_binary: phdr: paddr=0x1926ca0 memsz=0x1a9360
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x1ad0000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81926ea0
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81009000
(XEN) elf_xen_parse_note: FEATURES
"!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0x0
(XEN)     virt_offset      = 0xffffffff80000000
(XEN)     virt_kstart      = 0xffffffff81000000
(XEN)     virt_kend        = 0xffffffff81ad0000
(XEN)     virt_entry       = 0xffffffff81926ea0
(XEN)     p2m_base         = 0xffffffffffffffff
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1ad0000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   00000001b4000000->00000001b8000000 (114688 pages to be
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff81ad0000
(XEN)  Init. ramdisk: ffffffff81ad0000->ffffffff81ed7400
(XEN)  Phys-Mach map: ffffffff81ed8000->ffffffff81fd8000
(XEN)  Start info:    ffffffff81fd8000->ffffffff81fd84b4
(XEN)  Page tables:   ffffffff81fd9000->ffffffff81fee000
(XEN)  Boot stack:    ffffffff81fee000->ffffffff81fef000
(XEN)  TOTAL:         ffffffff80000000->ffffffff82400000
(XEN)  ENTRY ADDRESS: ffffffff81926ea0
(XEN) Dom0 has maximum 8 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff81829000
(XEN) elf_load_binary: phdr 1 at 0xffffffff81829000 -> 0xffffffff8190f108
(XEN) elf_load_binary: phdr 2 at 0xffffffff81910000 -> 0xffffffff81910888
(XEN) elf_load_binary: phdr 3 at 0xffffffff81911000 -> 0xffffffff81926ca0
(XEN) elf_load_binary: phdr 4 at 0xffffffff81926ca0 -> 0xffffffff819a4000
(XEN) Scrubbing Free RAM:
.......................................................done.
(XEN) Xen trace buffers: disabled
(XEN) tmem: initialized comp=0 global-lock=0
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to switch
input to Xen)
(XEN) Freed 156kB init memory.
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=0, irq=0
(XEN) ioapic_guest_write: new_entry=00010900
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=2, irq=0
(XEN) ioapic_guest_write: new_entry=00010900
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) irq.c:1431: dom0: pirq 0 or irq 3 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 4 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 5 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 6 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 7 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 8 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 9 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 10 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 11 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 12 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 13 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 14 already mapped
(XEN) irq.c:1431: dom0: pirq 0 or irq 15 already mapped
(XEN) allocated vector for irq:16
(XEN) irq.c:1431: dom0: pirq 0 or irq 16 already mapped
(XEN) allocated vector for irq:17
(XEN) irq.c:1431: dom0: pirq 0 or irq 17 already mapped
(XEN) allocated vector for irq:18
(XEN) irq.c:1431: dom0: pirq 0 or irq 18 already mapped
(XEN) allocated vector for irq:19
(XEN) irq.c:1431: dom0: pirq 0 or irq 19 already mapped
(XEN) allocated vector for irq:20
(XEN) irq.c:1431: dom0: pirq 0 or irq 20 already mapped
(XEN) allocated vector for irq:21
(XEN) irq.c:1431: dom0: pirq 0 or irq 21 already mapped
(XEN) allocated vector for irq:22
(XEN) irq.c:1431: dom0: pirq 0 or irq 22 already mapped
(XEN) allocated vector for irq:23
(XEN) irq.c:1431: dom0: pirq 0 or irq 23 already mapped
(XEN) allocated vector for irq:24
(XEN) irq.c:1431: dom0: pirq 0 or irq 24 already mapped
(XEN) allocated vector for irq:25
(XEN) irq.c:1431: dom0: pirq 0 or irq 25 already mapped
(XEN) allocated vector for irq:26
(XEN) irq.c:1431: dom0: pirq 0 or irq 26 already mapped
(XEN) allocated vector for irq:27
(XEN) irq.c:1431: dom0: pirq 0 or irq 27 already mapped
(XEN) allocated vector for irq:28
(XEN) irq.c:1431: dom0: pirq 0 or irq 28 already mapped
(XEN) allocated vector for irq:29
(XEN) irq.c:1431: dom0: pirq 0 or irq 29 already mapped
(XEN) allocated vector for irq:30
(XEN) irq.c:1431: dom0: pirq 0 or irq 30 already mapped
(XEN) allocated vector for irq:31
(XEN) irq.c:1431: dom0: pirq 0 or irq 31 already mapped
(XEN) allocated vector for irq:32
(XEN) irq.c:1431: dom0: pirq 0 or irq 32 already mapped
(XEN) allocated vector for irq:33
(XEN) irq.c:1431: dom0: pirq 0 or irq 33 already mapped
(XEN) allocated vector for irq:34
(XEN) irq.c:1431: dom0: pirq 0 or irq 34 already mapped
(XEN) allocated vector for irq:35
(XEN) irq.c:1431: dom0: pirq 0 or irq 35 already mapped
(XEN) allocated vector for irq:36
(XEN) irq.c:1431: dom0: pirq 0 or irq 36 already mapped
(XEN) allocated vector for irq:37
(XEN) irq.c:1431: dom0: pirq 0 or irq 37 already mapped
(XEN) allocated vector for irq:38
(XEN) irq.c:1431: dom0: pirq 0 or irq 38 already mapped
(XEN) allocated vector for irq:39
(XEN) irq.c:1431: dom0: pirq 0 or irq 39 already mapped
(XEN) allocated vector for irq:40
(XEN) irq.c:1431: dom0: pirq 0 or irq 40 already mapped
(XEN) allocated vector for irq:41
(XEN) irq.c:1431: dom0: pirq 0 or irq 41 already mapped
(XEN) allocated vector for irq:42
(XEN) irq.c:1431: dom0: pirq 0 or irq 42 already mapped
(XEN) allocated vector for irq:43
(XEN) irq.c:1431: dom0: pirq 0 or irq 43 already mapped
(XEN) allocated vector for irq:44
(XEN) irq.c:1431: dom0: pirq 0 or irq 44 already mapped
(XEN) allocated vector for irq:45
(XEN) irq.c:1431: dom0: pirq 0 or irq 45 already mapped
(XEN) allocated vector for irq:46
(XEN) irq.c:1431: dom0: pirq 0 or irq 46 already mapped
(XEN) allocated vector for irq:47
(XEN) irq.c:1431: dom0: pirq 0 or irq 47 already mapped
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=0, irq=0
(XEN) ioapic_guest_write: new_entry=00000900
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) PCI add device 00:00.0
(XEN) PCI add device 00:01.0
(XEN) PCI add device 00:03.0
(XEN) PCI add device 00:07.0
(XEN) PCI add device 00:14.0
(XEN) PCI add device 00:14.1
(XEN) PCI add device 00:14.2
(XEN) PCI add device 00:14.3
(XEN) PCI add device 00:1a.0
(XEN) PCI add device 00:1a.1
(XEN) PCI add device 00:1a.2
(XEN) PCI add device 00:1a.7
(XEN) PCI add device 00:1b.0
(XEN) PCI add device 00:1c.0
(XEN) PCI add device 00:1c.2
(XEN) PCI add device 00:1c.3
(XEN) PCI add device 00:1c.4
(XEN) PCI add device 00:1d.0
(XEN) PCI add device 00:1d.1
(XEN) PCI add device 00:1d.2
(XEN) PCI add device 00:1d.7
(XEN) PCI add device 00:1e.0
(XEN) PCI add device 00:1f.0
(XEN) PCI add device 00:1f.2
(XEN) PCI add device 00:1f.3
(XEN) PCI add device 02:00.0
(XEN) PCI add device 06:00.0
(XEN) PCI add device 05:00.0
(XEN) PCI add device 04:00.0
(XEN) PCI add device 04:00.1
(XEN) Set CPU acpi_id(1) cpuid(0) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(2) cpuid(1) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(3) cpuid(2) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(4) cpuid(3) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(5) cpuid(4) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(6) cpuid(5) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(7) cpuid(6) Px State info:
(XEN)   _PPC: 0
(XEN) Set CPU acpi_id(8) cpuid(7) Px State info:
(XEN)   _PPC: 0
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=16, irq=16
(XEN) ioapic_guest_write: new_entry=0001a910
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=23, irq=23
(XEN) ioapic_guest_write: new_entry=0001a917
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=19, irq=19
(XEN) ioapic_guest_write: new_entry=0001a913
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=18, irq=18
(XEN) ioapic_guest_write: new_entry=0001a912
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2291: 
(XEN) ioapic_guest_write: apic=0, pin=18, irq=18
(XEN) ioapic_guest_write: new_entry=0001a912
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!

Han, Weidong wrote:> 
> Fantu,
> 
> Can you try with the attached patch? and pls post the output of "lspci
> -v"?
> 
> Regards,
> Weidong
> 
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Weidong Han
> Sent: Friday, January 08, 2010 10:46 AM
> To: Pasi Kärkkäinen
> Cc: xen-devel@lists.xensource.com; Fantu
> Subject: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough
> to <any> DomU
> 
> Pasi Kärkkäinen wrote:
> 
> On Thu, Jan 07, 2010 at 08:14:47AM -0800, Fantu wrote:
> 
> 
> 
> i have grub2 also here and i have add dummy=foo before root...
> 
> 
> 
> xm dmesg
> 
> ])
> 
> (XEN) [VT-D]dmar.c:548: Host address width 39
> 
> (XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
> 
> (XEN) [VT-D]dmar.c:374:   dmaru->address = fbfff000
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1b.0
> 
> (XEN) [VT-D]dmar.c:557: found ACPI_DMAR_DRHD:
> 
> (XEN) [VT-D]dmar.c:374:   dmaru->address = fbffe000
> 
> (XEN) [VT-D]dmar.c:331:   IOAPIC: f0:1f.7
> 
> (XEN) [VT-D]dmar.c:331:   IOAPIC: 0:13.0
> 
> (XEN) [VT-D]dmar.c:386:   flags: INCLUDE_ALL
> 
> (XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
> 
> (XEN) [VT-D]dmar.c:446:   RMRR region: base_addr ec000 end_address effff
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.0
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.1
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.2
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1d.7
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.0
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.1
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.2
> 
> (XEN) [VT-D]dmar.c:326:   endpoint: 0:1a.7
> 
> (XEN) [VT-D]dmar.c:561: found ACPI_DMAR_RMRR:
> 
> (XEN) [VT-D]dmar.c:417: RMRR error: base_addr bf7dc000 end_address
> bf7dbfff
> 
> (XEN) Failed to parse ACPI DMAR.  Disabling VT-d.
> 
> 
> 
> 
> 
> 
> 
> This looks like you have a buggy BIOS, it doesn''t have correct
DMAR RMRR
> table..
> 
> 
> 
> When RMRR address range is not in reserved memory (BIOS issue), can use
> 
> ''iommu_inclusive_mapping=1'' to work around it.
> 
> 
> 
> So try that options for xen.gz in grub config.
> 
> 
> 
> http://wiki.xensource.com/xenwiki/VTdHowTo
> 
> 
> 
> 
> 
> 
> yes, obviously the RMRR is setting incorrectly in BIOS. The base address
> shouldn''t be greater than end address.
"iommu_inclusive_mapping=1" cannot
> help here. You should report this issue to the vendor and ask for a
> correct BIOS.
> 
> Regards,
> Weidong
> 
> 
> 
> 
> 
> 
> _______________________________________________
> 
> Xen-devel mailing list
> 
> Xen-devel@lists.xensource.com<mailto:Xen-devel@lists.xensource.com>
> 
> http://lists.xensource.com/xen-devel
> 
> 
> 
> 
>  
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 
-- 
View this message in context:
http://old.nabble.com/Memory-mapping-for-PEG-PCIe-Graphics-Passthrough-to-%3Cany%3E-DomU-tp22783371p27111077.html
Sent from the Xen - Dev mailing list archive at Nabble.com.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Mar 2009 - Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

[Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

[Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to and DomU

RE: Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

[Xen-devel] RE: VGA Passthrough

Re[3]: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: Re[3]: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

Re: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

RE: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU

RE: [Xen-devel] Memory mapping for PEG/PCIe Graphics Passthrough to <any> DomU