David Stone
2007-Dec-06 19:18 UTC
[Xen-devel] Using VT-D to grant a Windows DomU access to a PCI Express graphics card?
I''ve also been trying to get hardware-accelerate Direct3D to work in a (Windows) DomU via VT-D. I''ve got a PCI NIC passing through fine, but not a graphics carrd. The points below describe where I''m at and what is happening. I''d appreciate any help anyone could offer, and also I have these specific questions: Is PCI Express supported by VT-D and/or Xen''s VT-D support? The graphics card I''d like to pass through is PCI Express. If so, what PCI devices do I have to hide from Dom0 and pass to my Windows DomU? Do I need to do this for the "PCI Express Root Port" AND "PCI Express Port 1" PCI devices as well has the graphics card itself? What device types have been tested (informally) with Xen''s VT-D support? NICs I assume (plus NICs work fine for me anyway), anything else? Has all the testing been with a Linux HVM DomU, or also other OS DomU''s - I have a machine with a VT-D IOMMU (Dell Optiplex 755). - I''m using various Xen builds from xen-unstable. - I''ve verified that (for certain builds) Xen''s VT-D support is working for me. I used a NIC PCI device (for a couple of reasons) as a test of VT-D/Xen, and it works...I can see the actual NIC in Windows 2003 DomU''s Device Manager (not an emulated NIC nor a PV driver), and it interacts with the network fine. Furthermore I can RDP into the Windows DomU - The next step was to try passing the PCI Express graphics card (an ATI) through to the Windows DomU. - In order to save my graphics card for my Windows DomU, I tell various components (grub, Xen kernel, Linux kernel) to use the serial port instead of the graphics card. I also pciback.hide the PCI Express graphics card from Dom0 (just as I hid the PCI NIC). When booting up Xen/Dom0 the all I see on the monitor is the BIOS spash screen, "Grub Loading Stage2...", and after that just a black screen with a blinking cursor in the upper-left corner. Everything else starting with the Grub selection screen is through the serial port. - I start up my Windows DomU. For now the intent is to RDP into it and see if Windows sees the graphics card I passed through to it or not. (As stated above, this works fine if I don''t try to pass through the graphics card) But, after may 20 seconds (about the time it takes the Windows DomU to fully boot up normally), my physical machine hangs. The last thing is just an open-parenthesis on the serial console (where I have Xen directing it''s output)...I presume it is the beginning of a message from Xen that is related to the hang, because I don''t see any such message normally. lspci -nn 00:00.0 Host bridge [0600]: Intel Corporation DRAM Controller [8086:29b0] (rev 02) 00:01.0 PCI bridge [0604]: Intel Corporation PCI Express Root Port [8086:29b1] (rev 02) 00:03.0 Communication controller [0780]: Intel Corporation MEI Controller [8086:29b4] (rev 02) 00:03.2 IDE interface [0101]: Intel Corporation PT IDER Controller [8086:29b6] (rev 02) 00:03.3 Serial controller [0700]: Intel Corporation Serial KT Controller [8086:29b7] (rev 02) 00:1a.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller #4 [8086:2937] (rev 02) 00:1a.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller #5 [8086:2938] (rev 02) 00:1a.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller #2 [8086:293c] (rev 02) 00:1b.0 Audio device [0403]: Intel Corporation HD Audio Controller [8086:293e] (rev 02) 00:1c.0 PCI bridge [0604]: Intel Corporation PCI Express Port 1 [8086:2940] (rev 02) 00:1d.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller #1 [8086:2934] (rev 02) 00:1d.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller #2 [8086:2935] (rev 02) 00:1d.2 USB Controller [0c03]: Intel Corporation USB UHCI Controller #3 [8086:2936] (rev 02) 00:1d.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller #1 [8086:293a] (rev 02) 00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev 92) 00:1f.0 ISA bridge [0601]: Intel Corporation LPC Interface Controller [8086:2914] (rev 02) 00:1f.2 SATA controller [0106]: Intel Corporation 6 port SATA AHCI Controller [8086:2922] (rev 02) 00:1f.3 SMBus [0c05]: Intel Corporation SMBus Controller [8086:2930] (rev 02) 01:00.0 VGA compatible controller [0300]: ATI Technologies Inc Unknown device [1002:94c3] 03:00.0 Ethernet controller [0200]: Intel Corporation 82557/8/9 [Ethernet Pro 100] [8086:1229] (rev 05) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Han, Weidong
2007-Dec-07 02:03 UTC
RE: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card?
David Stone wrote:> I''ve also been trying to get hardware-accelerate Direct3D to work in > a (Windows) DomU via VT-D. I''ve got a PCI NIC passing through fine, > but > not a graphics carrd. The points below describe where I''m at and what > is happening. I''d appreciate any help anyone could offer, and also I > have these specific questions: > > Is PCI Express supported by VT-D and/or Xen''s VT-D support?Yes.> graphics card I''d like to pass through is PCI Express. > If so, what PCI devices do I have to hide from Dom0 and pass to my > Windows DomU? Do I need to do this for the "PCI Express Root Port" > AND "PCI Express Port 1" PCI devices as well has the graphics card > itself?Graphices card is special. Many tricky things need to be done, such as frame buffer, VGA guest bios, etc. So current VT-d in Xen doesn''t support assigning graphics card yet.> What device types have been tested (informally) with Xen''s VT-D > support? NICs I assume (plus NICs work fine for me anyway), anything > else? Has all the testing been with a Linux HVM DomU, or also other > OS DomU''s >NICs including PCI and PCIe are regularly tested, you can see test results in mailing list. Disk and USB devices should also work. Current VT-d only supports HVM guests including Windows and Linux. In addition, VT-d only works on 64-bit and 32PAE host.> > - I have a machine with a VT-D IOMMU (Dell Optiplex 755). > - I''m using various Xen builds from xen-unstable. > - I''ve verified that (for certain builds) Xen''s VT-D support is > working for me. I used a NIC PCI device (for a couple of reasons) as > a test of VT-D/Xen, and it works...I can see the actual NIC in Windows > 2003 DomU''s Device Manager (not an emulated NIC nor a PV driver), and > it interacts with the network fine. Furthermore I can RDP into the > Windows DomU > - The next step was to try passing the PCI Express graphics card (an > ATI) through to the Windows DomU. > - In order to save my graphics card for my Windows DomU, I tell > various components (grub, Xen kernel, Linux kernel) to use the serial > port instead of the graphics card. I also pciback.hide the PCI > Express graphics card from Dom0 (just as I hid the PCI NIC). When > booting up Xen/Dom0 the all I see on the monitor is the BIOS spash > screen, "Grub Loading Stage2...", and after that just a black screen > with a blinking cursor in the upper-left corner. Everything else > starting with the Grub selection screen is through the serial port. > - I start up my Windows DomU. For now the intent is to RDP into it > and see if Windows sees the graphics card I passed through to it or > not. (As stated above, this works fine if I don''t try to pass through > the graphics card) But, after may 20 seconds (about the time it takes > the Windows DomU to fully boot up normally), my physical machine > hangs. The last thing is just an open-parenthesis on the serial > console (where I have Xen directing it''s output)...I presume it is the > beginning of a message from Xen that is related to the hang, because I > don''t see any such message normally. > > lspci -nn > 00:00.0 Host bridge [0600]: Intel Corporation DRAM Controller > [8086:29b0] (rev 02) > 00:01.0 PCI bridge [0604]: Intel Corporation PCI Express Root Port > [8086:29b1] (rev 02) > 00:03.0 Communication controller [0780]: Intel Corporation MEI > Controller [8086:29b4] (rev 02) > 00:03.2 IDE interface [0101]: Intel Corporation PT IDER Controller > [8086:29b6] (rev 02) > 00:03.3 Serial controller [0700]: Intel Corporation Serial KT > Controller [8086:29b7] (rev 02) > 00:1a.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #4 [8086:2937] (rev 02) > 00:1a.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #5 [8086:2938] (rev 02) > 00:1a.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller > #2 [8086:293c] (rev 02) > 00:1b.0 Audio device [0403]: Intel Corporation HD Audio Controller > [8086:293e] (rev 02) > 00:1c.0 PCI bridge [0604]: Intel Corporation PCI Express Port 1 > [8086:2940] (rev 02) > 00:1d.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #1 [8086:2934] (rev 02) > 00:1d.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #2 [8086:2935] (rev 02) > 00:1d.2 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #3 [8086:2936] (rev 02) > 00:1d.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller > #1 [8086:293a] (rev 02) > 00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge > [8086:244e] (rev 92) > 00:1f.0 ISA bridge [0601]: Intel Corporation LPC Interface Controller > [8086:2914] (rev 02) > 00:1f.2 SATA controller [0106]: Intel Corporation 6 port SATA AHCI > Controller [8086:2922] (rev 02) > 00:1f.3 SMBus [0c05]: Intel Corporation SMBus Controller [8086:2930] > (rev 02) 01:00.0 VGA compatible controller [0300]: ATI Technologies > Inc Unknown device [1002:94c3] > 03:00.0 Ethernet controller [0200]: Intel Corporation 82557/8/9 > [Ethernet Pro 100] [8086:1229] (rev 05) > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
陈诚
2007-Dec-07 05:43 UTC
Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCI Express graphics card?
We are trying to pass a PCI-E graphics card to Vista(HVM) using the Neo-1-to-1 approach, but encountered the same problem with you. 2007/12/7, David Stone <unclestoner@gmail.com>:> > I''ve also been trying to get hardware-accelerate Direct3D to work in a > (Windows) DomU via VT-D. I''ve got a PCI NIC passing through fine, but > not a graphics carrd. The points below describe where I''m at and what > is happening. I''d appreciate any help anyone could offer, and also I > have these specific questions: > > Is PCI Express supported by VT-D and/or Xen''s VT-D support? The > graphics card I''d like to pass through is PCI Express. > If so, what PCI devices do I have to hide from Dom0 and pass to my > Windows DomU? Do I need to do this for the "PCI Express Root Port" > AND "PCI Express Port 1" PCI devices as well has the graphics card > itself? > What device types have been tested (informally) with Xen''s VT-D > support? NICs I assume (plus NICs work fine for me anyway), anything > else? Has all the testing been with a Linux HVM DomU, or also other > OS DomU''s > > > - I have a machine with a VT-D IOMMU (Dell Optiplex 755). > - I''m using various Xen builds from xen-unstable. > - I''ve verified that (for certain builds) Xen''s VT-D support is > working for me. I used a NIC PCI device (for a couple of reasons) as > a test of VT-D/Xen, and it works...I can see the actual NIC in Windows > 2003 DomU''s Device Manager (not an emulated NIC nor a PV driver), and > it interacts with the network fine. Furthermore I can RDP into the > Windows DomU > - The next step was to try passing the PCI Express graphics card (an > ATI) through to the Windows DomU. > - In order to save my graphics card for my Windows DomU, I tell > various components (grub, Xen kernel, Linux kernel) to use the serial > port instead of the graphics card. I also pciback.hide the PCI > Express graphics card from Dom0 (just as I hid the PCI NIC). When > booting up Xen/Dom0 the all I see on the monitor is the BIOS spash > screen, "Grub Loading Stage2...", and after that just a black screen > with a blinking cursor in the upper-left corner. Everything else > starting with the Grub selection screen is through the serial port. > - I start up my Windows DomU. For now the intent is to RDP into it > and see if Windows sees the graphics card I passed through to it or > not. (As stated above, this works fine if I don''t try to pass through > the graphics card) But, after may 20 seconds (about the time it takes > the Windows DomU to fully boot up normally), my physical machine > hangs. The last thing is just an open-parenthesis on the serial > console (where I have Xen directing it''s output)...I presume it is the > beginning of a message from Xen that is related to the hang, because I > don''t see any such message normally. > > lspci -nn > 00:00.0 Host bridge [0600]: Intel Corporation DRAM Controller > [8086:29b0] (rev 02) > 00:01.0 PCI bridge [0604]: Intel Corporation PCI Express Root Port > [8086:29b1] (rev 02) > 00:03.0 Communication controller [0780]: Intel Corporation MEI > Controller [8086:29b4] (rev 02) > 00:03.2 IDE interface [0101]: Intel Corporation PT IDER Controller > [8086:29b6] (rev 02) > 00:03.3 Serial controller [0700]: Intel Corporation Serial KT > Controller [8086:29b7] (rev 02) > 00:1a.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #4 [8086:2937] (rev 02) > 00:1a.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #5 [8086:2938] (rev 02) > 00:1a.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller > #2 [8086:293c] (rev 02) > 00:1b.0 Audio device [0403]: Intel Corporation HD Audio Controller > [8086:293e] (rev 02) > 00:1c.0 PCI bridge [0604]: Intel Corporation PCI Express Port 1 > [8086:2940] (rev 02) > 00:1d.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #1 [8086:2934] (rev 02) > 00:1d.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #2 [8086:2935] (rev 02) > 00:1d.2 USB Controller [0c03]: Intel Corporation USB UHCI Controller > #3 [8086:2936] (rev 02) > 00:1d.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller > #1 [8086:293a] (rev 02) > 00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge > [8086:244e] (rev 92) > 00:1f.0 ISA bridge [0601]: Intel Corporation LPC Interface Controller > [8086:2914] (rev 02) > 00:1f.2 SATA controller [0106]: Intel Corporation 6 port SATA AHCI > Controller [8086:2922] (rev 02) > 00:1f.3 SMBus [0c05]: Intel Corporation SMBus Controller [8086:2930] (rev > 02) > 01:00.0 VGA compatible controller [0300]: ATI Technologies Inc Unknown > device [1002:94c3] > 03:00.0 Ethernet controller [0200]: Intel Corporation 82557/8/9 > [Ethernet Pro 100] [8086:1229] (rev 05) > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Han, Weidong
2007-Dec-07 06:02 UTC
RE: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card?
Once more, graphics card is special, you need to do some tricky things to make it work. Randy (Weidong) ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of 陈诚 Sent: 2007年12月7日 13:43 To: David Stone; Xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card? We are trying to pass a PCI-E graphics card to Vista(HVM) using the Neo-1-to-1 approach, but encountered the same problem with you. 2007/12/7, David Stone <unclestoner@gmail.com>: I''ve also been trying to get hardware-accelerate Direct3D to work in a (Windows) DomU via VT-D. I''ve got a PCI NIC passing through fine, but not a graphics carrd. The points below describe where I''m at and what is happening. I''d appreciate any help anyone could offer, and also I have these specific questions: Is PCI Express supported by VT-D and/or Xen''s VT-D support? The graphics card I''d like to pass through is PCI Express. If so, what PCI devices do I have to hide from Dom0 and pass to my Windows DomU? Do I need to do this for the "PCI Express Root Port" AND "PCI Express Port 1" PCI devices as well has the graphics card itself? What device types have been tested (informally) with Xen''s VT-D support? NICs I assume (plus NICs work fine for me anyway), anything else? Has all the testing been with a Linux HVM DomU, or also other OS DomU''s - I have a machine with a VT-D IOMMU (Dell Optiplex 755). - I''m using various Xen builds from xen-unstable. - I''ve verified that (for certain builds) Xen''s VT-D support is working for me. I used a NIC PCI device (for a couple of reasons) as a test of VT-D/Xen, and it works...I can see the actual NIC in Windows 2003 DomU''s Device Manager (not an emulated NIC nor a PV driver), and it interacts with the network fine. Furthermore I can RDP into the Windows DomU - The next step was to try passing the PCI Express graphics card (an ATI) through to the Windows DomU. - In order to save my graphics card for my Windows DomU, I tell various components (grub, Xen kernel, Linux kernel) to use the serial port instead of the graphics card. I also pciback.hide the PCI Express graphics card from Dom0 (just as I hid the PCI NIC). When booting up Xen/Dom0 the all I see on the monitor is the BIOS spash screen, "Grub Loading Stage2...", and after that just a black screen with a blinking cursor in the upper-left corner. Everything else starting with the Grub selection screen is through the serial port. - I start up my Windows DomU. For now the intent is to RDP into it and see if Windows sees the graphics card I passed through to it or not. (As stated above, this works fine if I don''t try to pass through the graphics card) But, after may 20 seconds (about the time it takes the Windows DomU to fully boot up normally), my physical machine hangs. The last thing is just an open-parenthesis on the serial console (where I have Xen directing it''s output)...I presume it is the beginning of a message from Xen that is related to the hang, because I don''t see any such message normally. lspci -nn 00:00.0 Host bridge [0600]: Intel Corporation DRAM Controller [8086:29b0] (rev 02) 00:01.0 PCI bridge [0604]: Intel Corporation PCI Express Root Port [8086:29b1] (rev 02) 00:03.0 Communication controller [0780]: Intel Corporation MEI Controller [8086:29b4] (rev 02) 00:03.2 IDE interface [0101]: Intel Corporation PT IDER Controller [8086:29b6] (rev 02) 00:03.3 Serial controller [0700]: Intel Corporation Serial KT Controller [8086:29b7] (rev 02) 00:1a.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller #4 [8086:2937] (rev 02) 00:1a.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller #5 [8086:2938] (rev 02) 00:1a.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller #2 [8086:293c] (rev 02) 00:1b.0 Audio device [0403]: Intel Corporation HD Audio Controller [8086:293e] (rev 02) 00:1c.0 PCI bridge [0604]: Intel Corporation PCI Express Port 1 [8086:2940] (rev 02) 00:1d.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller #1 [8086:2934] (rev 02) 00: 1d.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller #2 [8086:2935] (rev 02) 00:1d.2 USB Controller [0c03]: Intel Corporation USB UHCI Controller #3 [8086:2936] (rev 02) 00:1d.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller #1 [8086:293a] (rev 02) 00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev 92) 00:1f.0 ISA bridge [0601]: Intel Corporation LPC Interface Controller [8086:2914] (rev 02) 00:1f.2 SATA controller [0106]: Intel Corporation 6 port SATA AHCI Controller [8086:2922] (rev 02) 00:1f.3 SMBus [0c05]: Intel Corporation SMBus Controller [8086:2930] (rev 02) 01:00.0 VGA compatible controller [0300]: ATI Technologies Inc Unknown device [1002:94c3] 03:00.0 Ethernet controller [0200]: Intel Corporation 82557/8/9 [Ethernet Pro 100] [8086:1229] (rev 05) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
陈诚
2007-Dec-07 06:16 UTC
Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card?
Yes, so currently we are working on it. Simply 1-to-1 map the framebuffer and the real vga bios does not work. And Some standard vga I/O ports should be passed through and the cirrus related stuff should be removed. 2007/12/7, Han, Weidong <weidong.han@intel.com>:> > Once more, graphics card is special, you need to do some tricky things > to make it work. > > Randy (Weidong) > > ------------------------------ > *From:* xen-devel-bounces@lists.xensource.com [mailto: > xen-devel-bounces@lists.xensource.com] *On Behalf Of *陈诚 > *Sent:* 2007年12月7日 13:43 > *To:* David Stone; Xen-devel@lists.xensource.com > *Subject:* Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a > PCIExpress graphics card? > > > We are trying to pass a PCI-E graphics card to Vista(HVM) > using the Neo-1-to-1 approach, but encountered the same > problem with you. > > 2007/12/7, David Stone <unclestoner@gmail.com>: > > > > I've also been trying to get hardware-accelerate Direct3D to work in a > > (Windows) DomU via VT-D. I've got a PCI NIC passing through fine, but > > not a graphics carrd. The points below describe where I'm at and what > > is happening. I'd appreciate any help anyone could offer, and also I > > have these specific questions: > > > > Is PCI Express supported by VT-D and/or Xen's VT-D support? The > > graphics card I'd like to pass through is PCI Express. > > If so, what PCI devices do I have to hide from Dom0 and pass to my > > Windows DomU? Do I need to do this for the "PCI Express Root Port" > > AND "PCI Express Port 1" PCI devices as well has the graphics card > > itself? > > What device types have been tested (informally) with Xen's VT-D > > support? NICs I assume (plus NICs work fine for me anyway), anything > > else? Has all the testing been with a Linux HVM DomU, or also other > > OS DomU's > > > > > > - I have a machine with a VT-D IOMMU (Dell Optiplex 755). > > - I'm using various Xen builds from xen-unstable. > > - I've verified that (for certain builds) Xen's VT-D support is > > working for me. I used a NIC PCI device (for a couple of reasons) as > > a test of VT-D/Xen, and it works...I can see the actual NIC in Windows > > 2003 DomU's Device Manager (not an emulated NIC nor a PV driver), and > > it interacts with the network fine. Furthermore I can RDP into the > > Windows DomU > > - The next step was to try passing the PCI Express graphics card (an > > ATI) through to the Windows DomU. > > - In order to save my graphics card for my Windows DomU, I tell > > various components (grub, Xen kernel, Linux kernel) to use the serial > > port instead of the graphics card. I also pciback.hide the PCI > > Express graphics card from Dom0 (just as I hid the PCI NIC). When > > booting up Xen/Dom0 the all I see on the monitor is the BIOS spash > > screen, "Grub Loading Stage2...", and after that just a black screen > > with a blinking cursor in the upper-left corner. Everything else > > starting with the Grub selection screen is through the serial port. > > - I start up my Windows DomU. For now the intent is to RDP into it > > and see if Windows sees the graphics card I passed through to it or > > not. (As stated above, this works fine if I don't try to pass through > > the graphics card) But, after may 20 seconds (about the time it takes > > the Windows DomU to fully boot up normally), my physical machine > > hangs. The last thing is just an open-parenthesis on the serial > > console (where I have Xen directing it's output)...I presume it is the > > beginning of a message from Xen that is related to the hang, because I > > don't see any such message normally. > > > > lspci -nn > > 00:00.0 Host bridge [0600]: Intel Corporation DRAM Controller > > [8086:29b0] (rev 02) > > 00:01.0 PCI bridge [0604]: Intel Corporation PCI Express Root Port > > [8086:29b1] (rev 02) > > 00:03.0 Communication controller [0780]: Intel Corporation MEI > > Controller [8086:29b4] (rev 02) > > 00:03.2 IDE interface [0101]: Intel Corporation PT IDER Controller > > [8086:29b6] (rev 02) > > 00:03.3 Serial controller [0700]: Intel Corporation Serial KT > > Controller [8086:29b7] (rev 02) > > 00:1a.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller > > #4 [8086:2937] (rev 02) > > 00:1a.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller > > #5 [8086:2938] (rev 02) > > 00:1a.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller > > #2 [8086:293c] (rev 02) > > 00:1b.0 Audio device [0403]: Intel Corporation HD Audio Controller > > [8086:293e] (rev 02) > > 00:1c.0 PCI bridge [0604]: Intel Corporation PCI Express Port 1 > > [8086:2940] (rev 02) > > 00:1d.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller > > #1 [8086:2934] (rev 02) > > 00: 1d.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller > > #2 [8086:2935] (rev 02) > > 00:1d.2 USB Controller [0c03]: Intel Corporation USB UHCI Controller > > #3 [8086:2936] (rev 02) > > 00:1d.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller > > #1 [8086:293a] (rev 02) > > 00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge > > [8086:244e] (rev 92) > > 00:1f.0 ISA bridge [0601]: Intel Corporation LPC Interface Controller > > [8086:2914] (rev 02) > > 00:1f.2 SATA controller [0106]: Intel Corporation 6 port SATA AHCI > > Controller [8086:2922] (rev 02) > > 00:1f.3 SMBus [0c05]: Intel Corporation SMBus Controller [8086:2930] > > (rev 02) > > 01:00.0 VGA compatible controller [0300]: ATI Technologies Inc Unknown > > device [1002:94c3] > > 03:00.0 Ethernet controller [0200]: Intel Corporation 82557/8/9 > > [Ethernet Pro 100] [8086:1229] (rev 05) > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Han, Weidong
2007-Dec-07 06:23 UTC
RE: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card?
Yes, I think you may also need to change int10 handler. Randy (Weidong) ________________________________ From: 陈诚 [mailto:concretechen@gmail.com] Sent: 2007年12月7日 14:16 To: Han, Weidong; Xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card? Yes, so currently we are working on it. Simply 1-to-1 map the framebuffer and the real vga bios does not work. And Some standard vga I/O ports should be passed through and the cirrus related stuff should be removed. 2007/12/7, Han, Weidong <weidong.han@intel.com>: Once more, graphics card is special, you need to do some tricky things to make it work. Randy (Weidong) ________________________________ From: xen-devel-bounces@lists.xensource.com [mailto: xen-devel-bounces@lists.xensource.com <mailto:xen-devel-bounces@lists.xensource.com> ] On Behalf Of 陈诚 Sent: 2007年12月7日 13:43 To: David Stone; Xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card? We are trying to pass a PCI-E graphics card to Vista(HVM) using the Neo-1-to-1 approach, but encountered the same problem with you. 2007/12/7, David Stone <unclestoner@gmail.com>: I''ve also been trying to get hardware-accelerate Direct3D to work in a (Windows) DomU via VT-D. I''ve got a PCI NIC passing through fine, but not a graphics carrd. The points below describe where I''m at and what is happening. I''d appreciate any help anyone could offer, and also I have these specific questions: Is PCI Express supported by VT-D and/or Xen''s VT-D support? The graphics card I''d like to pass through is PCI Express. If so, what PCI devices do I have to hide from Dom0 and pass to my Windows DomU? Do I need to do this for the "PCI Express Root Port" AND "PCI Express Port 1" PCI devices as well has the graphics card itself? What device types have been tested (informally) with Xen''s VT-D support? NICs I assume (plus NICs work fine for me anyway), anything else? Has all the testing been with a Linux HVM DomU, or also other OS DomU''s - I have a machine with a VT-D IOMMU (Dell Optiplex 755). - I''m using various Xen builds from xen-unstable. - I''ve verified that (for certain builds) Xen''s VT-D support is working for me. I used a NIC PCI device (for a couple of reasons) as a test of VT-D/Xen, and it works...I can see the actual NIC in Windows 2003 DomU''s Device Manager (not an emulated NIC nor a PV driver), and it interacts with the network fine. Furthermore I can RDP into the Windows DomU - The next step was to try passing the PCI Express graphics card (an ATI) through to the Windows DomU. - In order to save my graphics card for my Windows DomU, I tell various components (grub, Xen kernel, Linux kernel) to use the serial port instead of the graphics card. I also pciback.hide the PCI Express graphics card from Dom0 (just as I hid the PCI NIC). When booting up Xen/Dom0 the all I see on the monitor is the BIOS spash screen, "Grub Loading Stage2...", and after that just a black screen with a blinking cursor in the upper-left corner. Everything else starting with the Grub selection screen is through the serial port. - I start up my Windows DomU. For now the intent is to RDP into it and see if Windows sees the graphics card I passed through to it or not. (As stated above, this works fine if I don''t try to pass through the graphics card) But, after may 20 seconds (about the time it takes the Windows DomU to fully boot up normally), my physical machine hangs. The last thing is just an open-parenthesis on the serial console (where I have Xen directing it''s output)...I presume it is the beginning of a message from Xen that is related to the hang, because I don''t see any such message normally. lspci -nn 00:00.0 Host bridge [0600]: Intel Corporation DRAM Controller [8086:29b0] (rev 02) 00:01.0 PCI bridge [0604]: Intel Corporation PCI Express Root Port [8086:29b1] (rev 02) 00:03.0 Communication controller [0780]: Intel Corporation MEI Controller [8086:29b4] (rev 02) 00:03.2 IDE interface [0101]: Intel Corporation PT IDER Controller [8086:29b6] (rev 02) 00:03.3 Serial controller [0700]: Intel Corporation Serial KT Controller [8086:29b7] (rev 02) 00:1a.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller #4 [8086:2937] (rev 02) 00:1a.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller #5 [8086:2938] (rev 02) 00:1a.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller #2 [8086:293c] (rev 02) 00:1b.0 Audio device [0403]: Intel Corporation HD Audio Controller [8086:293e] (rev 02) 00:1c.0 PCI bridge [0604]: Intel Corporation PCI Express Port 1 [8086:2940] (rev 02) 00:1d.0 USB Controller [0c03]: Intel Corporation USB UHCI Controller #1 [8086:2934] (rev 02) 00: 1d.1 USB Controller [0c03]: Intel Corporation USB UHCI Controller #2 [8086:2935] (rev 02) 00:1d.2 USB Controller [0c03]: Intel Corporation USB UHCI Controller #3 [8086:2936] (rev 02) 00:1d.7 USB Controller [0c03]: Intel Corporation USB2 EHCI Controller #1 [8086:293a] (rev 02) 00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev 92) 00:1f.0 ISA bridge [0601]: Intel Corporation LPC Interface Controller [8086:2914] (rev 02) 00:1f.2 SATA controller [0106]: Intel Corporation 6 port SATA AHCI Controller [8086:2922] (rev 02) 00:1f.3 SMBus [0c05]: Intel Corporation SMBus Controller [8086:2930] (rev 02) 01:00.0 VGA compatible controller [0300]: ATI Technologies Inc Unknown device [1002:94c3] 03:00.0 Ethernet controller [0200]: Intel Corporation 82557/8/9 [Ethernet Pro 100] [8086:1229] (rev 05) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
David Stone
2007-Dec-07 13:55 UTC
Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card?
Thanks for your response.> > graphics card I''d like to pass through is PCI Express. > > If so, what PCI devices do I have to hide from Dom0 and pass to my > > Windows DomU? Do I need to do this for the "PCI Express Root Port" > > AND "PCI Express Port 1" PCI devices as well has the graphics card > > itself? > > Graphices card is special. Many tricky things need to be done, such as > frame buffer, VGA guest bios, etc. So current VT-d in Xen doesn''t > support assigning graphics card yet.I wonder if those complications (which I don''t pretend to understand) would go away if the graphics card did not have to produce any output to a locally attached monitor? My goal is not to render the Windows DomU graphics to the local monitor, but rather to just use the graphics device to do work like rendering to an off-screen buffer. I can then do whatever I want with that buffer, such as send it over the network to another machine for display. The Windows DomU would be running completely headless...I can RDP into it to administer it. Would that make it more feasible? Thanks, Dave _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Guy Zana
2007-Dec-07 14:03 UTC
RE: [Xen-devel] Using VT-D to grant a Windows DomU access to aPCIExpress graphics card?
It''s just that PCIe pass-through is not fully supported by Xen, so it is very card/driver dependant. If you are not working with the graphic''s card driver then what you describe sounds feasible and may actually work. Thanks, Guy.> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of > David Stone > Sent: Friday, December 07, 2007 3:56 PM > To: Han, Weidong > Cc: Xen Developers > Subject: Re: [Xen-devel] Using VT-D to grant a Windows DomU > access to aPCIExpress graphics card? > > Thanks for your response. > > > > graphics card I''d like to pass through is PCI Express. > > > If so, what PCI devices do I have to hide from Dom0 and > pass to my > > > Windows DomU? Do I need to do this for the "PCI Express > Root Port" > > > AND "PCI Express Port 1" PCI devices as well has the > graphics card > > > itself? > > > > Graphices card is special. Many tricky things need to be > done, such as > > frame buffer, VGA guest bios, etc. So current VT-d in Xen doesn''t > > support assigning graphics card yet. > > I wonder if those complications (which I don''t pretend to > understand) would go away if the graphics card did not have > to produce any output to a locally attached monitor? My goal > is not to render the Windows DomU graphics to the local > monitor, but rather to just use the graphics device to do > work like rendering to an off-screen buffer. I can then do > whatever I want with that buffer, such as send it over the > network to another machine for display. The Windows DomU > would be running completely headless...I can RDP into it to > administer it. > > Would that make it more feasible? > > Thanks, > Dave > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
David Stone
2007-Dec-07 15:07 UTC
Re: [Xen-devel] Using VT-D to grant a Windows DomU access to aPCIExpress graphics card?
> If you are not working with the graphic''s card driver then what you describe sounds feasible and may actually work.I don''t quite follow what you mean by "if you are notworking with the graphics card driver", could you clarify it for me? My hope would be to use the normal Direct3D API to send work to the card through the normal stack including the card''s driver; but I would specify an off-screen rendering target. Or alternatively, I would specify an on-screen rendering target but no monitor would actually attached to the card. Xen/Qemu would still present the normal Cirrus emulated graphics card to the Windows DomU (but there would be nothing to back it up so any writes to it would just be dropped?) Thanks, Dave _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Han, Weidong
2007-Dec-07 15:11 UTC
RE: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card?
David Stone wrote:> Thanks for your response. > >>> graphics card I''d like to pass through is PCI Express. >>> If so, what PCI devices do I have to hide from Dom0 and pass to my >>> Windows DomU? Do I need to do this for the "PCI Express Root Port" >>> AND "PCI Express Port 1" PCI devices as well has the graphics card >>> itself? >> >> Graphices card is special. Many tricky things need to be done, such >> as frame buffer, VGA guest bios, etc. So current VT-d in Xen doesn''t >> support assigning graphics card yet. > > I wonder if those complications (which I don''t pretend to understand) > would go away if the graphics card did not have to produce any output > to a locally attached monitor? My goal is not to render the Windows > DomU graphics to the local monitor, but rather to just use the > graphics device to do work like rendering to an off-screen buffer. I > can then do whatever I want with that buffer, such as send it over the > network to another machine for display. The Windows DomU would be > running completely headless...I can RDP into it to administer it. > > Would that make it more feasible? >I don''t think so. Without those complications, graphics card even can''t work on guest. I think displaying on local monitor is the same with displaying on remote machine in essence. Randy (Weidong) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
David Stone
2007-Dec-07 15:44 UTC
Re: [Xen-devel] Using VT-D to grant a Windows DomU access to a PCIExpress graphics card?
> >> Graphices card is special. Many tricky things need to be done, such > >> as frame buffer, VGA guest bios, etc. So current VT-d in Xen doesn''t > >> support assigning graphics card yet. > > > > I wonder if those complications (which I don''t pretend to understand) > > would go away if the graphics card did not have to produce any output > > to a locally attached monitor? My goal is not to render the Windows > > DomU graphics to the local monitor, but rather to just use the > > graphics device to do work like rendering to an off-screen buffer. I > > can then do whatever I want with that buffer, such as send it over the > > network to another machine for display. The Windows DomU would be > > running completely headless...I can RDP into it to administer it. > > > > Would that make it more feasible? > > > > I don''t think so. Without those complications, graphics card even can''t > work on guest. I think displaying on local monitor is the same with > displaying on remote machine in essence.OK. I wonder if you could you give me a brief description of these complications (why they are a problem, and what needs to be done in the Xen source to handle them)? Thanks, Dave _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Han, Weidong
2007-Dec-10 13:34 UTC
RE: [Xen-devel] Using VT-D to grant a Windows DomU access to aPCIExpress graphics card?
Graphics card assignment is complex, and not easy to implement. I am not an expert on it, I can tell you some things I know. By default, cirrus VGA is emulated for guest. If using VT-d to directly assign graphics card to guest, first should remove some cirrus VGA stuffs. It should need 1:1 mapping of frame buffer, VGA bios and VGA standard I/O ports, and permit directly access right to them. In addition, you may also need replace guest int10 and int42 handlers with host handlers. At last, if graphics card''s memory is close to or bigger than 256M, should enlarge guest e820 mmio size, because current guest mmio size is 256M. Randy (Weidong) David Stone wrote:>>>> Graphices card is special. Many tricky things need to be done, such >>>> as frame buffer, VGA guest bios, etc. So current VT-d in Xen >>>> doesn''t support assigning graphics card yet. >>> >>> I wonder if those complications (which I don''t pretend to >>> understand) would go away if the graphics card did not have to >>> produce any output to a locally attached monitor? My goal is not >>> to render the Windows DomU graphics to the local monitor, but >>> rather to just use the graphics device to do work like rendering to >>> an off-screen buffer. I can then do whatever I want with that >>> buffer, such as send it over the network to another machine for >>> display. The Windows DomU would be running completely headless...I >>> can RDP into it to administer it. >>> >>> Would that make it more feasible? >>> >> >> I don''t think so. Without those complications, graphics card even >> can''t work on guest. I think displaying on local monitor is the same >> with displaying on remote machine in essence. > > OK. I wonder if you could you give me a brief description of these > complications (why they are a problem, and what needs to be done in > the Xen source to handle them)? > > Thanks, > Dave > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
David Stone
2007-Dec-12 20:37 UTC
Re: [Xen-devel] Using VT-D to grant a Windows DomU access to aPCIExpress graphics card?
> Graphics card assignment is complex, and not easy to implement. I am not > an expert on it, I can tell you some things I know. > By default, cirrus VGA is emulated for guest. If using VT-d to directly > assign graphics card to guest, first should remove some cirrus VGA > stuffs. It should need 1:1 mapping of frame buffer, VGA bios and VGA > standard I/O ports, and permit directly access right to them. In > addition, you may also need replace guest int10 and int42 handlers with > host handlers. At last, if graphics card''s memory is close to or bigger > than 256M, should enlarge guest e820 mmio size, because current guest > mmio size is 256M. > > Randy (Weidong)Hmm...why can''t I just leave the Cirrus emulated card as it is? The Windows DomU would continue to see it and us it as the primary display device. I''ve already verified that I can hide the real graphics card from Dom0 and this works fine...there is not output from Dom0 or any DomU of course, but can connect to DomU via the serial port or ssh, and I can connect to Windows DomU via RDP. The idea is that the real PCI Express graphics card that is passed through to Windows DomU would appear to the Windows OS as a _secondary_ display display device. If one has a bare-metal OS with two graphics adapters, I assume only one of them (the "primary") is responsible for stuff like the VGA BIOS? I just want to write Direct3D code to cause the physical graphics card to render to an _off-screen_ surface...not to a framebuffer. I could then scrape the rendered bitmaps and do whatever I wanted with them. Thanks, Dave _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Han, Weidong
2007-Dec-13 05:25 UTC
RE: [Xen-devel] Using VT-D to grant a Windows DomU access toaPCIExpress graphics card?
Seems your usage model is feasible. Randy (Weidong) David Stone wrote:> Hmm...why can''t I just leave the Cirrus emulated card as it is? The > Windows DomU would continue to see it and us it as the primary display > device. I''ve already verified that I can hide the real graphics card > from Dom0 and this works fine...there is not output from Dom0 or any > DomU of course, but can connect to DomU via the serial port or ssh, > and I can connect to Windows DomU via RDP. > > The idea is that the real PCI Express graphics card that is passed > through to Windows DomU would appear to the Windows OS as a > _secondary_ display display device. If one has a bare-metal OS with > two graphics adapters, I assume only one of them (the "primary") is > responsible for stuff like the VGA BIOS? I just want to write > Direct3D code to cause the physical graphics card to render to an > _off-screen_ surface...not to a framebuffer. I could then scrape the > rendered bitmaps and do whatever I wanted with them. > > Thanks, > Dave > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel