Hi all I''m trying to enable VGA passthrough in linux domU without success. I want to pass through AMD Radeon HD 4850 - 03:00.0 VGA compatible controller: Advanced Micro Devices [AMD] nee ATI RV770 [Radeon HD 4850] as a _primary_ VGA for the VM. Nothing is showing up on the monitor that I have connected to graphics card''s HDMI port. I have IOMMU enabled. I searched mailing lists and I don''t know what to think about it. There is some information about ATI cards passthrough as primary VGA, some patches were posted but that was long time ago so my first question is: Is passthrough of ATI devices as primary VGA supposed to work now in Xen 4.2.1 without patches? My second question is: How can i debug the problem? I can''t access domU through SSH after it''s created. When I set gfx_passthru to 0 it''s starting normally. How can I tell what went wrong? When I create domU I get this in logs: dm-command: hot insert pass-through pci dev register_real_device: Assigning real physical device 03:00.0 ... register_real_device: Disable MSI translation via per device option register_real_device: Disable power management pt_iomul_init: Error: pt_iomul_init can''t open file /dev/xen/pci_iomul: No such file or directory: 0x3:0x0.0x0 pt_register_regions: IO region registered (size=0x10000000 base_addr=0xd000000c) pt_register_regions: IO region registered (size=0x00010000 base_addr=0xfe7f0004) pt_register_regions: IO region registered (size=0x00000100 base_addr=0x0000d001) pt_register_regions: Expansion ROM registered (size=0x00020000 base_addr=0xfe7c0002) pci_intx: intx=1 register_real_device: Real physical device 03:00.0 registered successfuly! I''m using Xen 4.2.1 and xl toolstack. I have the following in VM config file: builder = "hvm" gfx_passthru = 1 pci = [''03:00.0'',''03:00.1''] vm config file: full domU log: http://pastebin.com/raw.php?i=rc9xWcDJ part of lspci -vvv: http://pastebin.com/raw.php?i=wZXcJU7j xl info: http://pastebin.com/raw.php?i=zDrdNm9u xl dmesg: http://pastebin.com/raw.php?i=SmJ7MrB1 Please help.
Is there any special reason you use primary passthrough? because using the ati vga as secondary is A LOT easier and simpler (and as stable and performant). To be honest, i never got primary passthrough to work with any of my radeon cards, but secondary works quite well for 2,5 years now. So my advice: use secondary, boot and install domU normally, then install the catalyst driver and reboot. Within the start-process, it will automatically switch to the right vga near the end of the boot and as an additional benefit you can use the vnc view as additional input device if you for example don''t have pci-passthrough for input devices or want a remote interface anyway (with for example a mediacenter). 2013/2/27 povder <povder@gmail.com>:> Hi all > > I''m trying to enable VGA passthrough in linux domU without success. I > want to pass through AMD Radeon HD 4850 - 03:00.0 VGA compatible > controller: Advanced Micro Devices [AMD] nee ATI RV770 [Radeon HD > 4850] as a _primary_ VGA for the VM. Nothing is showing up on the > monitor that I have connected to graphics card''s HDMI port. I have > IOMMU enabled. I searched mailing lists and I don''t know what to think > about it. There is some information about ATI cards passthrough as > primary VGA, some patches were posted but that was long time ago so my > first question is: > Is passthrough of ATI devices as primary VGA supposed to work now in > Xen 4.2.1 without patches? > > My second question is: > How can i debug the problem? I can''t access domU through SSH after > it''s created. When I set gfx_passthru to 0 it''s starting normally. How > can I tell what went wrong? > > When I create domU I get this in logs: > dm-command: hot insert pass-through pci dev > register_real_device: Assigning real physical device 03:00.0 ... > register_real_device: Disable MSI translation via per device option > register_real_device: Disable power management > pt_iomul_init: Error: pt_iomul_init can''t open file > /dev/xen/pci_iomul: No such file or directory: 0x3:0x0.0x0 > pt_register_regions: IO region registered (size=0x10000000 base_addr=0xd000000c) > pt_register_regions: IO region registered (size=0x00010000 base_addr=0xfe7f0004) > pt_register_regions: IO region registered (size=0x00000100 base_addr=0x0000d001) > pt_register_regions: Expansion ROM registered (size=0x00020000 > base_addr=0xfe7c0002) > pci_intx: intx=1 > register_real_device: Real physical device 03:00.0 registered successfuly! > > I''m using Xen 4.2.1 and xl toolstack. I have the following in VM config file: > > builder = "hvm" > gfx_passthru = 1 > pci = [''03:00.0'',''03:00.1''] > > vm config file: > full domU log: http://pastebin.com/raw.php?i=rc9xWcDJ > part of lspci -vvv: http://pastebin.com/raw.php?i=wZXcJU7j > xl info: http://pastebin.com/raw.php?i=zDrdNm9u > xl dmesg: http://pastebin.com/raw.php?i=SmJ7MrB1 > > Please help. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users
Thanks for your reply. I guess I''ll give up the idea of passing through Radeon VGA as primary card. I was trying to pass through it as primary because I couldn''t get it to work as secondary because I wasn''t using catalyst drivers - now it''s working with catalyst drivers :) I have another problem now, but it''s probably not xen related. When I run glxinfo I get: X Error of failed request: BadRequest (invalid request code or no such operation) Major opcode of failed request: 139 (ATIFGLEXTENSION) Minor opcode of failed request: 66 () Serial number of failed request: 13 Current serial number in output stream: 13
Sounds like some wiki editing was needed.... So I added a note "Remember to install the graphics drivers in the DomU first, before expecting secondary passthrough to work." to these 2 pages: http://wiki.xen.org/wiki/Xen_VGA_Passthrough_Tested_Adapters http://wiki.xen.org/wiki/Xen_VGA_Passthrough#Using_Xen_VGA_graphics_passthru This page http://wiki.xen.org/wiki/SecondaryGPUPassthrough already said "After you have a defined stable DomU, enable the "pci" in the DomU config and install the necessary drivers. " but that''s confusing, saying to add pci, then install drivers, so I added "Secondary passthrough may not work before installing the graphics driver." And on a side note, I have found people trying to get primary passthrough to work anyway, because Linux DomUs don''t work very well with secondary passthrough. You will see the video from the monitor plugged into the GPU, but 3d things won''t work in X. ps. I found win8 to work without installing drivers first. One time it even worked in the CD installer... to my surprise, duplicating the output to both VNC and the monitor. On 2013-02-28 10:14, povder wrote:> Thanks for your reply. I guess I''ll give up the idea of passing > through Radeon VGA as primary card. I was trying to pass through it as > primary because I couldn''t get it to work as secondary because I > wasn''t using catalyst drivers - now it''s working with catalyst drivers > :) > > I have another problem now, but it''s probably not xen related. When I > run glxinfo I get: > > X Error of failed request: BadRequest (invalid request code or no > such operation) > Major opcode of failed request: 139 (ATIFGLEXTENSION) > Minor opcode of failed request: 66 () > Serial number of failed request: 13 > Current serial number in output stream: 13 > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users-- -------------------------------------------- Peter Maloney Brockmann Consult Max-Planck-Str. 2 21502 Geesthacht Germany Tel: +49 4152 889 300 Fax: +49 4152 889 333 E-mail: peter.maloney@brockmann-consult.de Internet: http://www.brockmann-consult.de --------------------------------------------
After looking high and low for a solution for primary PCI passthrough, I found the below patch works perfectly for doing primary passthrough of my 6570 adapter (it was also fine with a 5450 I was using previously): http://lists.xen.org/archives/html/xen-devel/2012-12/msg00009.html Note that I am running Xen 4.3-unstable and Linux kernel 3.7.8 on an AMD platform. I never tried it on 4.2. One tricky part is applying the patch at the right time, as QEMU is dynamically downloaded. During "make tools" I wait until right after the git download of qemu-traditional, CTRL-Z the make session to pause it, apply the patch against qemu-traditional, and then "fg" the make session to let it continue. Since QEMU is built first in tools, you don''t have to wait long. Apparently the Radeon cards are doing some oddball things in PCI config space and one of the MMIO''s. The above patch (which is based on a much older original patch) implements these quirks, taking into account any MMIO address translation, and implements the intrinsic IO and MMIO regions for a primary VGA device. Additionally, it ensures that the VGA rom is executed properly. The patch is only good against qemu-traditional, although I think the same techniques could be readily implemented in qemu-unstable. In fact, one of their developers recently put out a patch essentially doing this: http://lists.nongnu.org/archive/html/qemu-devel/2013-02/msg02759.html The patch was worked far and away better than any of the other schemes I have seen discussed. I can boot into a clean HVM, see a regular BIOS POST, run the Win 7 install disc, and install the Catalyst drivers (although sometimes the driver install will have issues). There is no need to do any tricks with installing with the emulated Cirrus video and then rebooting without it later - you can just start right off with a passthrough setup. Running the provided script for unbinding and binding, I can repeatedly shut down Windows, get back to a functional text console, and start up the Windows HVM again without rebooting (with some solutions, this does not work, and can even take down the system, unless you do things like eject the video card in Windows, etc.). Also the accompanying HDMI audio PCI device is also passed through and streams audio successfully. With this, I can run my HVM as an HTPC (although HDCP still blocks protected content playback), which was one of my goals with this system. I am also doing passthrough of one of my USB controllers to provide me with a keyboard and mouse, and ultimately a USB HDMI-CEC adapter. If you use the above patch, I think you should be able to painlessly implement primary VGA PCI passthrough with a Radeon card (assuming you''re up for compiling Xen). If the patch is cleaned up a little more (issues include: (1) I don''t think it instantiates the 0x3C0-DF VGA IO range for non-IGD/non-ATI cards, (2) it may try to implement the VGA ranges for all cards - not just the primary (observance of the VGA/ISA enable bits may help this), and (3) there may be a few too many hard-coded values), it would be nice to see this formally incorporated into 4.3, as it works very nicely.> Is there any special reason you use primary passthrough? because using > the ati vga as secondary is A LOT easier and simpler (and as stable > and performant). To be honest, i never got primary passthrough to work > with any of my radeon cards, but secondary works quite well for 2,5 > years now. > > So my advice: use secondary, boot and install domU normally, then > install the catalyst driver and reboot. Within the start-process, it > will automatically switch to the right vga near the end of the boot > and as an additional benefit you can use the vnc view as additional > input device if you for example don''t have pci-passthrough for input > devices or want a remote interface anyway (with for example a > mediacenter). > > 2013/2/27 povder <povder@xxxxxxxxx>: > > Hi all > > > > I''m trying to enable VGA passthrough in linux domU without success. I > > want to pass through AMD Radeon HD 4850 - 03:00.0 VGA compatible > > controller: Advanced Micro Devices [AMD] nee ATI RV770 [Radeon HD > > 4850] as a _primary_ VGA for the VM. Nothing is showing up on the > > monitor that I have connected to graphics card''s HDMI port. I have > > IOMMU enabled. I searched mailing lists and I don''t know what to think > > about it. There is some information about ATI cards passthrough as > > primary VGA, some patches were posted but that was long time ago so my > > first question is: > > Is passthrough of ATI devices as primary VGA supposed to work now in > > Xen 4.2.1 without patches? > > > > My second question is: > > How can i debug the problem? I can''t access domU through SSH after > > it''s created. When I set gfx_passthru to 0 it''s starting normally. How > > can I tell what went wrong? > > > > When I create domU I get this in logs: > > dm-command: hot insert pass-through pci dev > > register_real_device: Assigning real physical device 03:00.0 ... > > register_real_device: Disable MSI translation via per device option > > register_real_device: Disable power management > > pt_iomul_init: Error: pt_iomul_init can''t open file > > /dev/xen/pci_iomul: No such file or directory: 0x3:0x0.0x0 > > pt_register_regions: IO region registered (size=0x10000000 > > base_addr=0xd000000c) > > pt_register_regions: IO region registered (size=0x00010000 > > base_addr=0xfe7f0004) > > pt_register_regions: IO region registered (size=0x00000100 > > base_addr=0x0000d001) > > pt_register_regions: Expansion ROM registered (size=0x00020000 > > base_addr=0xfe7c0002) > > pci_intx: intx=1 > > register_real_device: Real physical device 03:00.0 registered successfuly! > > > > I''m using Xen 4.2.1 and xl toolstack. I have the following in VM config file: > > > > builder = "hvm" > > gfx_passthru = 1 > > pci = [''03:00.0'',''03:00.1''] > > > > vm config file: > > full domU log: http://pastebin.com/raw.php?i=rc9xWcDJ > > part of lspci -vvv: http://pastebin.com/raw.php?i=wZXcJU7j > > xl info: http://pastebin.com/raw.php?i=zDrdNm9u > > xl dmesg: http://pastebin.com/raw.php?i=SmJ7MrB1 > > > > Please help. > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@xxxxxxxxxxxxx > > http://lists.xen.org/xen-users > > _______________________________________________ > Xen-users mailing list > Xen-users@xxxxxxxxxxxxx > http://lists.xen.org/xen-users
On Feb 27, 1:14pm, povder wrote: } Subject: [Xen-users] VGA passthrough radeon 4850 as primary card> Hi allGood morning, hope the weekend is going well for everyone. I posted the availability of the following to xen-devel but have just recently started to following xen-users as well. Since there seemed to be generic interest in this I thought it may be worthwhile to re-post here.> I''m trying to enable VGA passthrough in linux domU without > success. I want to pass through AMD Radeon HD 4850 - 03:00.0 VGA > compatible controller: Advanced Micro Devices [AMD] nee ATI RV770 > [Radeon HD 4850] as a _primary_ VGA for the VM. Nothing is showing > up on the monitor that I have connected to graphics card''s HDMI > port. I have IOMMU enabled. I searched mailing lists and I don''t > know what to think about it. There is some information about ATI > cards passthrough as primary VGA, some patches were posted but that > was long time ago so my first question is: > > Is passthrough of ATI devices as primary VGA supposed to work now in > Xen 4.2.1 without patches?We have run thousands of sessions of primary VGA pass-through using ATI cards. We currently use stock Windows 7 with no additional drivers. Patches are needed to QEMU in order to support primary pass-through. The following URL references patches which will work for the 4.1.x and 4.2.x releases: ftp://ftp.enjellic.com/pub/xen/xen-4.1.3.ati-passthrough.patch ftp://ftp.enjellic.com/pub/xen/xen-4.2.0.ati-passthrough.patch The patches can be applied and the updated qemu-dm can be dropped into place, canonically in the /usr/lib/xen/bin directory. The following is a sample configuration: --------------------------------------------------------------------------- builder=''hvm'' memory = 3072 name = "Windows" viridian = 1 vif = [ ''type=ioemu, bridge=bridge0, model=e1000'' ] acpi = 1 apic = 1 disk = [ ''phy:/dev/localvg1/winsnap,hda,w'' ] boot="c" sdl=0 vnc=0 soundhw=''ac97'' stdvga=1 gfx_passthru=1 pci=[ ''01:00.0'', ''00:1a.0'' ] --------------------------------------------------------------------------- The BDF definitions for the pci pass-through directive are for the ATI card and the USB host controller which drives the keyboard and mouse. The trick to getting this right is to automate the detach and re-attachment of the PCI devices to dom0. The following script provides a framework for doing this: ftp://ftp.enjellic.com/pub/xen/run-passthrough It should be salted according to the flavor of your particular installation. Access to the vbetool utility, which most distribution include to support suspend/resume, is desirable in order to run the BIOS-post initialization functions so the graphics card works properly upon return of control to dom0. You will want to have an SSH session open from an alternate location to whatever platform you are testing this all on. It will be the only way to control the system if something goes awry during the setup/teardown of the pass-through session. Hope the above is helpful. Have a good weekend. Greg }-- End of excerpt from povder As always, Dr. G.W. Wettstein, Ph.D. Enjellic Systems Development, LLC. 4206 N. 19th Ave. Specializing in information infra-structure Fargo, ND 58102 development. PH: 701-281-1686 FAX: 701-281-3949 EMAIL: greg@enjellic.com ------------------------------------------------------------------------------ "One uses power by grasping it lightly. To grasp with too much force is to be taken over by power, thus becoming its victim." -- Bene Gesserit Axiom
2013/3/2 Dr. Greg Wettstein <greg@wind.enjellic.com>:> > We have run thousands of sessions of primary VGA pass-through using > ATI cards. We currently use stock Windows 7 with no additional > drivers. > > Patches are needed to QEMU in order to support primary pass-through. > The following URL references patches which will work for the 4.1.x and > 4.2.x releases: > > ftp://ftp.enjellic.com/pub/xen/xen-4.1.3.ati-passthrough.patch > > ftp://ftp.enjellic.com/pub/xen/xen-4.2.0.ati-passthrough.patch > > The patches can be applied and the updated qemu-dm can be dropped into > place, canonically in the /usr/lib/xen/bin directory. > > The following is a sample configuration: > --------------------------------------------------------------------------- > builder=''hvm'' > memory = 3072 > name = "Windows" > viridian = 1 > vif = [ ''type=ioemu, bridge=bridge0, model=e1000'' ] > acpi = 1 > apic = 1 > disk = [ ''phy:/dev/localvg1/winsnap,hda,w'' ] > boot="c" > sdl=0 > vnc=0 > soundhw=''ac97'' > stdvga=1 > gfx_passthru=1 > pci=[ ''01:00.0'', ''00:1a.0'' ] > --------------------------------------------------------------------------- > > The BDF definitions for the pci pass-through directive are for the ATI > card and the USB host controller which drives the keyboard and mouse. > > The trick to getting this right is to automate the detach and > re-attachment of the PCI devices to dom0. The following script > provides a framework for doing this: > > ftp://ftp.enjellic.com/pub/xen/run-passthrough > > It should be salted according to the flavor of your particular > installation. Access to the vbetool utility, which most distribution > include to support suspend/resume, is desirable in order to run the > BIOS-post initialization functions so the graphics card works properly > upon return of control to dom0. > > You will want to have an SSH session open from an alternate location > to whatever platform you are testing this all on. It will be the only > way to control the system if something goes awry during the > setup/teardown of the pass-through session. > > Hope the above is helpful. >Thank you for this valuable information.
Hi Greg, I''m actually pretty impressed by your patch! It didn''t worked for me with primary passthrough (boot got stuck at windows loading screen after the first things like boot select displayed on primary card well), but it actually seemed to have fixed my BSOD (the atikmdag.sys 0x0116 one many people have) issue in secondary passthrough I had for a while and therefor did more for my system stability then the last like 200 unstable changesets combined ;) So in my opinion this patch should be cleaned up and definitely be included in xen-unstable (or better to say in xen-qemu) cause it really improves vga passthrough stability, which more and more people using xen for. One issue I had with this patch, though: You do an additional "#include <sys/io.h>" in pt-graphics.c which actually breaks the stubdom build (which is included when you do a ''make world'') because the stubdom build doesn''t uses the system''s include path but it''s own where io.h simply isn''t included. I don''t know much about the stubdom build process and how to fix this issue but other then this the patch (i had to clean it up a bit but this was only correcting paths for xen-unstable, i can provide my version if you want) seems fine for me. I would really like to see this beeing included in at least the unstable branches as soon as possible :) 2013/3/3 povder <povder@gmail.com>:> 2013/3/2 Dr. Greg Wettstein <greg@wind.enjellic.com>: >> >> We have run thousands of sessions of primary VGA pass-through using >> ATI cards. We currently use stock Windows 7 with no additional >> drivers. >> >> Patches are needed to QEMU in order to support primary pass-through. >> The following URL references patches which will work for the 4.1.x and >> 4.2.x releases: >> >> ftp://ftp.enjellic.com/pub/xen/xen-4.1.3.ati-passthrough.patch >> >> ftp://ftp.enjellic.com/pub/xen/xen-4.2.0.ati-passthrough.patch >> >> The patches can be applied and the updated qemu-dm can be dropped into >> place, canonically in the /usr/lib/xen/bin directory. >> >> The following is a sample configuration: >> --------------------------------------------------------------------------- >> builder=''hvm'' >> memory = 3072 >> name = "Windows" >> viridian = 1 >> vif = [ ''type=ioemu, bridge=bridge0, model=e1000'' ] >> acpi = 1 >> apic = 1 >> disk = [ ''phy:/dev/localvg1/winsnap,hda,w'' ] >> boot="c" >> sdl=0 >> vnc=0 >> soundhw=''ac97'' >> stdvga=1 >> gfx_passthru=1 >> pci=[ ''01:00.0'', ''00:1a.0'' ] >> --------------------------------------------------------------------------- >> >> The BDF definitions for the pci pass-through directive are for the ATI >> card and the USB host controller which drives the keyboard and mouse. >> >> The trick to getting this right is to automate the detach and >> re-attachment of the PCI devices to dom0. The following script >> provides a framework for doing this: >> >> ftp://ftp.enjellic.com/pub/xen/run-passthrough >> >> It should be salted according to the flavor of your particular >> installation. Access to the vbetool utility, which most distribution >> include to support suspend/resume, is desirable in order to run the >> BIOS-post initialization functions so the graphics card works properly >> upon return of control to dom0. >> >> You will want to have an SSH session open from an alternate location >> to whatever platform you are testing this all on. It will be the only >> way to control the system if something goes awry during the >> setup/teardown of the pass-through session. >> >> Hope the above is helpful. >> > > Thank you for this valuable information. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xen.org > http://lists.xen.org/xen-users
Hi all, I have Ubuntu 12.04 with xen 4.1.4 installed. I am able to passthrough 2 AMD displays, a primary and a secondary to 2 different windows 7 VMs. But the Primary VGA, I have to Eject it before reboot or Shutdown the VM. I would like to try the vfio patches. Would you mind sharing your setup and the script you use to unbind and bind devices? Also, you have kernel 3.9 to be able to use vfio right? Thanks -- View this message in context: http://xen.1045712.n5.nabble.com/VGA-passthrough-radeon-4850-as-primary-card-tp5714485p5716638.html Sent from the Xen - User mailing list archive at Nabble.com.