Hi,
I have an HP workstation here with two identical AMD V5900 graphics
cards in it. I''m finding it really challenging following the
instructions to get VGA passthrough working. Thanks for the help you''ve
given me so far. The host OS is Ubuntu 12.04 LTS 64-bit with the XFCE
desktop.
The Xen install has a Windows guest and I want to pass one of the cards
through to it. The cards each appear as two devices (a graphics card and
an associated sound card that provides the HDMI audio out). So I have:
* 00:04:00.0 - Graphics card
* 00:04:00.1 - Audio device
* 00:05:00.0 - Graphics card
* 00:05:00.1 - Audio device
I swapped which device was primary in the BIOS as it made the layout of
the wiring on the back panel more logical.
My first stupid question is: How do I work out which card is the primary
and secondary to know which one to pass into the guest please?
I think I need to use gfx_passthru=1
I am already passing other PCI devices through to the guest
successfully. Say for example 00:06:00.0.
How do I write this for the conf file please. I tried things like:
gfx_passthru=1
pci=[''0000:04:00.0'',''0000:04:00.1'',''0000:06:00.0'']
and
gfx_passthru=1
pci=[''0000:04:00.0'']
pci=[''0000:04:00.1'',''0000:06:00.0'']
I''ve had no success. Are these ways of writing it equivalent or do the
PCI IDs need to be grouped in a specific way please?
I''ve seen some guides saying that I need to extract the contents of the
EEPROM and bring it into the guest and some not mention it. Do I need to
do this with an AMD card please?
I''ve also see reference to needing a copy of Xen after changeset 25070
for graphics passthrough. I know I''m running Xen 4.2.0 as downloaded
from the website but I don''t still have the source code we compiled it
from so I don''t know if it''s the right patch level. How can I
work out
if the copy I have is OK from an installed machine please?
Thanks,
Paul.
On 04/09/2013 08:49 PM, Paul Stimpson wrote:> Hi, > > I have an HP workstation here with two identical AMD V5900 graphics > cards in it. I''m finding it really challenging following the > instructions to get VGA passthrough working. Thanks for the help > you''ve given me so far. The host OS is Ubuntu 12.04 LTS 64-bit with > the XFCE desktop. >Is that a firepro? This page mentions the FirePro V5700, which I guess means it works with secondary passthrough. http://wiki.xen.org/wiki/Xen_VGA_Passthrough_Tested_Adapters#ATI.2FAMD_display_adapters> The Xen install has a Windows guest and I want to pass one of the > cards through to it. The cards each appear as two devices (a graphics > card and an associated sound card that provides the HDMI audio out). > So I have: > > * 00:04:00.0 - Graphics card > * 00:04:00.1 - Audio device > * 00:05:00.0 - Graphics card > * 00:05:00.1 - Audio device > > I swapped which device was primary in the BIOS as it made the layout > of the wiring on the back panel more logical. > > My first stupid question is: How do I work out which card is the > primary and secondary to know which one to pass into the guest please? >Not sure... there''s probably some X command that should tell you which one is the one used by X. Otherwise, you can just try unbinding the driver for one and see :) (if the screen goes black, you picked the one dom0 is using)> I think I need to use gfx_passthru=1 > > I am already passing other PCI devices through to the guest > successfully. Say for example 00:06:00.0. > > How do I write this for the conf file please. I tried things like: > > gfx_passthru=1 > pci=[''0000:04:00.0'',''0000:04:00.1'',''0000:06:00.0'']# 0 is secondary passthrough; 1 is primary passthrough gfx_passthru=0 # without the 0000: (not sure if it is a problem though) pci=[''04:00.0'',''04:00.1'',''06:00.0'']> > and > > gfx_passthru=1 > pci=[''0000:04:00.0''] > > pci=[''0000:04:00.1'',''0000:06:00.0''] > > I''ve had no success. Are these ways of writing it equivalent or do the > PCI IDs need to be grouped in a specific way please? > > I''ve seen some guides saying that I need to extract the contents of > the EEPROM and bring it into the guest and some not mention it. Do I > need to do this with an AMD card please?You may need this for primary passthrough, but not AMD secondary passthrough> > I''ve also see reference to needing a copy of Xen after changeset 25070 > for graphics passthrough. I know I''m running Xen 4.2.0 as downloaded > from the website but I don''t still have the source code we compiled it > from so I don''t know if it''s the right patch level. How can I work out > if the copy I have is OK from an installed machine please?AMD HD cards with secondary passthrough shouldn''t need any special patches, just the ordinary 4.1.4 or 4.2.1 should work fine.> > Thanks, > Paul.Also, remember to install the drivers in the guest before you expect the secondary passthrough to work.
Hi Peter, Thank you. On 09/04/13 22:11, Peter Maloney wrote:> On 04/09/2013 08:49 PM, Paul Stimpson wrote: >> Hi, >> >> I have an HP workstation here with two identical AMD V5900 graphics >> cards in it. I''m finding it really challenging following the >> instructions to get VGA passthrough working. Thanks for the help >> you''ve given me so far. The host OS is Ubuntu 12.04 LTS 64-bit with >> the XFCE desktop. >> > Is that a firepro? > This page mentions the FirePro V5700, which I guess means it works with > secondary passthrough. > http://wiki.xen.org/wiki/Xen_VGA_Passthrough_Tested_Adapters#ATI.2FAMD_display_adaptersSorry. My mistake. it is a Firepro v5700. I was talking to HP about the v5900 as they are dropping the v5700 and wanted us to consider it as an alternative.>> The Xen install has a Windows guest and I want to pass one of the >> cards through to it. The cards each appear as two devices (a graphics >> card and an associated sound card that provides the HDMI audio out). >> So I have: >> >> * 00:04:00.0 - Graphics card >> * 00:04:00.1 - Audio device >> * 00:05:00.0 - Graphics card >> * 00:05:00.1 - Audio device >> >> I swapped which device was primary in the BIOS as it made the layout >> of the wiring on the back panel more logical. >> >> My first stupid question is: How do I work out which card is the >> primary and secondary to know which one to pass into the guest please? >> > Not sure... there''s probably some X command that should tell you which > one is the one used by X. Otherwise, you can just try unbinding the > driver for one and see :) (if the screen goes black, you picked the one > dom0 is using)I looked in Xorg.0.log and it it has a * next to the 04:00.0 card and it''s listed as card0 so I think it''s probably that one.>> I think I need to use gfx_passthru=1 >> >> I am already passing other PCI devices through to the guest >> successfully. Say for example 00:06:00.0. >> >> How do I write this for the conf file please. I tried things like: >> >> gfx_passthru=1 >> pci=[''0000:04:00.0'',''0000:04:00.1'',''0000:06:00.0''] > # 0 is secondary passthrough; 1 is primary passthrough > gfx_passthru=0 > # without the 0000: (not sure if it is a problem though) > pci=[''04:00.0'',''04:00.1'',''06:00.0'']I tried this (I put the other card in /etc/modprobe.d/pciback.conf to hide it) but Xen says the card is not assignable. When I run the machine up, there is what looks like Linux boot logging on the second monitor that stays there after the machine comes up. I''m obviously doing something wrong but, right now, I''m not sure what it is. Thanks, Paul.
On 2013-04-15 13:09, Paul Stimpson wrote:> Hi Peter, >>> I am already passing other PCI devices through to the guest >>> successfully. Say for example 00:06:00.0. >>> >>> How do I write this for the conf file please. I tried things like: >>> >>> gfx_passthru=1 >>> pci=[''0000:04:00.0'',''0000:04:00.1'',''0000:06:00.0''] >> # 0 is secondary passthrough; 1 is primary passthrough >> gfx_passthru=0 >> # without the 0000: (not sure if it is a problem though) >> pci=[''04:00.0'',''04:00.1'',''06:00.0''] > > I tried this (I put the other card in /etc/modprobe.d/pciback.conf to > hide it) but Xen says the card is not assignable. When I run the > machine up, there is what looks like Linux boot logging on the second > monitor that stays there after the machine comes up. > > I''m obviously doing something wrong but, right now, I''m not sure what > it is.I''m not sure about using modprobe.d for this... modprobe would rely on the device being unbound before it would work. The boot line method works because nothing is bound before the boot. But modprobe would need an unbind too, not just a list of devices. What I use is this script I wrote (which is simple to use, because you don''t need to edit your boot lines, or rebuild your kernel): http://wiki.xen.org/wiki/Bind_lib.bash As it says on the top of that page, xl has this functionality built in. I wrote that for xm.
On 15/04/13 12:39, Peter Maloney wrote:> On 2013-04-15 13:09, Paul Stimpson wrote: >> Hi Peter, >>>> I am already passing other PCI devices through to the guest >>>> successfully. Say for example 00:06:00.0. >>>> >>>> How do I write this for the conf file please. I tried things like: >>>> >>>> gfx_passthru=1 >>>> pci=[''0000:04:00.0'',''0000:04:00.1'',''0000:06:00.0''] >>> # 0 is secondary passthrough; 1 is primary passthrough >>> gfx_passthru=0 >>> # without the 0000: (not sure if it is a problem though) >>> pci=[''04:00.0'',''04:00.1'',''06:00.0''] >> I tried this (I put the other card in /etc/modprobe.d/pciback.conf to >> hide it) but Xen says the card is not assignable. When I run the >> machine up, there is what looks like Linux boot logging on the second >> monitor that stays there after the machine comes up. >> >> I''m obviously doing something wrong but, right now, I''m not sure what >> it is. > I''m not sure about using modprobe.d for this... modprobe would rely on > the device being unbound before it would work. The boot line method > works because nothing is bound before the boot. But modprobe would need > an unbind too, not just a list of devices. > > What I use is this script I wrote (which is simple to use, because you > don''t need to edit your boot lines, or rebuild your kernel): > http://wiki.xen.org/wiki/Bind_lib.bash > > As it says on the top of that page, xl has this functionality built in. > I wrote that for xm. > >Thank you for all your help. We are using xl. I had some major progress yesterday. I did the following: * echo "0000:05:00.0" > /sys/bus/pci/devices/0000\:05\:00.0/driver/unbind * pci-assignable-add 05:00.0 * pci-attach Windows-guest 05:00.0 The card then appeared in Windows. I had to access the VM via VNC as the AMD Catalyst driver has a wonderful feature that you can''t install it unless the hardware is already in the machine and detected by the driver installer. It installed and, after a guest reboot, I got a lovely full-res Windows desktop on the right card''s output. My next challenge is to make it all work automatically. I tried adding "xen_pciback" to /etc/initramfs-tools/modules then doing an "update-initramfs -vu". After that I added "xen-pciback.passthrough=1 xen-pciback.hide=(0000:00:1a.0)(0000:00:1d.0)(0000:05:00.0)(0000:05:00.1)(0000:06:00.0)(0000:08:00.0)" to the end of GRUB_CMDLINE_XEN in /etc/default/grub and fired off an update-grub. I did this because a colleague said this should work. It didn''t so I added the matching pciback hide to /etc/modprobe.d/pciback.conf just in case as pciback was still a module. This also didn''t work. I feel like I''m making progress as I at least have it working manually. Was my colleague correct that adding the module to initramfs works or should I blow the cobwebs out of my brain and try to remember how to roll a custom kernel please? Thanks, Paul.
On 2013-04-17 11:32, Paul Stimpson wrote:> On 15/04/13 12:39, Peter Maloney wrote: >> On 2013-04-15 13:09, Paul Stimpson wrote: >>> Hi Peter, >>>>> I am already passing other PCI devices through to the guest >>>>> successfully. Say for example 00:06:00.0. >>>>> >>>>> How do I write this for the conf file please. I tried things like: >>>>> >>>>> gfx_passthru=1 >>>>> pci=[''0000:04:00.0'',''0000:04:00.1'',''0000:06:00.0''] >>>> # 0 is secondary passthrough; 1 is primary passthrough >>>> gfx_passthru=0 >>>> # without the 0000: (not sure if it is a problem though) >>>> pci=[''04:00.0'',''04:00.1'',''06:00.0''] >>> I tried this (I put the other card in /etc/modprobe.d/pciback.conf to >>> hide it) but Xen says the card is not assignable. When I run the >>> machine up, there is what looks like Linux boot logging on the second >>> monitor that stays there after the machine comes up. >>> >>> I''m obviously doing something wrong but, right now, I''m not sure what >>> it is. >> I''m not sure about using modprobe.d for this... modprobe would rely on >> the device being unbound before it would work. The boot line method >> works because nothing is bound before the boot. But modprobe would need >> an unbind too, not just a list of devices. >> >> What I use is this script I wrote (which is simple to use, because you >> don''t need to edit your boot lines, or rebuild your kernel): >> http://wiki.xen.org/wiki/Bind_lib.bash >> >> As it says on the top of that page, xl has this functionality built in. >> I wrote that for xm. >> >> > > Thank you for all your help. > > We are using xl. I had some major progress yesterday. I did the > following: > > * echo "0000:05:00.0" > > /sys/bus/pci/devices/0000\:05\:00.0/driver/unbind > * pci-assignable-add 05:00.0 > * pci-attach Windows-guest 05:00.0 > > The card then appeared in Windows. I had to access the VM via VNC as > the AMD Catalyst driver has a wonderful feature that you can''t install > it unless the hardware is already in the machine and detected by the > driver installer. It installed and, after a guest reboot, I got a > lovely full-res Windows desktop on the right card''s output. > > My next challenge is to make it all work automatically.You can do this simpy by putting your attach in a pci = [...] line in your config file, and running the pci-assignable-add before starting the domain (xl create). one time setup: vim mydomu put pci = ... line in domain every reboot of dom0, run this: modprobe pciback xl pci-assignable-add 05:00.0 xl create mydomu Skip your unbind step because pci-assignable-add does it for you, and even records what was previously bound so you can give it back to dom0.> I feel like I''m making progress as I at least have it working > manually. Was my colleague correct that adding the module to initramfs > works or should I blow the cobwebs out of my brain and try to remember > how to roll a custom kernel please?You can simply use modprobe + pci-assignable-add. Building a kernel with xen-pciback compiled in is not necessary unless you are using the boot line method.