Xen users - Jul 2013 - VGA Passthrough of AMD HD8670D IGP to HVM Win7 results in "Code 43"

Hello everyone,

pardon me If I''m doing anything wrong, this is my first post to this
list.

For the past few days, I''ve been trying to pass the GPU of my AMD A-10
6800K APU to a HVM Windows 7 guest, but haven''t had any luck yet.

My relevant hardware is as follows:
AMD A-10 6800K with HD 8670 integrated graphics processor
ASRock FM2A75 Pro4 with latest firmware, so that it (supposedly) supports
IOMMU
8GB RAM, 1GB of which has been assigned to the IGP in the BIOS/EFI settings

As you may notice, the IGP is the only graphics device present.

I''ve tried to follow numerous guides to get VGA passthrough to work;
currently, I''m running Linux Mint 13 XFCE and did most of what this
guide
proposes:
http://forums.linuxmint.com/viewtopic.php?f=42&t=112013

However, with Mint as well as with Debian wheezy, I wasn''t able to
start
DomUs with the xen from the repositories; some seconds after xm create or
xl create, the host computer would reboot; I haven''t figured out why.
The
same holds with xen 4.3 compiled from source, at least on Mint. However,
the most recent xen from the mercurial repository allows me to start DomUs.
Also, when using Mint, I had to upgrade from the stock 3.2.0-23 kernel to
3.8.0-26 from the backports, otherwise the machine would reset immediately
or shortly after xen tried to load the linux kernel. Again, having limited
experience with debugging linux or xen problems, I was unable to figure out
why.

Finally having installed Windows 7 I installed the most recent AMD catalyst
drivers in the DomU. After that, in the Device Manager, the graphics card
shows up, but with a yellow triangle; a double click on the GPU gave me
"Code 43" as an explanation of what went wrong. What I found with
google
only points to nVidia-users having that problem.

I also tried to set gfx_passthru to 1, but then xl create would complain:
libxl: error: libxl_dm.c:1275:device_model_spawn_outcome: domain 2 device
model: spawn failed (rc=-3)
libxl: error: libxl_create.c:1075:domcreate_devmodel_started: device model
did not start: -3
libxl: error: libxl_dm.c:1306:libxl__destroy_device_model: Device Model
already exited
and /var/log/xen/qemu-dm-orthowin.log would contain
qemu-system-i386: -gfx_passthru: invalid option

For reference, here is the kernel config: http://pastebin.com/kwUWkyP2
My DomU configuration: http://pastebin.com/E9jkkJXj
The output of xl info: http://pastebin.com/nj1ykFXJ
The output of xl dmesg: http://pastebin.com/MS96knmL
The output of dmesg: http://pastebin.com/2sQFuCuJ
Please note, as it might be related to my issue, that what comes at the end
of the dmesg output seems suspicious to me (RIP  [<ffffffff81012861>]
xen_spin_lock+0x21/0x50 and the lines around that)

I hope that I have provided enough information for further investigation.
The computer is not in any kind of production-use, so please feel free to
request things that will or may require me to reinstall the operating
system or some of its components. As the hardware is new, I''d not be
happy
if I had to do something that would risk permanent damage. :)
Should this be the wrong mailing list for this kind of post, please let me
know where I can send it to instead.

Thank you very much for your time, any help is highly appreciated, not only
regarding my primary problem (getting VGA passthrough to work) but also the
others mentioned, especially since they might be related.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Hi Gustav,

I will try not to spam this mailing list :).

Got a HD 7970 and it works both for Win7 (as domU) and Linux (as domU). It works
perfectly with 8GB of RAM for domU

Here is a quick summary

- STEP 1) A few Xen features in your kernel are configured as modules (= m) ! I
will suggest to set everything directly built in the kernel (= Y) . It is a bit
pain to configure the kernel  manually.  My latest test was for kernel 3.8.13.
If you can download the kernel and build it yourself that I can sent you my own
configuration file for the kernel (3.8.13). After that you will have to update
your grub file

- STEP 2) You are testing Xen 4.4 unstable. This branch has to be patched. In
Marsh/April the latest patch for ATI has been sent to this mailing list.
So you have to rebuild a patched Xen version
(http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram)

- STEP 3) Your configuration file for domU  is not well formed. There are
missings options.

I am currently at work for the moment . I will try to share my own configuration
file for domU when I am back to home.

Regards.

David




________________________________
 De : Gustav Sorenson <gu.sorenson@gmail.com>
À : xen-users@lists.xen.org 
Envoyé le : Mercredi 10 juillet 2013 12h31
Objet : [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM Win7 results in
"Code 43"
 


Hello everyone,

pardon me If I''m doing anything wrong, this is my first post to this
list.

For the past few days, I''ve been trying to pass the GPU of my AMD A-10
6800K APU to a HVM Windows 7 guest, but haven''t had any luck yet.

My relevant hardware is as follows:
AMD A-10 6800K with HD 8670 integrated graphics processor
ASRock FM2A75 Pro4 with latest firmware, so that it (supposedly) supports IOMMU
8GB RAM, 1GB of which has been assigned to the IGP in the BIOS/EFI settings

As you may notice, the IGP is the only graphics device present.


I''ve tried to follow numerous guides to get VGA passthrough to work;
currently, I''m running Linux Mint 13 XFCE and did most of what this
guide proposes:
http://forums.linuxmint.com/viewtopic.php?f=42&t=112013


However, with Mint as well as with Debian wheezy, I wasn''t able to
start DomUs with the xen from the repositories; some seconds after xm create or
xl create, the host computer would reboot; I haven''t figured out why.
The same holds with xen 4.3 compiled from source, at least on Mint. However, the
most recent xen from the mercurial repository allows me to start DomUs.

Also, when using Mint, I had to upgrade from the stock 3.2.0-23 kernel to
3.8.0-26 from the backports, otherwise the machine would reset immediately or
shortly after xen tried to load the linux kernel. Again, having limited
experience with debugging linux or xen problems, I was unable to figure out why.


Finally having installed Windows 7 I installed the most recent AMD catalyst
drivers in the DomU. After that, in the Device Manager, the graphics card shows
up, but with a yellow triangle; a double click on the GPU gave me "Code
43" as an explanation of what went wrong. What I found with google only
points to nVidia-users having that problem.


I also tried to set gfx_passthru to 1, but then xl create would complain:
libxl: error: libxl_dm.c:1275:device_model_spawn_outcome: domain 2 device model:
spawn failed (rc=-3)
libxl: error: libxl_create.c:1075:domcreate_devmodel_started: device model did
not start: -3
libxl: error: libxl_dm.c:1306:libxl__destroy_device_model: Device Model already
exited

and /var/log/xen/qemu-dm-orthowin.log would contain
qemu-system-i386: -gfx_passthru: invalid option


For reference, here is the kernel config: http://pastebin.com/kwUWkyP2

My DomU configuration: http://pastebin.com/E9jkkJXj

The output of xl info: http://pastebin.com/nj1ykFXJ

The output of xl dmesg: http://pastebin.com/MS96knmL

The output of dmesg: http://pastebin.com/2sQFuCuJ

Please note, as it might be related to my issue, that what comes at the end of
the dmesg output seems suspicious to me (RIP  [<ffffffff81012861>]
xen_spin_lock+0x21/0x50 and the lines around that)


I hope that I have provided enough information for further investigation. The
computer is not in any kind of production-use, so please feel free to request
things that will or may require me to reinstall the operating system or some of
its components. As the hardware is new, I''d not be happy if I had to do
something that would risk permanent damage. :)

Should this be the wrong mailing list for this kind of post, please let me know
where I can send it to instead.

Thank you very much for your time, any help is highly appreciated, not only
regarding my primary problem (getting VGA passthrough to work) but also the
others mentioned, especially since they might be related.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Hello Gustav,

As David suggested, adding the kernel configuration flags as "y" and
patching the source will fix a number of problems.  The xm toolstack is
deprecated so I recommend the xl toolstack.  It would certainly be worth
checking out David''s blog post as he supplies a script with it as well
(besides the patch).

As for the configuration, you should give the [man pages](
http://wiki.xen.org/wiki/Xen_4.3_Man_Pages) a read.  Here is a copy of your
config that is xl compatible:
http://pastebin.com/e0bRwdvK

The error you received about gfx_passthru is because upstream-qemu does not
have that option in it (that is part of qemu-xen-traditional), and if you
review the man page documentation you can omit a number of options since
they have default values.

Hope this helps,

~Casey


On Wed, Jul 10, 2013 at 8:36 AM, David TECHER <davidtecher@yahoo.fr>
wrote:
> Hi Gustav,
>
> I will try not to spam this mailing list :).
>
> Got a HD 7970 and it works both for Win7 (as domU) and Linux (as domU). It
> works perfectly with 8GB of RAM for domU
>
> Here is a quick summary
>
> - STEP 1) A few Xen features in your kernel are configured as modules (>
m) ! I will suggest to set everything directly built in the kernel (= Y) .
> It is a bit pain to configure the kernel  manually.  My latest test was for
> kernel 3.8.13. If you can download the kernel and build it yourself that I
> can sent you my own configuration file for the kernel (3.8.13). After that
> you will have to update your grub file
>
> - STEP 2) You are testing Xen 4.4 unstable. This branch has to be patched.
> In Marsh/April the latest patch for ATI has been sent to this mailing list.
> So you have to rebuild a patched Xen version (
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> )
>
> - STEP 3) Your configuration file for domU  is not well formed. There are
> missings options.
>
> I am currently at work for the moment . I will try to share my own
> configuration file for domU when I am back to home.
>
> Regards.
>
> David
>
>
>   ------------------------------
>  *De :* Gustav Sorenson <gu.sorenson@gmail.com>
> *À :* xen-users@lists.xen.org
> *Envoyé le :* Mercredi 10 juillet 2013 12h31
> *Objet :* [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM Win7
> results in "Code 43"
>
> Hello everyone,
>
> pardon me If I''m doing anything wrong, this is my first post to
this list.
>
> For the past few days, I''ve been trying to pass the GPU of my AMD
A-10
> 6800K APU to a HVM Windows 7 guest, but haven''t had any luck yet.
>
> My relevant hardware is as follows:
> AMD A-10 6800K with HD 8670 integrated graphics processor
> ASRock FM2A75 Pro4 with latest firmware, so that it (supposedly) supports
> IOMMU
> 8GB RAM, 1GB of which has been assigned to the IGP in the BIOS/EFI settings
>
> As you may notice, the IGP is the only graphics device present.
>
> I''ve tried to follow numerous guides to get VGA passthrough to
work;
> currently, I''m running Linux Mint 13 XFCE and did most of what
this guide
> proposes:
> http://forums.linuxmint.com/viewtopic.php?f=42&t=112013
>
> However, with Mint as well as with Debian wheezy, I wasn''t able to
start
> DomUs with the xen from the repositories; some seconds after xm create or
> xl create, the host computer would reboot; I haven''t figured out
why. The
> same holds with xen 4.3 compiled from source, at least on Mint. However,
> the most recent xen from the mercurial repository allows me to start DomUs.
> Also, when using Mint, I had to upgrade from the stock 3.2.0-23 kernel to
> 3.8.0-26 from the backports, otherwise the machine would reset immediately
> or shortly after xen tried to load the linux kernel. Again, having limited
> experience with debugging linux or xen problems, I was unable to figure out
> why.
>
> Finally having installed Windows 7 I installed the most recent AMD
> catalyst drivers in the DomU. After that, in the Device Manager, the
> graphics card shows up, but with a yellow triangle; a double click on the
> GPU gave me "Code 43" as an explanation of what went wrong. What
I found
> with google only points to nVidia-users having that problem.
>
> I also tried to set gfx_passthru to 1, but then xl create would complain:
> libxl: error: libxl_dm.c:1275:device_model_spawn_outcome: domain 2 device
> model: spawn failed (rc=-3)
> libxl: error: libxl_create.c:1075:domcreate_devmodel_started: device model
> did not start: -3
> libxl: error: libxl_dm.c:1306:libxl__destroy_device_model: Device Model
> already exited
> and /var/log/xen/qemu-dm-orthowin.log would contain
> qemu-system-i386: -gfx_passthru: invalid option
>
> For reference, here is the kernel config: http://pastebin.com/kwUWkyP2
> My DomU configuration: http://pastebin.com/E9jkkJXj
> The output of xl info: http://pastebin.com/nj1ykFXJ
> The output of xl dmesg: http://pastebin.com/MS96knmL
> The output of dmesg: http://pastebin.com/2sQFuCuJ
> Please note, as it might be related to my issue, that what comes at the
> end of the dmesg output seems suspicious to me (RIP 
[<ffffffff81012861>]
> xen_spin_lock+0x21/0x50 and the lines around that)
>
> I hope that I have provided enough information for further investigation.
> The computer is not in any kind of production-use, so please feel free to
> request things that will or may require me to reinstall the operating
> system or some of its components. As the hardware is new, I''d not
be happy
> if I had to do something that would risk permanent damage. :)
> Should this be the wrong mailing list for this kind of post, please let me
> know where I can send it to instead.
>
> Thank you very much for your time, any help is highly appreciated, not
> only regarding my primary problem (getting VGA passthrough to work) but
> also the others mentioned, especially since they might be related.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Hello everyone,

first of all, thank you David and Casey for your help. Unfortunately, I
still haven''t succeeded.

After David generously mailed me his configuration, I tried to mimic it as
close as I could.

I installed Debian wheezy amd64 again (since it gave me less issues than
Mint).
I downloaded kernel 3.8.13 and used David''s config, with the exception
that
I disabled CONFIG_SYSFS_DEPRECATED and CONFIG_SYSFS_DEPRECATED_V2, since
having those compiled in gave me severe warnings while booting and I was
unable to use LVM (PVs would not be detected). The kernel config as I used
it can be found here: http://pastebin.com/KN74FWE6

Then I compiled xen, referring to the instructions at
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
with the exception that I used xen revision 27214 (the most recent to this
date), because even xen 4.3 had caused host system reboots when creating
DomUs in my previous installations on this hardware. However, with kernel
3.8.13, I haven''t tried any other revision of xen yet.

The output of lspci is here: http://pastebin.com/kf94viHM

Since I want to pass my integrated GPU as well as some USB ports, I edited
/boot/grub/grub.cfg so that the relevant boot entry contains this line:
module  /boot/vmlinuz-3.8.13 placeholder
root=UUID=5fed3d65-274c-4a14-b72a-0f9bb6d21e41 ro  quiet
xen-pciback.hide=(00:01.0)(00:01.1)(00:12.0)(00:12.2) xen-pciback.permissive

After having installed Windows 7 x64, my DomU configuration looks like
this: http://pastebin.com/m0wLZzkF
Now, I start the DomU with this script (again, thanks to David):
http://pastebin.com/pkUdLvuw

However, in the Windows Device Manager, I still get "Code 43" shown
for my
passed-through GPU.

What may be interesting is that after I had installed the AMD graphics
drivers in the Windows DomU, and after shutting down the DomU and rebooting
the host system, I tried to access the Windows Device Manager from within
the DomU, when suddenly DomU and Dom0 locked up. After some time, on the
physical LCD screen (which still seems to display either xen or linux
kernel output) I started to receive the message
"hda: DMA interrupt recovery"
with quite some time (on the order of about a minute) in between. hda is my
host system SATA disk (and the only one in the machine). After
hard-resetting the host, this problem hasn''t occured anymore. I
don''t know
whether this may be related to my problem.

Again, any attempts to help are very highly appreciated.

Thanks!

On Wed, Jul 10, 2013 at 11:22 PM, David TECHER <davidtecher@yahoo.fr>
wrote:
> Sound good!
>
> Please be informed that it may require a lot of time before being able to
> set up VGA Passhtrough with Xen for ATI. So there will be a lot of mails
>
> Attached are my files.
>
> config-3.8.13: kernel configuration file
> mercury-xen09.cfg : domU - Ubuntu 12.04 32 Bit
> mercury-xen10.cfg: domU - Window 7 64 Bit
> run-passthrough.sh: script to boot dimU
>
> My suggested plan is
>
> 1) build your kernel using my file (config-3.8.13) hoping you have
> required experience for kernel
>
> download kernel 3.8.13
> decompress
> copy my file in the decompressed folder as ''.config'' file
>
> make menuconfig
> make bzImage modules
> make install modules_install
> mkinitramfs -o /boot/initrd.img-3.8.13 3.8.13
>
> update your grub
>
> 2) Xen - refer to
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
>
> WHEN AND ONLY WHEN  item #1 and item #2 above ARE OK THEN
>
> 3) Your grub file may have something like that. You may have to update the
> parameter xen-pciback.hide=(....)(....).... ----> list of devices
>
> --------------------------
> menuentry ''Debian GNU/Linux, with Linux 3.8.13 and XEN
4.3-unstable''
> --class debian --class gnu-linux --class gnu --class os --class xen {
>         insmod part_gpt
>         insmod ext2
>         set root=''(/dev/sda,gpt1)''
>         search --no-floppy --fs-uuid --set
> 1cd457ae-85f4-4626-8f94-1f444fcf6d5c
>         echo    ''Chargement de Linux 3.8.13 ...''
>         multiboot   /boot/xen-4.3-unstable.gz placeholder  dom0_mem=2048MB
>         module  /boot/vmlinuz-3.8.13 placeholder
> root=UUID=1cd457ae-85f4-4626-8f94-1f444fcf6d5c ro intel_iommu=on
> xen-pciback.hide=(01:00.1)(00:1b.0)(00:1a.0)(00:1d.0)
> xen-pciback.permissive quiet
>         echo    ''Chargement du disque mémoire initial
...''
>         module  /boot/initrd.img-3.8.13
> }
> ------------------------
>
>
>   ------------------------------
>  *De :* Gustav Sorenson <gu.sorenson@gmail.com>
> *À :* David TECHER <davidtecher@yahoo.fr>
> *Envoyé le :* Mercredi 10 juillet 2013 20h08
> *Objet :* Re: [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM Win7
> results in "Code 43"
>
> Hello,
>
> thank you for your kind answer.
> I''m not home right now either, but will try what you suggested
once I''m
> back.
>
> I''d be very thankful if you could provide your domU configuration,
kernel
> configuration and any other files that might be of help.
>
> I write this mail to you only and not to the list, since I think it
> doesn''t contribute to the thread. Should this be frowned upon in
the xen
> community, please let me know and I will post it to the list as well.
>
> Thank you very much.
>
>
> On Wed, Jul 10, 2013 at 2:36 PM, David TECHER
<davidtecher@yahoo.fr>wrote:
>
> Hi Gustav,
>
> I will try not to spam this mailing list :).
>
> Got a HD 7970 and it works both for Win7 (as domU) and Linux (as domU). It
> works perfectly with 8GB of RAM for domU
>
> Here is a quick summary
>
> - STEP 1) A few Xen features in your kernel are configured as modules (>
m) ! I will suggest to set everything directly built in the kernel (= Y) .
> It is a bit pain to configure the kernel  manually.  My latest test was for
> kernel 3.8.13. If you can download the kernel and build it yourself that I
> can sent you my own configuration file for the kernel (3.8.13). After that
> you will have to update your grub file
>
> - STEP 2) You are testing Xen 4.4 unstable. This branch has to be patched.
> In Marsh/April the latest patch for ATI has been sent to this mailing list.
> So you have to rebuild a patched Xen version (
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> )
>
> - STEP 3) Your configuration file for domU  is not well formed. There are
> missings options.
>
> I am currently at work for the moment . I will try to share my own
> configuration file for domU when I am back to home.
>
> Regards.
>
> David
>
>
>   ------------------------------
>  *De :* Gustav Sorenson <gu.sorenson@gmail.com>
> *À :* xen-users@lists.xen.org
> *Envoyé le :* Mercredi 10 juillet 2013 12h31
> *Objet :* [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM Win7
> results in "Code 43"
>
> Hello everyone,
>
> pardon me If I''m doing anything wrong, this is my first post to
this list.
>
> For the past few days, I''ve been trying to pass the GPU of my AMD
A-10
> 6800K APU to a HVM Windows 7 guest, but haven''t had any luck yet.
>
> My relevant hardware is as follows:
> AMD A-10 6800K with HD 8670 integrated graphics processor
> ASRock FM2A75 Pro4 with latest firmware, so that it (supposedly) supports
> IOMMU
> 8GB RAM, 1GB of which has been assigned to the IGP in the BIOS/EFI settings
>
> As you may notice, the IGP is the only graphics device present.
>
> I''ve tried to follow numerous guides to get VGA passthrough to
work;
> currently, I''m running Linux Mint 13 XFCE and did most of what
this guide
> proposes:
> http://forums.linuxmint.com/viewtopic.php?f=42&t=112013
>
> However, with Mint as well as with Debian wheezy, I wasn''t able to
start
> DomUs with the xen from the repositories; some seconds after xm create or
> xl create, the host computer would reboot; I haven''t figured out
why. The
> same holds with xen 4.3 compiled from source, at least on Mint. However,
> the most recent xen from the mercurial repository allows me to start DomUs.
> Also, when using Mint, I had to upgrade from the stock 3.2.0-23 kernel to
> 3.8.0-26 from the backports, otherwise the machine would reset immediately
> or shortly after xen tried to load the linux kernel. Again, having limited
> experience with debugging linux or xen problems, I was unable to figure out
> why.
>
> Finally having installed Windows 7 I installed the most recent AMD
> catalyst drivers in the DomU. After that, in the Device Manager, the
> graphics card shows up, but with a yellow triangle; a double click on the
> GPU gave me "Code 43" as an explanation of what went wrong. What
I found
> with google only points to nVidia-users having that problem.
>
> I also tried to set gfx_passthru to 1, but then xl create would complain:
> libxl: error: libxl_dm.c:1275:device_model_spawn_outcome: domain 2 device
> model: spawn failed (rc=-3)
> libxl: error: libxl_create.c:1075:domcreate_devmodel_started: device model
> did not start: -3
> libxl: error: libxl_dm.c:1306:libxl__destroy_device_model: Device Model
> already exited
> and /var/log/xen/qemu-dm-orthowin.log would contain
> qemu-system-i386: -gfx_passthru: invalid option
>
> For reference, here is the kernel config: http://pastebin.com/kwUWkyP2
> My DomU configuration: http://pastebin.com/E9jkkJXj
> The output of xl info: http://pastebin.com/nj1ykFXJ
> The output of xl dmesg: http://pastebin.com/MS96knmL
>  The output of dmesg: http://pastebin.com/2sQFuCuJ
> Please note, as it might be related to my issue, that what comes at the
> end of the dmesg output seems suspicious to me (RIP 
[<ffffffff81012861>]
> xen_spin_lock+0x21/0x50 and the lines around that)
>
> I hope that I have provided enough information for further investigation.
> The computer is not in any kind of production-use, so please feel free to
> request things that will or may require me to reinstall the operating
> system or some of its components. As the hardware is new, I''d not
be happy
> if I had to do something that would risk permanent damage. :)
> Should this be the wrong mailing list for this kind of post, please let me
> know where I can send it to instead.
>
> Thank you very much for your time, any help is highly appreciated, not
> only regarding my primary problem (getting VGA passthrough to work) but
> also the others mentioned, especially since they might be related.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>
>
>
>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Have you made sure you are passing <= 2GB of RAM to the domU,
 and tried without any 3rd party patches? Also, can you try
 passing the GPU as secondary (add a cheap low-end card as
 a primary for dom0 if you haven't already).

 Gordan

 On Thu, 11 Jul 2013 14:15:21 +0200, Gustav Sorenson 
 <gu.sorenson@gmail.com> wrote:
> Hello everyone,
>
> first of all, thank you David and Casey for your help. Unfortunately,
> I still haven't succeeded.
>
> After David generously mailed me his configuration, I tried to mimic
> it as close as I could.
>
> I installed Debian wheezy amd64 again (since it gave me less issues
> than Mint).
> I downloaded kernel 3.8.13 and used David's config, with the 
> exception
> that I disabled CONFIG_SYSFS_DEPRECATED and
> CONFIG_SYSFS_DEPRECATED_V2, since having those compiled in gave me
> severe warnings while booting and I was unable to use LVM (PVs would
> not be detected). The kernel config as I used it can be found here:
> http://pastebin.com/KN74FWE6 [1]
>
> Then I compiled xen, referring to the instructions at
>
> 
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> [2]
>  with the exception that I used xen revision 27214 (the most recent 
> to
> this date), because even xen 4.3 had caused host system reboots when
> creating DomUs in my previous installations on this hardware. 
> However,
> with kernel 3.8.13, I haven't tried any other revision of xen yet.
>
> The output of lspci is here: http://pastebin.com/kf94viHM [3]
>
> Since I want to pass my integrated GPU as well as some USB ports, I
> edited /boot/grub/grub.cfg so that the relevant boot entry contains
> this line:
>  module  /boot/vmlinuz-3.8.13 placeholder
> root=UUID=5fed3d65-274c-4a14-b72a-0f9bb6d21e41 ro  quiet
> xen-pciback.hide=(00:01.0)(00:01.1)(00:12.0)(00:12.2)
> xen-pciback.permissive
>
> After having installed Windows 7 x64, my DomU configuration looks 
> like
> this: http://pastebin.com/m0wLZzkF [4]
>
> Now, I start the DomU with this script (again, thanks to David):
> http://pastebin.com/pkUdLvuw [5]
>
> However, in the Windows Device Manager, I still get "Code 43"
shown
> for my passed-through GPU.
>
> What may be interesting is that after I had installed the AMD 
> graphics
> drivers in the Windows DomU, and after shutting down the DomU and
> rebooting the host system, I tried to access the Windows Device
> Manager from within the DomU, when suddenly DomU and Dom0 locked up.
> After some time, on the physical LCD screen (which still seems to
> display either xen or linux kernel output) I started to receive the
> message
>  "hda: DMA interrupt recovery"
>
> with quite some time (on the order of about a minute) in between. hda
> is my host system SATA disk (and the only one in the machine). After
> hard-resetting the host, this problem hasn't occured anymore. I
don't
> know whether this may be related to my problem.
>
> Again, any attempts to help are very highly appreciated.
>
> Thanks!
>
> On Wed, Jul 10, 2013 at 11:22 PM, David TECHER  wrote:
>
> Sound good!
>
>  Please be informed that it may require a lot of time before being
> able to set up VGA Passhtrough with Xen for ATI. So there will be a
> lot of mails
>
> Attached are my files.
>
> config-3.8.13: kernel configuration file
>  mercury-xen09.cfg : domU - Ubuntu 12.04 32 Bit
> mercury-xen10.cfg: domU - Window 7 64 Bit
> run-passthrough.sh: script to boot dimU
>
> My suggested plan is
>
> 1) build your kernel using my file (config-3.8.13) hoping you have
> required experience for kernel
>
> download kernel 3.8.13
> decompress
> copy my file in the decompressed folder as '.config' file
>
> make menuconfig
> make bzImage modules
> make install modules_install
> mkinitramfs -o /boot/initrd.img-3.8.13 3.8.13
>
> update your grub
>
> 2) Xen - refer to
> 
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> [7]
>
> WHEN AND ONLY WHEN  item #1 and item #2 above ARE OK THEN
>
> 3) Your grub file may have something like that. You may have to 
> update
> the parameter xen-pciback.hide=(....)(....).... ----> list of devices
>
> --------------------------
>  menuentry 'Debian GNU/Linux, with Linux 3.8.13 and XEN
4.3-unstable'
> --class debian --class gnu-linux --class gnu --class os --class xen {
>         insmod part_gpt
>         insmod ext2
>         set root='(/dev/sda,gpt1)'
>         search --no-floppy --fs-uuid --set
> 1cd457ae-85f4-4626-8f94-1f444fcf6d5c
>         echo    'Chargement de Linux 3.8.13 ...'
>          multiboot   /boot/xen-4.3-unstable.gz placeholder 
> dom0_mem=2048MB
>         module  /boot/vmlinuz-3.8.13 placeholder
> root=UUID=1cd457ae-85f4-4626-8f94-1f444fcf6d5c ro intel_iommu=on
> xen-pciback.hide=(01:00.1)(00:1b.0)(00:1a.0)(00:1d.0)
> xen-pciback.permissive quiet
>          echo    'Chargement du disque mémoire initial ...'
>         module  /boot/initrd.img-3.8.13
> }
> ------------------------
>
> -------------------------
>   DE : Gustav Sorenson
>  À : David TECHER
>  ENVOYÉ LE : Mercredi 10 juillet 2013 20h08
>  OBJET : Re: [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM
> Win7 results in "Code 43"
>
> Hello,
>
> thank you for your kind answer.
> I'm not home right now either, but will try what you suggested once
> I'm back.
>
> I'd be very thankful if you could provide your domU configuration,
> kernel configuration and any other files that might be of help.
>
> I write this mail to you only and not to the list, since I think it
> doesn't contribute to the thread. Should this be frowned upon in the
> xen community, please let me know and I will post it to the list as
> well.
>
> Thank you very much.
>
> On Wed, Jul 10, 2013 at 2:36 PM, David TECHER  wrote:
>
> Hi Gustav,
>
> I will try not to spam this mailing list :).
>
> Got a HD 7970 and it works both for Win7 (as domU) and Linux (as
> domU). It works perfectly with 8GB of RAM for domU
>
> Here is a quick summary
>
> - STEP 1) A few Xen features in your kernel are configured as modules
> (= m) ! I will suggest to set everything directly built in the kernel
> (= Y) . It is a bit pain to configure the kernel  manually.  My
> latest test was for kernel 3.8.13. If you can download the kernel and
> build it yourself that I can sent you my own configuration file for
> the kernel (3.8.13). After that you will have to update your grub 
> file
>
> - STEP 2) You are testing Xen 4.4 unstable. This branch has to be
> patched. In Marsh/April the latest patch for ATI has been sent to 
> this
> mailing list.
> So you have to rebuild a patched Xen version
> 
>
(http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> [11])
>
> - STEP 3) Your configuration file for domU  is not well formed. There
> are missings options.
>
>  I am currently at work for the moment . I will try to share my own
> configuration file for domU when I am back to home.
>
> Regards.
>
> David
>
> -------------------------
>   DE : Gustav Sorenson
>  À : xen-users@lists.xen.org [13]
>  ENVOYÉ LE : Mercredi 10 juillet 2013 12h31
>  OBJET : [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM Win7
> results in "Code 43"
>
> Hello everyone,
>
> pardon me If I'm doing anything wrong, this is my first post to this
> list.
>
> For the past few days, I've been trying to pass the GPU of my AMD 
> A-10
> 6800K APU to a HVM Windows 7 guest, but haven't had any luck yet.
>
> My relevant hardware is as follows:
> AMD A-10 6800K with HD 8670 integrated graphics processor
> ASRock FM2A75 Pro4 with latest firmware, so that it (supposedly)
> supports IOMMU
> 8GB RAM, 1GB of which has been assigned to the IGP in the BIOS/EFI
> settings
>
> As you may notice, the IGP is the only graphics device present.
>
> I've tried to follow numerous guides to get VGA passthrough to work;
> currently, I'm running Linux Mint 13 XFCE and did most of what this
> guide proposes:
>  http://forums.linuxmint.com/viewtopic.php?f=42&t=112013 [14]
>
> However, with Mint as well as with Debian wheezy, I wasn't able to
> start DomUs with the xen from the repositories; some seconds after xm
> create or xl create, the host computer would reboot; I haven't 
> figured
> out why. The same holds with xen 4.3 compiled from source, at least 
> on
> Mint. However, the most recent xen from the mercurial repository
> allows me to start DomUs.
>
> Also, when using Mint, I had to upgrade from the stock 3.2.0-23 
> kernel
> to 3.8.0-26 from the backports, otherwise the machine would reset
> immediately or shortly after xen tried to load the linux kernel.
> Again, having limited experience with debugging linux or xen 
> problems,
> I was unable to figure out why.
>
> Finally having installed Windows 7 I installed the most recent AMD
> catalyst drivers in the DomU. After that, in the Device Manager, the
> graphics card shows up, but with a yellow triangle; a double click on
> the GPU gave me "Code 43" as an explanation of what went wrong.
What
> I
> found with google only points to nVidia-users having that problem.
>
> I also tried to set gfx_passthru to 1, but then xl create would
> complain:
> libxl: error: libxl_dm.c:1275:device_model_spawn_outcome: domain 2
> device model: spawn failed (rc=-3)
> libxl: error: libxl_create.c:1075:domcreate_devmodel_started: device
> model did not start: -3
>  libxl: error: libxl_dm.c:1306:libxl__destroy_device_model: Device
> Model already exited
>
> and /var/log/xen/qemu-dm-orthowin.log would contain
> qemu-system-i386: -gfx_passthru: invalid option
>
> For reference, here is the kernel config: 
> http://pastebin.com/kwUWkyP2
> [15]
>
> My DomU configuration: http://pastebin.com/E9jkkJXj [16]
>
> The output of xl info: http://pastebin.com/nj1ykFXJ [17]
>
> The output of xl dmesg: http://pastebin.com/MS96knmL [18]
>
> The output of dmesg: http://pastebin.com/2sQFuCuJ [19]
>
> Please note, as it might be related to my issue, that what comes at
> the end of the dmesg output seems suspicious to me (RIP  []
> xen_spin_lock+0x21/0x50 and the lines around that)
>
> I hope that I have provided enough information for further
> investigation. The computer is not in any kind of production-use, so
> please feel free to request things that will or may require me to
> reinstall the operating system or some of its components. As the
> hardware is new, I'd not be happy if I had to do something that would
> risk permanent damage. :)
>
> Should this be the wrong mailing list for this kind of post, please
> let me know where I can send it to instead.
>
> Thank you very much for your time, any help is highly appreciated, 
> not
> only regarding my primary problem (getting VGA passthrough to work)
> but also the others mentioned, especially since they might be 
> related.
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org [20]
> http://lists.xen.org/xen-users [21]
>
>
>
> Links:
> ------
> [1] http://pastebin.com/KN74FWE6
> [2]
> 
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> [3] http://pastebin.com/kf94viHM
> [4] http://pastebin.com/m0wLZzkF
> [5] http://pastebin.com/pkUdLvuw
> [6] mailto:davidtecher@yahoo.fr
> [7]
> 
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> [8] mailto:gu.sorenson@gmail.com
> [9] mailto:davidtecher@yahoo.fr
> [10] mailto:davidtecher@yahoo.fr
> [11]
> 
>
http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram
> [12] mailto:gu.sorenson@gmail.com
> [13] mailto:xen-users@lists.xen.org
> [14] http://forums.linuxmint.com/viewtopic.php?f=42&t=112013
> [15] http://pastebin.com/kwUWkyP2
> [16] http://pastebin.com/E9jkkJXj
> [17] http://pastebin.com/nj1ykFXJ
> [18] http://pastebin.com/MS96knmL
> [19] http://pastebin.com/2sQFuCuJ
> [20] mailto:Xen-users@lists.xen.org
> [21] http://lists.xen.org/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Hello Gordan,

thanks for your reply.

I wasn''t aware that there was a restriction regarding the DomU RAM.
However, it just so happens that I always tried with "memory=2048". I
assume this satisfies the <= 2GB RAM requirement, although I''m not
sure
since I don''t know whether you refer to "base-10 G" or
"base-2 G". :)
Furthermore, I don''t know whether the setting "shadow_memory =
512" counts
towards that limit, or whether the 1024MB that has been assigned to the
integrated GPU in BIOS does.

Yes, I have tried without any first party patches, as I had written in my
first mail in this thread. However, I haven''t tried
"unpatched" xen with
kernel 3.8.13 yet - should I?

Unfortunately, I don''t have a spare dedicated graphics card I could add
right now. I will try to get one (is it important whether it''s PCI or
PCIe,
nVidia or ATI, or some specific model?) and report back (if) when I get
hold of one.
However, my primary goal was to build a computer that would serve two
independent VMs with 3D acceleration each, e.g. so that two people could
play games at the same time. My plan was to buy the hardware as I have it
right now, test whether I could make xen pass-through the one (integrated)
GPU I have right now, and then buy a not-too-expensive but still quite
powerful dedicated GPU. I would be much happier if I didn''t have to buy
and
install a third GPU, although I realize me wishing that doesn''t
necessarily
make it so. ;)

Thanks!


On Thu, Jul 11, 2013 at 2:44 PM, Gordan Bobic <gordan@bobich.net> wrote:
> Have you made sure you are passing <= 2GB of RAM to the domU,
> and tried without any 3rd party patches? Also, can you try
> passing the GPU as secondary (add a cheap low-end card as
> a primary for dom0 if you haven''t already).
>
> Gordan
>
>
> On Thu, 11 Jul 2013 14:15:21 +0200, Gustav Sorenson
<gu.sorenson@gmail.com>
> wrote:
>
>> Hello everyone,
>>
>> first of all, thank you David and Casey for your help. Unfortunately,
>> I still haven''t succeeded.
>>
>> After David generously mailed me his configuration, I tried to mimic
>> it as close as I could.
>>
>> I installed Debian wheezy amd64 again (since it gave me less issues
>> than Mint).
>> I downloaded kernel 3.8.13 and used David''s config, with the
exception
>> that I disabled CONFIG_SYSFS_DEPRECATED and
>> CONFIG_SYSFS_DEPRECATED_V2, since having those compiled in gave me
>> severe warnings while booting and I was unable to use LVM (PVs would
>> not be detected). The kernel config as I used it can be found here:
>> http://pastebin.com/KN74FWE6 [1]
>>
>>
>> Then I compiled xen, referring to the instructions at
>>
>>
>> http://www.davidgis.fr/blog/**index.php?2013/04/05/937-xen-**
>> 43-unstable-vga-passthrough-**hd-7970-windows-7-64-bits-**
>>
with-more-than-3gb-for-ram<http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram>
>> [2]
>>
>>  with the exception that I used xen revision 27214 (the most recent to
>> this date), because even xen 4.3 had caused host system reboots when
>> creating DomUs in my previous installations on this hardware. However,
>> with kernel 3.8.13, I haven''t tried any other revision of xen
yet.
>>
>> The output of lspci is here: http://pastebin.com/kf94viHM [3]
>>
>>
>> Since I want to pass my integrated GPU as well as some USB ports, I
>> edited /boot/grub/grub.cfg so that the relevant boot entry contains
>> this line:
>>  module  /boot/vmlinuz-3.8.13 placeholder
>> root=UUID=5fed3d65-274c-4a14-**b72a-0f9bb6d21e41 ro  quiet
>> xen-pciback.hide=(00:01.0)(00:**01.1)(00:12.0)(00:12.2)
>> xen-pciback.permissive
>>
>> After having installed Windows 7 x64, my DomU configuration looks like
>> this: http://pastebin.com/m0wLZzkF [4]
>>
>>
>> Now, I start the DomU with this script (again, thanks to David):
>> http://pastebin.com/pkUdLvuw [5]
>>
>>
>> However, in the Windows Device Manager, I still get "Code 43"
shown
>> for my passed-through GPU.
>>
>> What may be interesting is that after I had installed the AMD graphics
>> drivers in the Windows DomU, and after shutting down the DomU and
>> rebooting the host system, I tried to access the Windows Device
>> Manager from within the DomU, when suddenly DomU and Dom0 locked up.
>> After some time, on the physical LCD screen (which still seems to
>> display either xen or linux kernel output) I started to receive the
>> message
>>  "hda: DMA interrupt recovery"
>>
>> with quite some time (on the order of about a minute) in between. hda
>> is my host system SATA disk (and the only one in the machine). After
>> hard-resetting the host, this problem hasn''t occured anymore.
I don''t
>> know whether this may be related to my problem.
>>
>> Again, any attempts to help are very highly appreciated.
>>
>> Thanks!
>>
>> On Wed, Jul 10, 2013 at 11:22 PM, David TECHER  wrote:
>>
>>
>> Sound good!
>>
>>  Please be informed that it may require a lot of time before being
>> able to set up VGA Passhtrough with Xen for ATI. So there will be a
>> lot of mails
>>
>> Attached are my files.
>>
>> config-3.8.13: kernel configuration file
>>  mercury-xen09.cfg : domU - Ubuntu 12.04 32 Bit
>> mercury-xen10.cfg: domU - Window 7 64 Bit
>> run-passthrough.sh: script to boot dimU
>>
>> My suggested plan is
>>
>> 1) build your kernel using my file (config-3.8.13) hoping you have
>> required experience for kernel
>>
>> download kernel 3.8.13
>> decompress
>> copy my file in the decompressed folder as ''.config''
file
>>
>> make menuconfig
>> make bzImage modules
>> make install modules_install
>> mkinitramfs -o /boot/initrd.img-3.8.13 3.8.13
>>
>> update your grub
>>
>> 2) Xen - refer to
>>
>> http://www.davidgis.fr/blog/**index.php?2013/04/05/937-xen-**
>> 43-unstable-vga-passthrough-**hd-7970-windows-7-64-bits-**
>>
with-more-than-3gb-for-ram<http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram>
>> [7]
>>
>>
>> WHEN AND ONLY WHEN  item #1 and item #2 above ARE OK THEN
>>
>> 3) Your grub file may have something like that. You may have to update
>> the parameter xen-pciback.hide=(....)(....).**... ----> list of
devices
>>
>> --------------------------
>>  menuentry ''Debian GNU/Linux, with Linux 3.8.13 and XEN
4.3-unstable''
>> --class debian --class gnu-linux --class gnu --class os --class xen {
>>         insmod part_gpt
>>         insmod ext2
>>         set root=''(/dev/sda,gpt1)''
>>         search --no-floppy --fs-uuid --set
>> 1cd457ae-85f4-4626-8f94-**1f444fcf6d5c
>>         echo    ''Chargement de Linux 3.8.13 ...''
>>          multiboot   /boot/xen-4.3-unstable.gz placeholder
>> dom0_mem=2048MB
>>         module  /boot/vmlinuz-3.8.13 placeholder
>> root=UUID=1cd457ae-85f4-4626-**8f94-1f444fcf6d5c ro intel_iommu=on
>> xen-pciback.hide=(01:00.1)(00:**1b.0)(00:1a.0)(00:1d.0)
>> xen-pciback.permissive quiet
>>          echo    ''Chargement du disque mémoire initial
...''
>>         module  /boot/initrd.img-3.8.13
>> }
>> ------------------------
>>
>> -------------------------
>>   DE : Gustav Sorenson
>>  À : David TECHER
>>  ENVOYÉ LE : Mercredi 10 juillet 2013 20h08
>>  OBJET : Re: [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM
>>
>> Win7 results in "Code 43"
>>
>> Hello,
>>
>> thank you for your kind answer.
>> I''m not home right now either, but will try what you suggested
once
>> I''m back.
>>
>> I''d be very thankful if you could provide your domU
configuration,
>> kernel configuration and any other files that might be of help.
>>
>> I write this mail to you only and not to the list, since I think it
>> doesn''t contribute to the thread. Should this be frowned upon
in the
>> xen community, please let me know and I will post it to the list as
>> well.
>>
>> Thank you very much.
>>
>> On Wed, Jul 10, 2013 at 2:36 PM, David TECHER  wrote:
>>
>>
>> Hi Gustav,
>>
>> I will try not to spam this mailing list :).
>>
>> Got a HD 7970 and it works both for Win7 (as domU) and Linux (as
>> domU). It works perfectly with 8GB of RAM for domU
>>
>> Here is a quick summary
>>
>> - STEP 1) A few Xen features in your kernel are configured as modules
>> (= m) ! I will suggest to set everything directly built in the kernel
>> (= Y) . It is a bit pain to configure the kernel  manually.  My
>> latest test was for kernel 3.8.13. If you can download the kernel and
>> build it yourself that I can sent you my own configuration file for
>> the kernel (3.8.13). After that you will have to update your grub file
>>
>> - STEP 2) You are testing Xen 4.4 unstable. This branch has to be
>> patched. In Marsh/April the latest patch for ATI has been sent to this
>> mailing list.
>> So you have to rebuild a patched Xen version
>>
>> (http://www.davidgis.fr/blog/**index.php?2013/04/05/937-xen-**
>> 43-unstable-vga-passthrough-**hd-7970-windows-7-64-bits-**
>>
with-more-than-3gb-for-ram<http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram>
>> [11])
>>
>>
>> - STEP 3) Your configuration file for domU  is not well formed. There
>> are missings options.
>>
>>  I am currently at work for the moment . I will try to share my own
>> configuration file for domU when I am back to home.
>>
>> Regards.
>>
>> David
>>
>> -------------------------
>>   DE : Gustav Sorenson
>>  À : xen-users@lists.xen.org [13]
>>  ENVOYÉ LE : Mercredi 10 juillet 2013 12h31
>>  OBJET : [Xen-users] VGA Passthrough of AMD HD8670D IGP to HVM Win7
>>
>> results in "Code 43"
>>
>> Hello everyone,
>>
>> pardon me If I''m doing anything wrong, this is my first post
to this
>> list.
>>
>> For the past few days, I''ve been trying to pass the GPU of my
AMD A-10
>> 6800K APU to a HVM Windows 7 guest, but haven''t had any luck
yet.
>>
>> My relevant hardware is as follows:
>> AMD A-10 6800K with HD 8670 integrated graphics processor
>> ASRock FM2A75 Pro4 with latest firmware, so that it (supposedly)
>> supports IOMMU
>> 8GB RAM, 1GB of which has been assigned to the IGP in the BIOS/EFI
>> settings
>>
>> As you may notice, the IGP is the only graphics device present.
>>
>> I''ve tried to follow numerous guides to get VGA passthrough to
work;
>> currently, I''m running Linux Mint 13 XFCE and did most of what
this
>> guide proposes:
>> 
http://forums.linuxmint.com/**viewtopic.php?f=42&t=112013<http://forums.linuxmint.com/viewtopic.php?f=42&t=112013>[14]
>>
>>
>> However, with Mint as well as with Debian wheezy, I wasn''t
able to
>> start DomUs with the xen from the repositories; some seconds after xm
>> create or xl create, the host computer would reboot; I haven''t
figured
>> out why. The same holds with xen 4.3 compiled from source, at least on
>> Mint. However, the most recent xen from the mercurial repository
>> allows me to start DomUs.
>>
>> Also, when using Mint, I had to upgrade from the stock 3.2.0-23 kernel
>> to 3.8.0-26 from the backports, otherwise the machine would reset
>> immediately or shortly after xen tried to load the linux kernel.
>> Again, having limited experience with debugging linux or xen problems,
>> I was unable to figure out why.
>>
>> Finally having installed Windows 7 I installed the most recent AMD
>> catalyst drivers in the DomU. After that, in the Device Manager, the
>> graphics card shows up, but with a yellow triangle; a double click on
>> the GPU gave me "Code 43" as an explanation of what went
wrong. What I
>> found with google only points to nVidia-users having that problem.
>>
>> I also tried to set gfx_passthru to 1, but then xl create would
>> complain:
>> libxl: error: libxl_dm.c:1275:device_model_**spawn_outcome: domain 2
>> device model: spawn failed (rc=-3)
>> libxl: error: libxl_create.c:1075:domcreate_**devmodel_started: device
>> model did not start: -3
>>  libxl: error: libxl_dm.c:1306:libxl__**destroy_device_model: Device
>> Model already exited
>>
>> and /var/log/xen/qemu-dm-orthowin.**log would contain
>> qemu-system-i386: -gfx_passthru: invalid option
>>
>> For reference, here is the kernel config: http://pastebin.com/kwUWkyP2
>> [15]
>>
>> My DomU configuration: http://pastebin.com/E9jkkJXj [16]
>>
>> The output of xl info: http://pastebin.com/nj1ykFXJ [17]
>>
>> The output of xl dmesg: http://pastebin.com/MS96knmL [18]
>>
>> The output of dmesg: http://pastebin.com/2sQFuCuJ [19]
>>
>>
>> Please note, as it might be related to my issue, that what comes at
>> the end of the dmesg output seems suspicious to me (RIP  []
>>
>> xen_spin_lock+0x21/0x50 and the lines around that)
>>
>> I hope that I have provided enough information for further
>> investigation. The computer is not in any kind of production-use, so
>> please feel free to request things that will or may require me to
>> reinstall the operating system or some of its components. As the
>> hardware is new, I''d not be happy if I had to do something
that would
>> risk permanent damage. :)
>>
>> Should this be the wrong mailing list for this kind of post, please
>> let me know where I can send it to instead.
>>
>> Thank you very much for your time, any help is highly appreciated, not
>> only regarding my primary problem (getting VGA passthrough to work)
>> but also the others mentioned, especially since they might be related.
>>
>> ______________________________**_________________
>> Xen-users mailing list
>> Xen-users@lists.xen.org [20]
>> http://lists.xen.org/xen-users [21]
>>
>>
>>
>> Links:
>> ------
>> [1] http://pastebin.com/KN74FWE6
>> [2]
>>
>> http://www.davidgis.fr/blog/**index.php?2013/04/05/937-xen-**
>> 43-unstable-vga-passthrough-**hd-7970-windows-7-64-bits-**
>>
with-more-than-3gb-for-ram<http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram>
>> [3] http://pastebin.com/kf94viHM
>> [4] http://pastebin.com/m0wLZzkF
>> [5] http://pastebin.com/pkUdLvuw
>> [6] mailto:davidtecher@yahoo.fr
>> [7]
>>
>> http://www.davidgis.fr/blog/**index.php?2013/04/05/937-xen-**
>> 43-unstable-vga-passthrough-**hd-7970-windows-7-64-bits-**
>>
with-more-than-3gb-for-ram<http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram>
>> [8] mailto:gu.sorenson@gmail.com
>> [9] mailto:davidtecher@yahoo.fr
>> [10] mailto:davidtecher@yahoo.fr
>> [11]
>>
>> http://www.davidgis.fr/blog/**index.php?2013/04/05/937-xen-**
>> 43-unstable-vga-passthrough-**hd-7970-windows-7-64-bits-**
>>
with-more-than-3gb-for-ram<http://www.davidgis.fr/blog/index.php?2013/04/05/937-xen-43-unstable-vga-passthrough-hd-7970-windows-7-64-bits-with-more-than-3gb-for-ram>
>> [12] mailto:gu.sorenson@gmail.com
>> [13] mailto:xen-users@lists.xen.org
>> [14]
http://forums.linuxmint.com/**viewtopic.php?f=42&t=112013<http://forums.linuxmint.com/viewtopic.php?f=42&t=112013>
>> [15] http://pastebin.com/kwUWkyP2
>> [16] http://pastebin.com/E9jkkJXj
>> [17] http://pastebin.com/nj1ykFXJ
>> [18] http://pastebin.com/MS96knmL
>> [19] http://pastebin.com/2sQFuCuJ
>> [20] mailto:Xen-users@lists.xen.org
>> [21] http://lists.xen.org/xen-users
>>
>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

> I wasn''t aware that there was a restriction
> regarding the DomU RAM. However, it just so
> happens that I always tried with "memory=2048".
 So far so good.
> I assume this satisfies the <= 2GB RAM
> requirement, although I''m not sure since I
> don''t know whether you refer to "base-10 G"
> or "base-2 G". :)
 Base 2. Perhaps I should have said 2GiB. :)
> Furthermore, I don''t know whether the setting
> "shadow_memory = 512" counts towards that
> limit,
 What does that do?
> or whether the 1024MB that has been assigned
> to the integrated GPU in BIOS does.
 Your GPU has a 1GB BAR? Really? I don''t think
 I''ve ever seen GPU BARs > 256MB. That could
 potentially shrink the amount of RAM you
 can reliably give the domU. Try with
 memory=1024

 Note: This should be fixed in 4.3, but I haven''t
 tried it yet.
> Yes, I have tried without any first party
> patches, as I had written in my first mail in
> this thread. However, I haven''t tried
> "unpatched" xen with kernel 3.8.13 yet - should I?
 I got this working with 4.2.2 + XSA patches
 (mainly because that is what is in the RPMs I
 use, I''m on EL6).
> Unfortunately, I don''t have a spare dedicated
> graphics card I could add right now. I will try
> to get one (is it important whether it''s PCI or
> PCIe, nVidia or ATI, or some specific model?)
> and report back (if) when I get hold of one.
 There is no particular requirement but it will
 make it simpler if you have a dom0 card that
 doesn''t use the same drivers as the domU cards.
 Otherwise, if you have xen-pciback as a module
 you have to do some extra configuration/scripting
 to ensure that upon loading the GPU driver, the
 xen-pciback gets assigned the GPUs you don''t want
 the dom0 driver to handle. My setup is like this
 at the moment and it works fine, but to make my
 life easier I am going to switch to using
 an old ATI 4850 (best I can get ATI-wise that is
 single-slot and has two DL-DVI ports) for dom0,
 and a pair of GeForce GTX480 modified into
 Quadro 6000s for two separate domUs. I''m
 currently using a GeForce 8800GT for dom0,
 which is what I''ll be replacing with a HD4850
 in the near future (i.e. when I get around to
 acquiring one).
> However, my primary goal was to build a
> computer that would serve two independent VMs
> with 3D acceleration each, e.g. so that two
> people could play games at the same time.
 This is _precisely_ my use case, with the
 extra requirement of also being my primary
 workstation at the same time (without that
 purpose having to be interrupted for gaming
 purposes).
> My plan was to buy the hardware as I have it
> right now, test whether I could make xen
> pass-through the one (integrated) GPU I have
> right now,
 Oh, is THIS what you were referring to by
 1GiB for the GPU? That''s not the same as BARs,
 disregard what I said above.
> and then buy a not-too-expensive
> but still quite powerful dedicated GPU.
> I would be much happier if I didn''t have
> to buy and install a third GPU, although
> I realize me wishing that doesn''t necessarily
> make it so. ;)
 My general experience is that integrated GPUs
 aren''t really up to gaming requirements in the
 majority of cases, so my advice would be to get
 a pair of different GPUs for VGA passthrough
 and stick with the integrated one for dom0.

 My experience with ATI for VGA passthrough has
 been somewhat poor, but provided you avoid
 PCI memory stomps and weird IRQ clash issues,
 the experience with Nvidia Quadros has been
 very positive. GeForce cards won''t work
 until/unless you modify them into equivalent
 Quadros. See:
 http://www.altechnative.net/2013/06/23/nvidia-cards-geforce-quadro-and-geforce-modified-into-a-quadro-for-virtualized-gaming/

 One thing that ATI users seem to be experiencing
 is progressive graphics slow-down after reboots
 in domUs with VGA passthrough, which requires
 host reboot to fix. I have not experienced this
 with my Quadrified GeForce cards.

 It took a lot of effort to get this working and
 work around all the issues, though. Which slots
 you have the hardware in makes a difference, as
 does the nature and number of PCIe bridges
 involved, as well as the combination of hardware
 you are passing through. ACS support on the
 PCIe bridges may also help ensure that a bug
 causing a PCI memory stomp doesn''t crash dom0,
 although I have managed to get things working
 stably and reliably without it.

 Gordan

Hi Gustav,

First, some more references:

- [Official Kernel Docs](
http://wiki.xen.org/wiki/Mainline_Linux_Kernel_Configs)
- [Official Compiling Xen Docs](
http://wiki.xen.org/wiki/Compiling_Xen_From_Source)

While they do not include steps for patching they may supplement
David''s
guide.

---

Second, I reviewed your DomU config, under the assumption that you are
using the `xl` toolstack and have some questions:

- What is `firmware_override`?
- What is `xen_extended_power_mgmt`?
- What is `monitor`?
- What is `audio`?

- Why set `extra` for an HVM?
- Why override the device model paths?
- Why use `shadow_memory` when you have HAP?
- Why set `serial` for a Windows VM?

- Why set `on_poweroff` to its default value?
- Why set `on_reboot` to its default value?
- Why set `sdl` to its default value?
- Why set `acpi` to its default value?
- Why set `apic` to its default value?

According to the Xen Man Pages, the xl configuration does not have a
`firmware_override`, `monitor`, `audio` or a `xen_extended_power_mgmt`
option.  The `firmware_override` option is for PV guests, aimed at linux
(nouveau being a linux nvidia driver?).  You can omit options instead of
setting them to their default value.

AFAIK the 2GB RAM limit applies to qemu upstream and you have set your
device model to traditional, so you _should not_ be subject to this
limitation (I can''t speak for all cases, but I have never had the
memory
problem with traditional).

Integrated isn''t great for gaming, but should be perfectly fine for
Dom0 to
use when you pass the discrete cards to VM''s.

As Gordon has mentioned, AMD cards do work without patches to Xen source,
but not as primary and they also experience performance degredation.  Since
moving to upstream qemu I have encountered the RAM limit and the ejection
trick no-longer works the same.  When I eject it does not automatically
reinitialize, instead it disappears from the DomU (and puts errors in my
qemu logs), but if I eject before shutting down or restarting I can
shutdown from VNC or SDL and the card works without the degredation when
started back up.  Otherwise the whole machine has to be restarted to
resolve performance problems.

I am still finding 4.3 to be buggy, so I haven''t nailed down my own
steps
to share, but I hope this information is helpful.

~Casey


On Thu, Jul 11, 2013 at 9:49 AM, Gordan Bobic <gordan@bobich.net> wrote:
>
>  I wasn''t aware that there was a restriction
>> regarding the DomU RAM. However, it just so
>> happens that I always tried with "memory=2048".
>>
>
> So far so good.
>
>
>  I assume this satisfies the <= 2GB RAM
>> requirement, although I''m not sure since I
>> don''t know whether you refer to "base-10 G"
>> or "base-2 G". :)
>>
>
> Base 2. Perhaps I should have said 2GiB. :)
>
>
>  Furthermore, I don''t know whether the setting
>> "shadow_memory = 512" counts towards that
>> limit,
>>
>
> What does that do?
>
>
>  or whether the 1024MB that has been assigned
>> to the integrated GPU in BIOS does.
>>
>
> Your GPU has a 1GB BAR? Really? I don''t think
> I''ve ever seen GPU BARs > 256MB. That could
> potentially shrink the amount of RAM you
> can reliably give the domU. Try with
> memory=1024
>
> Note: This should be fixed in 4.3, but I haven''t
> tried it yet.
>
>
>  Yes, I have tried without any first party
>> patches, as I had written in my first mail in
>> this thread. However, I haven''t tried
>> "unpatched" xen with kernel 3.8.13 yet - should I?
>>
>
> I got this working with 4.2.2 + XSA patches
> (mainly because that is what is in the RPMs I
> use, I''m on EL6).
>
>
>  Unfortunately, I don''t have a spare dedicated
>> graphics card I could add right now. I will try
>> to get one (is it important whether it''s PCI or
>> PCIe, nVidia or ATI, or some specific model?)
>> and report back (if) when I get hold of one.
>>
>
> There is no particular requirement but it will
> make it simpler if you have a dom0 card that
> doesn''t use the same drivers as the domU cards.
> Otherwise, if you have xen-pciback as a module
> you have to do some extra configuration/scripting
> to ensure that upon loading the GPU driver, the
> xen-pciback gets assigned the GPUs you don''t want
> the dom0 driver to handle. My setup is like this
> at the moment and it works fine, but to make my
> life easier I am going to switch to using
> an old ATI 4850 (best I can get ATI-wise that is
> single-slot and has two DL-DVI ports) for dom0,
> and a pair of GeForce GTX480 modified into
> Quadro 6000s for two separate domUs. I''m
> currently using a GeForce 8800GT for dom0,
> which is what I''ll be replacing with a HD4850
> in the near future (i.e. when I get around to
> acquiring one).
>
>
>  However, my primary goal was to build a
>> computer that would serve two independent VMs
>> with 3D acceleration each, e.g. so that two
>> people could play games at the same time.
>>
>
> This is _precisely_ my use case, with the
> extra requirement of also being my primary
> workstation at the same time (without that
> purpose having to be interrupted for gaming
> purposes).
>
>
>  My plan was to buy the hardware as I have it
>> right now, test whether I could make xen
>> pass-through the one (integrated) GPU I have
>> right now,
>>
>
> Oh, is THIS what you were referring to by
> 1GiB for the GPU? That''s not the same as BARs,
> disregard what I said above.
>
>
>  and then buy a not-too-expensive
>> but still quite powerful dedicated GPU.
>> I would be much happier if I didn''t have
>> to buy and install a third GPU, although
>> I realize me wishing that doesn''t necessarily
>> make it so. ;)
>>
>
> My general experience is that integrated GPUs
> aren''t really up to gaming requirements in the
> majority of cases, so my advice would be to get
> a pair of different GPUs for VGA passthrough
> and stick with the integrated one for dom0.
>
> My experience with ATI for VGA passthrough has
> been somewhat poor, but provided you avoid
> PCI memory stomps and weird IRQ clash issues,
> the experience with Nvidia Quadros has been
> very positive. GeForce cards won''t work
> until/unless you modify them into equivalent
> Quadros. See:
> http://www.altechnative.net/**2013/06/23/nvidia-cards-**
> geforce-quadro-and-geforce-**modified-into-a-quadro-for-**
>
virtualized-gaming/<http://www.altechnative.net/2013/06/23/nvidia-cards-geforce-quadro-and-geforce-modified-into-a-quadro-for-virtualized-gaming/>
>
> One thing that ATI users seem to be experiencing
> is progressive graphics slow-down after reboots
> in domUs with VGA passthrough, which requires
> host reboot to fix. I have not experienced this
> with my Quadrified GeForce cards.
>
> It took a lot of effort to get this working and
> work around all the issues, though. Which slots
> you have the hardware in makes a difference, as
> does the nature and number of PCIe bridges
> involved, as well as the combination of hardware
> you are passing through. ACS support on the
> PCIe bridges may also help ensure that a bug
> causing a PCI memory stomp doesn''t crash dom0,
> although I have managed to get things working
> stably and reliably without it.
>
> Gordan
>
>
> ______________________________**_________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Hello everyone,

thank you for your replies, Gordan and Casey.

@Gordan:
What are XSA patches, and should I give them a try?
In my setup, I won''t have the need to run more than two displays at a
time,
so I''d be happy if I was able to do with just the GPU integrated in the
APU
plus one dedicated GPU. Also, I know that with the integrated GPU, I
won''t
be able to play newer games at maximum settings in high resolutions, but
that''s fine for me.

Reducing the DomU RAM to 1024 MB didn''t solve my problem.

@Casey:
Your assumption is correct: I''m using the xl toolstack, sorry for not
mentioning that earlier.
Your questions regarding my settings in the DomU configuration all share
the same answer: Because David, who had succeeded in doing what I plan to
do, had those settings in his config, and I copied them over, only changing
things in cases where I was sure I knew the implications.

I''ve read about the performance degradation issues with AMD. While
it''s a
pity that this issue exists, it would be one I would be able to accept,
reading that nVidia only seems to work better when using Quadro cards which
are out of my budget range. Since I haven''t done any hardware hacking
before and therefore wouldn''t feel confident voiding my warranty,
hacking a
GeForce to get a cheaper Quadro isn''t really an option either.

Thank you both for your help.

However, I''ve run out of things I could try to get passthrough of the
integrated GPU to work, so I''d be very grateful for any further input.
Should this be a scenario that ought to be supported already, of course
I''m
willing to do the best I can to help debug this problem; please just let me
know how.

On Thu, Jul 11, 2013 at 4:51 PM, Casey DeLorme <cdelorme@gmail.com> wrote:
> Hi Gustav,
>
> First, some more references:
>
> - [Official Kernel Docs](
> http://wiki.xen.org/wiki/Mainline_Linux_Kernel_Configs)
> - [Official Compiling Xen Docs](
> http://wiki.xen.org/wiki/Compiling_Xen_From_Source)
>
> While they do not include steps for patching they may supplement
David''s
> guide.
>
> ---
>
> Second, I reviewed your DomU config, under the assumption that you are
> using the `xl` toolstack and have some questions:
>
> - What is `firmware_override`?
> - What is `xen_extended_power_mgmt`?
> - What is `monitor`?
> - What is `audio`?
>
> - Why set `extra` for an HVM?
> - Why override the device model paths?
> - Why use `shadow_memory` when you have HAP?
> - Why set `serial` for a Windows VM?
>
> - Why set `on_poweroff` to its default value?
> - Why set `on_reboot` to its default value?
> - Why set `sdl` to its default value?
> - Why set `acpi` to its default value?
> - Why set `apic` to its default value?
>
> According to the Xen Man Pages, the xl configuration does not have a
> `firmware_override`, `monitor`, `audio` or a `xen_extended_power_mgmt`
> option.  The `firmware_override` option is for PV guests, aimed at linux
> (nouveau being a linux nvidia driver?).  You can omit options instead of
> setting them to their default value.
>
> AFAIK the 2GB RAM limit applies to qemu upstream and you have set your
> device model to traditional, so you _should not_ be subject to this
> limitation (I can''t speak for all cases, but I have never had the
memory
> problem with traditional).
>
> Integrated isn''t great for gaming, but should be perfectly fine
for Dom0
> to use when you pass the discrete cards to VM''s.
>
> As Gordon has mentioned, AMD cards do work without patches to Xen source,
> but not as primary and they also experience performance degredation.  Since
> moving to upstream qemu I have encountered the RAM limit and the ejection
> trick no-longer works the same.  When I eject it does not automatically
> reinitialize, instead it disappears from the DomU (and puts errors in my
> qemu logs), but if I eject before shutting down or restarting I can
> shutdown from VNC or SDL and the card works without the degredation when
> started back up.  Otherwise the whole machine has to be restarted to
> resolve performance problems.
>
> I am still finding 4.3 to be buggy, so I haven''t nailed down my
own steps
> to share, but I hope this information is helpful.
>
> ~Casey
>
>
> On Thu, Jul 11, 2013 at 9:49 AM, Gordan Bobic <gordan@bobich.net>
wrote:
>
>>
>>  I wasn''t aware that there was a restriction
>>> regarding the DomU RAM. However, it just so
>>> happens that I always tried with "memory=2048".
>>>
>>
>> So far so good.
>>
>>
>>  I assume this satisfies the <= 2GB RAM
>>> requirement, although I''m not sure since I
>>> don''t know whether you refer to "base-10 G"
>>> or "base-2 G". :)
>>>
>>
>> Base 2. Perhaps I should have said 2GiB. :)
>>
>>
>>  Furthermore, I don''t know whether the setting
>>> "shadow_memory = 512" counts towards that
>>> limit,
>>>
>>
>> What does that do?
>>
>>
>>  or whether the 1024MB that has been assigned
>>> to the integrated GPU in BIOS does.
>>>
>>
>> Your GPU has a 1GB BAR? Really? I don''t think
>> I''ve ever seen GPU BARs > 256MB. That could
>> potentially shrink the amount of RAM you
>> can reliably give the domU. Try with
>> memory=1024
>>
>> Note: This should be fixed in 4.3, but I haven''t
>> tried it yet.
>>
>>
>>  Yes, I have tried without any first party
>>> patches, as I had written in my first mail in
>>> this thread. However, I haven''t tried
>>> "unpatched" xen with kernel 3.8.13 yet - should I?
>>>
>>
>> I got this working with 4.2.2 + XSA patches
>> (mainly because that is what is in the RPMs I
>> use, I''m on EL6).
>>
>>
>>  Unfortunately, I don''t have a spare dedicated
>>> graphics card I could add right now. I will try
>>> to get one (is it important whether it''s PCI or
>>> PCIe, nVidia or ATI, or some specific model?)
>>> and report back (if) when I get hold of one.
>>>
>>
>> There is no particular requirement but it will
>> make it simpler if you have a dom0 card that
>> doesn''t use the same drivers as the domU cards.
>> Otherwise, if you have xen-pciback as a module
>> you have to do some extra configuration/scripting
>> to ensure that upon loading the GPU driver, the
>> xen-pciback gets assigned the GPUs you don''t want
>> the dom0 driver to handle. My setup is like this
>> at the moment and it works fine, but to make my
>> life easier I am going to switch to using
>> an old ATI 4850 (best I can get ATI-wise that is
>> single-slot and has two DL-DVI ports) for dom0,
>> and a pair of GeForce GTX480 modified into
>> Quadro 6000s for two separate domUs. I''m
>> currently using a GeForce 8800GT for dom0,
>> which is what I''ll be replacing with a HD4850
>> in the near future (i.e. when I get around to
>> acquiring one).
>>
>>
>>  However, my primary goal was to build a
>>> computer that would serve two independent VMs
>>> with 3D acceleration each, e.g. so that two
>>> people could play games at the same time.
>>>
>>
>> This is _precisely_ my use case, with the
>> extra requirement of also being my primary
>> workstation at the same time (without that
>> purpose having to be interrupted for gaming
>> purposes).
>>
>>
>>  My plan was to buy the hardware as I have it
>>> right now, test whether I could make xen
>>> pass-through the one (integrated) GPU I have
>>> right now,
>>>
>>
>> Oh, is THIS what you were referring to by
>> 1GiB for the GPU? That''s not the same as BARs,
>> disregard what I said above.
>>
>>
>>  and then buy a not-too-expensive
>>> but still quite powerful dedicated GPU.
>>> I would be much happier if I didn''t have
>>> to buy and install a third GPU, although
>>> I realize me wishing that doesn''t necessarily
>>> make it so. ;)
>>>
>>
>> My general experience is that integrated GPUs
>> aren''t really up to gaming requirements in the
>> majority of cases, so my advice would be to get
>> a pair of different GPUs for VGA passthrough
>> and stick with the integrated one for dom0.
>>
>> My experience with ATI for VGA passthrough has
>> been somewhat poor, but provided you avoid
>> PCI memory stomps and weird IRQ clash issues,
>> the experience with Nvidia Quadros has been
>> very positive. GeForce cards won''t work
>> until/unless you modify them into equivalent
>> Quadros. See:
>> http://www.altechnative.net/**2013/06/23/nvidia-cards-**
>> geforce-quadro-and-geforce-**modified-into-a-quadro-for-**
>>
virtualized-gaming/<http://www.altechnative.net/2013/06/23/nvidia-cards-geforce-quadro-and-geforce-modified-into-a-quadro-for-virtualized-gaming/>
>>
>> One thing that ATI users seem to be experiencing
>> is progressive graphics slow-down after reboots
>> in domUs with VGA passthrough, which requires
>> host reboot to fix. I have not experienced this
>> with my Quadrified GeForce cards.
>>
>> It took a lot of effort to get this working and
>> work around all the issues, though. Which slots
>> you have the hardware in makes a difference, as
>> does the nature and number of PCIe bridges
>> involved, as well as the combination of hardware
>> you are passing through. ACS support on the
>> PCIe bridges may also help ensure that a bug
>> causing a PCI memory stomp doesn''t crash dom0,
>> although I have managed to get things working
>> stably and reliably without it.
>>
>> Gordan
>>
>>
>> ______________________________**_________________
>> Xen-users mailing list
>> Xen-users@lists.xen.org
>> http://lists.xen.org/xen-users
>>
>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Gustav,

I am not sure I understand exactly where you are getting held up at, but
were you able to get any video out of the card during passthrough or did it
immediately go to Code 43?  Also, can you share your lspci -v && lspci
--tv
output with us?


Toolstack is still a matter of preference, I use xl now so I don''t have
to
worry about the eventual transition, but it might be worth trying the xm
toolstack.  It''s clear that David has had much more luck using it than
xl.

However, I would update your DomU configuration first and give it another
try without David''s script.  His script is meant to pass the devices at
run-time, which is helpful if you have two DomU''s and only two graphics
cards (eg. your discrete & integrated), but if you are leaving the
integrated attached to Dom0 (or using ssh) that''s not really a problem.


One of the biggest problems I have had with passthrough is the degraded
state.  Not everyone has had the same experiences, but for me whenever the
card is initialized (eg. Dom0 boot without passthrough, any DomU boot or
reboot) subsequent initializations cause the degraded state.  For me while
in that degraded state installing and removing drivers has never succeeded.
 It also is unreversible in the Windows 7 VM''s I tried.  So my approach
is
generally to setup the system over VNC or SDL first, omitting all pci
devices, then creating a dd backup of the logical volume to restore in the
event of a problem during graphics install.

I won''t be home for few more hours, but let me know if you want me to
send
the documentation I have currently.  My current system is not entirely
error-free (degradation and RAM limit exist and it has behaved a bit odd
during some tests), but I have a working Windows DomU /w passthrough.

~Casey


On Thu, Jul 11, 2013 at 4:53 PM, Gustav Sorenson
<gu.sorenson@gmail.com>wrote:
> Hello everyone,
>
> thank you for your replies, Gordan and Casey.
>
> @Gordan:
> What are XSA patches, and should I give them a try?
> In my setup, I won''t have the need to run more than two displays
at a
> time, so I''d be happy if I was able to do with just the GPU
integrated in
> the APU plus one dedicated GPU. Also, I know that with the integrated GPU,
> I won''t be able to play newer games at maximum settings in high
> resolutions, but that''s fine for me.
>
> Reducing the DomU RAM to 1024 MB didn''t solve my problem.
>
> @Casey:
> Your assumption is correct: I''m using the xl toolstack, sorry for
not
> mentioning that earlier.
> Your questions regarding my settings in the DomU configuration all share
> the same answer: Because David, who had succeeded in doing what I plan to
> do, had those settings in his config, and I copied them over, only changing
> things in cases where I was sure I knew the implications.
>
> I''ve read about the performance degradation issues with AMD. While
it''s a
> pity that this issue exists, it would be one I would be able to accept,
> reading that nVidia only seems to work better when using Quadro cards which
> are out of my budget range. Since I haven''t done any hardware
hacking
> before and therefore wouldn''t feel confident voiding my warranty,
hacking a
> GeForce to get a cheaper Quadro isn''t really an option either.
>
> Thank you both for your help.
>
> However, I''ve run out of things I could try to get passthrough of
the
> integrated GPU to work, so I''d be very grateful for any further
input.
> Should this be a scenario that ought to be supported already, of course
> I''m willing to do the best I can to help debug this problem;
please just
> let me know how.
>
> On Thu, Jul 11, 2013 at 4:51 PM, Casey DeLorme <cdelorme@gmail.com>
wrote:
>
>> Hi Gustav,
>>
>> First, some more references:
>>
>> - [Official Kernel Docs](
>> http://wiki.xen.org/wiki/Mainline_Linux_Kernel_Configs)
>> - [Official Compiling Xen Docs](
>> http://wiki.xen.org/wiki/Compiling_Xen_From_Source)
>>
>> While they do not include steps for patching they may supplement
David''s
>> guide.
>>
>> ---
>>
>> Second, I reviewed your DomU config, under the assumption that you are
>> using the `xl` toolstack and have some questions:
>>
>> - What is `firmware_override`?
>> - What is `xen_extended_power_mgmt`?
>> - What is `monitor`?
>> - What is `audio`?
>>
>> - Why set `extra` for an HVM?
>> - Why override the device model paths?
>> - Why use `shadow_memory` when you have HAP?
>> - Why set `serial` for a Windows VM?
>>
>> - Why set `on_poweroff` to its default value?
>> - Why set `on_reboot` to its default value?
>> - Why set `sdl` to its default value?
>>  - Why set `acpi` to its default value?
>> - Why set `apic` to its default value?
>>
>> According to the Xen Man Pages, the xl configuration does not have a
>> `firmware_override`, `monitor`, `audio` or a `xen_extended_power_mgmt`
>> option.  The `firmware_override` option is for PV guests, aimed at
linux
>> (nouveau being a linux nvidia driver?).  You can omit options instead
of
>> setting them to their default value.
>>
>> AFAIK the 2GB RAM limit applies to qemu upstream and you have set your
>> device model to traditional, so you _should not_ be subject to this
>> limitation (I can''t speak for all cases, but I have never had
the memory
>> problem with traditional).
>>
>> Integrated isn''t great for gaming, but should be perfectly
fine for Dom0
>> to use when you pass the discrete cards to VM''s.
>>
>> As Gordon has mentioned, AMD cards do work without patches to Xen
source,
>> but not as primary and they also experience performance degredation. 
Since
>> moving to upstream qemu I have encountered the RAM limit and the
ejection
>> trick no-longer works the same.  When I eject it does not automatically
>> reinitialize, instead it disappears from the DomU (and puts errors in
my
>> qemu logs), but if I eject before shutting down or restarting I can
>> shutdown from VNC or SDL and the card works without the degredation
when
>> started back up.  Otherwise the whole machine has to be restarted to
>> resolve performance problems.
>>
>> I am still finding 4.3 to be buggy, so I haven''t nailed down
my own steps
>> to share, but I hope this information is helpful.
>>
>> ~Casey
>>
>>
>> On Thu, Jul 11, 2013 at 9:49 AM, Gordan Bobic <gordan@bobich.net>
wrote:
>>
>>>
>>>  I wasn''t aware that there was a restriction
>>>> regarding the DomU RAM. However, it just so
>>>> happens that I always tried with "memory=2048".
>>>>
>>>
>>> So far so good.
>>>
>>>
>>>  I assume this satisfies the <= 2GB RAM
>>>> requirement, although I''m not sure since I
>>>> don''t know whether you refer to "base-10 G"
>>>> or "base-2 G". :)
>>>>
>>>
>>> Base 2. Perhaps I should have said 2GiB. :)
>>>
>>>
>>>  Furthermore, I don''t know whether the setting
>>>> "shadow_memory = 512" counts towards that
>>>> limit,
>>>>
>>>
>>> What does that do?
>>>
>>>
>>>  or whether the 1024MB that has been assigned
>>>> to the integrated GPU in BIOS does.
>>>>
>>>
>>> Your GPU has a 1GB BAR? Really? I don''t think
>>> I''ve ever seen GPU BARs > 256MB. That could
>>> potentially shrink the amount of RAM you
>>> can reliably give the domU. Try with
>>> memory=1024
>>>
>>> Note: This should be fixed in 4.3, but I haven''t
>>> tried it yet.
>>>
>>>
>>>  Yes, I have tried without any first party
>>>> patches, as I had written in my first mail in
>>>> this thread. However, I haven''t tried
>>>> "unpatched" xen with kernel 3.8.13 yet - should I?
>>>>
>>>
>>> I got this working with 4.2.2 + XSA patches
>>> (mainly because that is what is in the RPMs I
>>> use, I''m on EL6).
>>>
>>>
>>>  Unfortunately, I don''t have a spare dedicated
>>>> graphics card I could add right now. I will try
>>>> to get one (is it important whether it''s PCI or
>>>> PCIe, nVidia or ATI, or some specific model?)
>>>> and report back (if) when I get hold of one.
>>>>
>>>
>>> There is no particular requirement but it will
>>> make it simpler if you have a dom0 card that
>>> doesn''t use the same drivers as the domU cards.
>>> Otherwise, if you have xen-pciback as a module
>>> you have to do some extra configuration/scripting
>>> to ensure that upon loading the GPU driver, the
>>> xen-pciback gets assigned the GPUs you don''t want
>>> the dom0 driver to handle. My setup is like this
>>> at the moment and it works fine, but to make my
>>> life easier I am going to switch to using
>>> an old ATI 4850 (best I can get ATI-wise that is
>>> single-slot and has two DL-DVI ports) for dom0,
>>> and a pair of GeForce GTX480 modified into
>>> Quadro 6000s for two separate domUs. I''m
>>> currently using a GeForce 8800GT for dom0,
>>> which is what I''ll be replacing with a HD4850
>>> in the near future (i.e. when I get around to
>>> acquiring one).
>>>
>>>
>>>  However, my primary goal was to build a
>>>> computer that would serve two independent VMs
>>>> with 3D acceleration each, e.g. so that two
>>>> people could play games at the same time.
>>>>
>>>
>>> This is _precisely_ my use case, with the
>>> extra requirement of also being my primary
>>> workstation at the same time (without that
>>> purpose having to be interrupted for gaming
>>> purposes).
>>>
>>>
>>>  My plan was to buy the hardware as I have it
>>>> right now, test whether I could make xen
>>>> pass-through the one (integrated) GPU I have
>>>> right now,
>>>>
>>>
>>> Oh, is THIS what you were referring to by
>>> 1GiB for the GPU? That''s not the same as BARs,
>>> disregard what I said above.
>>>
>>>
>>>  and then buy a not-too-expensive
>>>> but still quite powerful dedicated GPU.
>>>> I would be much happier if I didn''t have
>>>> to buy and install a third GPU, although
>>>> I realize me wishing that doesn''t necessarily
>>>> make it so. ;)
>>>>
>>>
>>> My general experience is that integrated GPUs
>>> aren''t really up to gaming requirements in the
>>> majority of cases, so my advice would be to get
>>> a pair of different GPUs for VGA passthrough
>>> and stick with the integrated one for dom0.
>>>
>>> My experience with ATI for VGA passthrough has
>>> been somewhat poor, but provided you avoid
>>> PCI memory stomps and weird IRQ clash issues,
>>> the experience with Nvidia Quadros has been
>>> very positive. GeForce cards won''t work
>>> until/unless you modify them into equivalent
>>> Quadros. See:
>>> http://www.altechnative.net/**2013/06/23/nvidia-cards-**
>>> geforce-quadro-and-geforce-**modified-into-a-quadro-for-**
>>>
virtualized-gaming/<http://www.altechnative.net/2013/06/23/nvidia-cards-geforce-quadro-and-geforce-modified-into-a-quadro-for-virtualized-gaming/>
>>>
>>> One thing that ATI users seem to be experiencing
>>> is progressive graphics slow-down after reboots
>>> in domUs with VGA passthrough, which requires
>>> host reboot to fix. I have not experienced this
>>> with my Quadrified GeForce cards.
>>>
>>> It took a lot of effort to get this working and
>>> work around all the issues, though. Which slots
>>> you have the hardware in makes a difference, as
>>> does the nature and number of PCIe bridges
>>> involved, as well as the combination of hardware
>>> you are passing through. ACS support on the
>>> PCIe bridges may also help ensure that a bug
>>> causing a PCI memory stomp doesn''t crash dom0,
>>> although I have managed to get things working
>>> stably and reliably without it.
>>>
>>> Gordan
>>>
>>>
>>> ______________________________**_________________
>>> Xen-users mailing list
>>> Xen-users@lists.xen.org
>>> http://lists.xen.org/xen-users
>>>
>>
>>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

On 07/11/2013 09:53 PM, Gustav Sorenson wrote:
> Hello everyone,
>
> thank you for your replies, Gordan and Casey.
>
> @Gordan:
> What are XSA patches, and should I give them a try?
Xen Security Advisories. Probably no need to pursue them in a fully 
private environment, at least until you have everything working the way 
you want it.
> In my setup, I won''t have the need to run more than two displays
at a
> time, so I''d be happy if I was able to do with just the GPU
integrated
> in the APU plus one dedicated GPU. Also, I know that with the integrated
> GPU, I won''t be able to play newer games at maximum settings in
high
> resolutions, but that''s fine for me.
It''s not a question of performance, it''s a question of primary
passthrough still being problematic. If your BIOS lets you select the 
primary POST VGA card, it might be worth asking around among local 
contacts if anyone has an old PCI VGA card (e.g. I have an ancient PCI 
ATI Mach64 in one of my machines purely to drive the primary console). 
That would leave you the on-board GPU untainted by being dom0 primary 
free for passthrough.
> @Casey:
> Your assumption is correct: I''m using the xl toolstack, sorry for
not
> mentioning that earlier.
I got my VMs working with both xm and xl tool stacks with no notable 
changes to anything - I don''t think this is an issue any more. I
started
with xm because that was what libvirt was using, but since then I 
ditched libvirt and virt-namager and now I just use text domU configs 
with xl. Much more visible, and future proof since xm is getting deprecated.

One thing worth noting is that my 4.2.2 packages only seem to include 
qemu-dm, so that''s what I''m using.
> I''ve read about the performance degradation issues with AMD. While
it''s
> a pity that this issue exists, it would be one I would be able to
> accept, reading that nVidia only seems to work better when using Quadro
> cards which are out of my budget range.
Check the article I linked earlier - you can grab a GeForce card and 
modify it into a Quadro pretty painlessly:

GTS450 -> Quadro 2000
GTX470 -> Quadro 5000
GTX480 -> Quadro 6000
GTX580 -> Quadro 7000 (Note: Unsupported for VGA passthrough and thus 
non-applicable for this purpose.)

I''m still perfecting the process for GTX680/GTX690 -> Grid K2
modding.
Grid cards are primarily designed for virtualization and reportedly work 
great, but I won''t be able to confirm that until the required modding 
bits and pieces arrive in the post.

I haven''t gotten around to writing up the details of the recipe for 
doing it yet, but I will as soon as I can find some time.
> Since I haven''t done any
> hardware hacking before and therefore wouldn''t feel confident
voiding my
> warranty, hacking a GeForce to get a cheaper Quadro isn''t really
an
> option either.
Warranty? What warranty? GTX480 cards are all out of warranty by now, 
but they are essentially the same GPU as the GTX580 (better in some 
ways, at least once you modify them). Quadrified GTX480 is probably the 
best value option there is for this sort of thing.
> However, I''ve run out of things I could try to get passthrough of
the
> integrated GPU to work, so I''d be very grateful for any further
input.
I''m not sure I can help much with this specific scenario - I discarded 
the primary GPU passthrough option pre-emptively because initial 
research suggested it was problematic, required out-of-tree patches and 
generally had a lower success rate than secondary passthrough.

Gordan

On 07/11/2013 10:46 PM, Casey DeLorme wrote:
> One of the biggest problems I have had with passthrough is the degraded
> state.  Not everyone has had the same experiences, but for me whenever
> the card is initialized (eg. Dom0 boot without passthrough, any DomU
> boot or reboot) subsequent initializations cause the degraded state.
That''s a fair point - Gustav, do you have the radeon driver
blacklisted?
In lspci -vvv, does it show up as bound to the radeon driver or the 
pciback driver?
>   For me while in that degraded state installing and removing drivers
> has never succeeded.  It also is unreversible in the Windows 7
VM''s I
> tried.  So my approach is generally to setup the system over VNC or SDL
> first, omitting all pci devices, then creating a dd backup of the
> logical volume to restore in the event of a problem during graphics
install.
Snapshots help. I have learned to love ZFS for this kind of thing.

Gordan

Hello everyone,

thanks again for your help.

I had created a file /etc/modprobe.d/blacklist-radeon.conf which contains
the line "blacklist radeon". lsmod doesn''t show radeon, as
expected.

Output of lspci -tv: http://pastebin.com/Tnh4afkQ
Output of lspci -v: http://pastebin.com/7L8gNgLa
Output of lspci -vvv: http://pastebin.com/2CupdqnD

If I read that correctly, the integrated GPU devices are listed as taken by
pciback.

Still looking forward to any suggestions. :)

Thanks!


On Fri, Jul 12, 2013 at 12:35 AM, Gordan Bobic <gordan@bobich.net> wrote:
> On 07/11/2013 10:46 PM, Casey DeLorme wrote:
>
>  One of the biggest problems I have had with passthrough is the degraded
>> state.  Not everyone has had the same experiences, but for me whenever
>> the card is initialized (eg. Dom0 boot without passthrough, any DomU
>> boot or reboot) subsequent initializations cause the degraded state.
>>
>
> That''s a fair point - Gustav, do you have the radeon driver
blacklisted?
> In lspci -vvv, does it show up as bound to the radeon driver or the pciback
> driver?
>
>
>    For me while in that degraded state installing and removing drivers
>> has never succeeded.  It also is unreversible in the Windows 7
VM''s I
>> tried.  So my approach is generally to setup the system over VNC or SDL
>> first, omitting all pci devices, then creating a dd backup of the
>> logical volume to restore in the event of a problem during graphics
>> install.
>>
>
> Snapshots help. I have learned to love ZFS for this kind of thing.
>
> Gordan
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

On 12/07/2013 20:02, Gustav Sorenson wrote:
> Hello everyone,
>
> thanks again for your help.
>
> I had created a file /etc/modprobe.d/blacklist-radeon.conf which 
> contains the line "blacklist radeon". lsmod doesn''t show
radeon, as
> expected.
>
> Output of lspci -tv: http://pastebin.com/Tnh4afkQ
> Output of lspci -v: http://pastebin.com/7L8gNgLa
> Output of lspci -vvv: http://pastebin.com/2CupdqnD
>
> If I read that correctly, the integrated GPU devices are listed as 
> taken by pciback.
>
> Still looking forward to any suggestions. :)
>
> Thanks!
>
>
> On Fri, Jul 12, 2013 at 12:35 AM, Gordan Bobic <gordan@bobich.net 
> <mailto:gordan@bobich.net>> wrote:
>
>     On 07/11/2013 10:46 PM, Casey DeLorme wrote:
>
>         One of the biggest problems I have had with passthrough is the
>         degraded
>         state.  Not everyone has had the same experiences, but for me
>         whenever
>         the card is initialized (eg. Dom0 boot without passthrough,
>         any DomU
>         boot or reboot) subsequent initializations cause the degraded
>         state.
>
>
>     That''s a fair point - Gustav, do you have the radeon driver
>     blacklisted? In lspci -vvv, does it show up as bound to the radeon
>     driver or the pciback driver?
>
>
>           For me while in that degraded state installing and removing
>         drivers
>         has never succeeded.  It also is unreversible in the Windows 7
>         VM''s I
>         tried.  So my approach is generally to setup the system over
>         VNC or SDL
>         first, omitting all pci devices, then creating a dd backup of the
>         logical volume to restore in the event of a problem during
>         graphics install.
>
>
>     Snapshots help. I have learned to love ZFS for this kind of thing.
>
>     Gordan
>
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
Dear Gustav,

Have you solved your Code 43 error? I am having the same Code 43 error 
as you.

-- 
Yours sincerely,

Singapore Citizen Mr. Teo En Ming (Zhang Enming)



_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

> Have you solved your Code 43 error? I am having the same Code 43 error as
> you.
No, unfortunately, this still does not work for me. Please do let me
know in case you solve your issue.

Thanks.

On 31/07/2013 23:55, Gustav Sorenson wrote:
>> Have you solved your Code 43 error? I am having the same Code 43 error
as
>> you.
> No, unfortunately, this still does not work for me. Please do let me
> know in case you solve your issue.
>
> Thanks.
>
Dear Gustav,

I will let you know once I solve my problem. If you solve your problem, 
please let me know as well. Meanwhile, why not try using the guides I 
have written to see if vga passthrough works in your case? You can 
download my tutorials from the xen-users thread titled "Latest Xen 
Tutorials". I will also email you the pdf documents separately.

-- 
Yours sincerely,

Singapore Citizen Mr. Teo En Ming (Zhang Enming)