Xen devel - Jul 2013 - Xen 4.3 Release VGA Passthrough Questions

Hello xen-devel,

I was hoping to ask some VGA passthrough questions with regards to the 4.3
release, and going forward.  I love Xen, and want to help iron out the
problems in relation to passthrough.  I am very impressed with the
performance of upstream qemu, especially network management.


I use Debian as my Dom0, as it is the linux distro I am most familiar with.
 My key hardware includes an Intel Core i7 3770 IvyBridge CPU, ASRock Z77
Extreme9 Motherboard, and an AMD Radeon HD 6870 GPU.  My system has 32GB of
RAM and I have been running an average of 4 virtual machines at a time when
I was using Xen 4.2.


The problems I would like to address area:

- RAM Limitations /w Upstream Qemu
- Performance Degredation
- Primary Passthrough


Upstream Qemu and VGA Passthrough limits my supplied RAM.  In my case
supplying anything more than 3584MB and the machine boots but the graphics
card is not used (but does exist).  I tried applying a patch that was sent
around the xen-users list, but with that patch my Windows HVM (even without
any changes to RAM) goes to an infinite boot screen in windows (as seen
through sdl or vnc).


While I am able to overcome these limitations by switching to qemu
traditional, I have other problems that occassionally kill my network when
I hit heavy traffic.  These may be related to GPLPV and not Xen, but
attempting to re-enable the adpater fails, the adapter disappears from the
system, and my only option is to reboot.  Disabling the adapter with
upstream and GPLPV has the same issue, but I have not encountered the same
problem with the network adapter crashing under load.


The performance degradation problem exists in both upstream and
traditional, and may have nothing to do with Xen.  I don''t expect the
devels to solve this, but it would be nice if they could share some
knowledge since my understanding of linux is likely vastly inferior.

A change with upstream is ejecting the card in Windows does not
automatically re-attach, in fact it disappears from the DomU.  The
degredation is gone when I reboot, so I can do these before rebooting
instead of after like with traditional, but the difference is worth noting.

Ideally I want to find a way to reset the card from Dom0, and thus far I
have tried several things in sysfs unsuccessfully.  If anyone has
suggestions I would be glad to try them.


Finally, I tried primary passthrough as suggested on the xen-users list.
 After a bit of struggling trying to apply the patch (due to stubdom not
building) I ended up with this process:

    make world
    cd tools
    make clean
    cd ..
    cat ../xen-* | patch -p1
    make dist-tools
    fakeroot sh ./tools/misc/mkdeb /home/cdelorme/git/xen 4.3.0

The patch applied and I was able to use a .deb for easy install and
removal.  I don''t know if there is a solution to the lack of a sys/io.h
in
stubdom, but any suggestions are welcomed.

I can boot machines the same as I used to prior to this patch, both
traditional and upstream appear to work normally.  However, the degradation
still occurs, and primary passthrough fails.  From what I can tell the
vCPU''s never even spin up.

I have created a pastebin with my windows config, the output of xl -vvv
create, the qemu log, and the xl list showing 1 vcpu of the 4 supplied:

http://pastebin.com/YmqkY8qS


I hope that I can be of use going forward, and that someone in the devel
list has some ideas for me to try out.

Thanks for your time,

~Casey


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Fri, Jul 19, 2013 at 11:55 PM, Casey DeLorme <cdelorme@gmail.com>
wrote:
> Hello xen-devel,
>
> I was hoping to ask some VGA passthrough questions with regards to the 4.3
> release, and going forward.  I love Xen, and want to help iron out the
> problems in relation to passthrough.  I am very impressed with the
> performance of upstream qemu, especially network management.
>
>
> I use Debian as my Dom0, as it is the linux distro I am most familiar with.
> My key hardware includes an Intel Core i7 3770 IvyBridge CPU, ASRock Z77
> Extreme9 Motherboard, and an AMD Radeon HD 6870 GPU.  My system has 32GB of
> RAM and I have been running an average of 4 virtual machines at a time when
> I was using Xen 4.2.
>
>
> The problems I would like to address area:
>
> - RAM Limitations /w Upstream Qemu
> - Performance Degredation
> - Primary Passthrough
>
>
> Upstream Qemu and VGA Passthrough limits my supplied RAM.  In my case
> supplying anything more than 3584MB and the machine boots but the graphics
> card is not used (but does exist).  I tried applying a patch that was sent
> around the xen-users list, but with that patch my Windows HVM (even without
> any changes to RAM) goes to an infinite boot screen in windows (as seen
> through sdl or vnc).
What guest are you using?

There is a known issue with qemu-upstream and PCI pass-through that we
found too late in the release cycle to do a proper fix, so we had to
do a work-around.  Behavior like what you describe was known to be one
of the possible side-effects of the work-around, particularly for
32-bit guests, but it''s still better than having the guest crash
(which is what would happen without the work-around).

Re your other issues, it''s better if you send one issue per e-mail.
They also need more detail to be useful -- see below.
> While I am able to overcome these limitations by switching to qemu
> traditional, I have other problems that occassionally kill my network when
I
> hit heavy traffic.  These may be related to GPLPV and not Xen, but
> attempting to re-enable the adpater fails, the adapter disappears from the
> system, and my only option is to reboot.  Disabling the adapter with
> upstream and GPLPV has the same issue, but I have not encountered the same
> problem with the network adapter crashing under load.
I''ve heard this issue before -- CC the GPLPV maintainer when you send
your mail.
> The performance degradation problem exists in both upstream and
traditional,
> and may have nothing to do with Xen.  I don''t expect the devels to
solve
> this, but it would be nice if they could share some knowledge since my
> understanding of linux is likely vastly inferior.
Well you haven''t said exactly what you''re seeing, so short of
doing a
full brain dump, it''s not clear what particular knowledge to share
with you. :-)  You need to describe in more detail your setup, and how
exactly you''re measuring the performance.

 -George