Pkg xen devel - Feb 2017 - Bug#776742: xen-utils-common: no support for VGA Passthrough

This bug, at its core, is that currently there is no supported solution 
for VGA
passthrough on Xen for stable version Jessie from Debian.

After browsing Xen's repositories, I found out that Xen did not claim to 
support
VGA passthrough with the upstream qemu-xen device model until Sep 25, 
2015, the
date the xl.cfg man page was updated to indicate support for VGA passthrough
with upstream qemu-xen. This change to the xl.cfg man page was only made 
on the
Xen version 4.7 and 4.8 branches, so if you want to use VGA passthrough 
without
the traditional qemu-dm binary, you must upgrade to at least Xen version 
4.7.
Debian testing (currently stretch) uses Xen 4.8 and it presumably 
supports VGA
passthrough without qemu-xen-traditional but I have not tried it.

This situation leaves users of Debian stable (currently Jessie) with no
supported solution from Debian for VGA passthrough on Xen. Obviously 
there are
two solutions. Backport Xen 4.7 or greater to Jessie, or restore the
traditional qemu-dm binary to the Xen 4.4.x package for Jessie.

A couple of months ago I decided to try and rebuild the Xen source package
for Jessie with support for qemu-xen-traditional from upstream included. 
It did
not take long to get a working package that solves this bug. I 
discovered the
following facts:

1. Adding qemu-xen-traditional in a way supported by Xen also requires 
rombios
    which, like qemu-xen-traditional, is disabled in Debian's official 
build of
    Xen for Jessie.

2. After configuring the build for qemu-xen-traditional and rombios, the 
only
    binary package that is modified significantly is xen-utils-4.4, which is
    where the qemu-dm binary is installed, and where a modified 
hvmloader binary
    is installed so it includes the statically linked rombios modules.

3. With these changes to xen-utils-4.4, together with a few simple 
configuration
    tweaks such as the location of the qemu keymaps directory, VGA 
passthrough
    on Debian Jessie works with HVM domains that use qemu-xen-traditional,
    and this feature can be enabled simply by adding 2 lines to the config
    file: device_model_version = 'qemu-xen-traditional'
          gfx_passthru = '1'
    I think the second option is only needed if you want the passed through
    video card to be the primary video adapter, but I have never tried VGA
    passthrough as a secondary adapter in the HVM. Your mileage may vary.

4. Also, with these changes to xen-utils, the qemu-xen upstream device model
    and the seabios bios remain the defaults, so these changes will not do
    violence to existing installations that use upstream qemu-xen and 
seabios.

5. It is not possible to build the Xen source package with 
qemu-xen-traditional
    in such a way that it meets all the requirements for inclusion in 
the main
    area of Debian's archive of packages. Specifically, the build must 
access
    remote repositories to download qemu-xen-traditional source files from
    xen.org and ipxe source files from ipxe.org. Later I found it is 
possible to
    build and configure a working package that supports VGA passthrough 
without
    ipxe by making a couple of small configuration tweaks that remove 
etherboot
    and ipxe support from the rombios modules that are statically linked 
into
    hvmloader. But the requirement that the build not access the 
internet for
    source files cannot be met unless the xen source package is modified
    by adding another original source tar archive, something like the
    xen_4.1.6.lts1.orig-qemu.tar.xz source file for the xen package for 
oldstable
    Wheezy that does include support for qemu-xen-traditional and VGA
    passthrough, in order for the build to succeed without needing to access
    a remote repository.

With these discoveries, I would like to make the observation that 
although it is
not trivial, it seems like it would not be too dificult for the Debian 
Xen team
to add support for qemu-xen-traditional and rombios to the Xen source 
package so
the resulting binary packages would support VGA passthrough when an HVM 
domain
is configured to use qemu-xen-traditional.

After having said all this, the stage is set for my question: Why hasn't the
Debian Xen team added support for VGA passthrough in Debian Jessie?

 From Archived Bug #688311, Bastian Blank asserted that qemu-dm will not 
come
back and that you have to use the normal (now called upstream) qemu.

Ian Campbell responded: "Just FYI position of upstream is that the 
'traditional'
qemu-dm will continue to be maintained for the foreseeable future for the
benefit of users who are running guest OSes which are not as happy to 
have the
"motherboard" changed from under them as Linux is."

That was over four years ago, and Debian still has not restored qemu-dm so
users of a stable Debian system can have a working VGA Passthrough for any
guest OS, not even Linux, much less users of those Oses that don't like the
motherboard changed from other them.

As far as I can tell, Xen still maintains the traditional qemu-dm, and I was
able to recently rebuild the xen package for Jessie and get VGA Passthrough
working on Jessie with the most recent version of the traditional device 
model
that is available from Xen for the stable 4.4 release of Xen.

So I ask again, why can't the Debian Xen team restore qemu-dm to its 
official
Xen package for Jessie?

The only reasonable reasons I can think of is that there is some free 
software
licensing issue with the rombios modules that are statically linked to 
hvmloader
or with some necessary component of qemu-dm, or the Debian Xen team has 
too few
resources and is devoting its efforts to developing Xen for stretch 
rather than
adding feaatures that did not make it into Jessie when it was released. 
But why
should oldstable Wheezy have a feature that stable Jessie does not have?

In any case, I hope the Debian Xen team can explain why qemu-dm cannot be
restored to Debian Jessie's offical xen package.

Thank you for your consideration of my question.

Sincerely,

Chuck Zmudzinski

can you add your hw specs and also what about Debian-Stretch?
J

----- On 12 Feb, 2017, at 18:05, Chuck Zmudzinski brchuckz at netscape.net
wrote:
> This bug, at its core, is that currently there is no supported solution
> for VGA
> passthrough on Xen for stable version Jessie from Debian.
> 
> After browsing Xen's repositories, I found out that Xen did not claim
to
> support
> VGA passthrough with the upstream qemu-xen device model until Sep 25,
> 2015, the
> date the xl.cfg man page was updated to indicate support for VGA
passthrough
> with upstream qemu-xen. This change to the xl.cfg man page was only made
> on the
> Xen version 4.7 and 4.8 branches, so if you want to use VGA passthrough
> without
> the traditional qemu-dm binary, you must upgrade to at least Xen version
> 4.7.
> Debian testing (currently stretch) uses Xen 4.8 and it presumably
> supports VGA
> passthrough without qemu-xen-traditional but I have not tried it.
> 
> This situation leaves users of Debian stable (currently Jessie) with no
> supported solution from Debian for VGA passthrough on Xen. Obviously
> there are
> two solutions. Backport Xen 4.7 or greater to Jessie, or restore the
> traditional qemu-dm binary to the Xen 4.4.x package for Jessie.
> 
> A couple of months ago I decided to try and rebuild the Xen source package
> for Jessie with support for qemu-xen-traditional from upstream included.
> It did
> not take long to get a working package that solves this bug. I
> discovered the
> following facts:
> 
> 1. Adding qemu-xen-traditional in a way supported by Xen also requires
> rombios
>    which, like qemu-xen-traditional, is disabled in Debian's official
> build of
>    Xen for Jessie.
> 
> 2. After configuring the build for qemu-xen-traditional and rombios, the
> only
>    binary package that is modified significantly is xen-utils-4.4, which is
>    where the qemu-dm binary is installed, and where a modified
> hvmloader binary
>    is installed so it includes the statically linked rombios modules.
> 
> 3. With these changes to xen-utils-4.4, together with a few simple
> configuration
>    tweaks such as the location of the qemu keymaps directory, VGA
> passthrough
>    on Debian Jessie works with HVM domains that use qemu-xen-traditional,
>    and this feature can be enabled simply by adding 2 lines to the config
>    file: device_model_version = 'qemu-xen-traditional'
>          gfx_passthru = '1'
>    I think the second option is only needed if you want the passed through
>    video card to be the primary video adapter, but I have never tried VGA
>    passthrough as a secondary adapter in the HVM. Your mileage may vary.
> 
> 4. Also, with these changes to xen-utils, the qemu-xen upstream device
model
>    and the seabios bios remain the defaults, so these changes will not do
>    violence to existing installations that use upstream qemu-xen and
> seabios.
> 
> 5. It is not possible to build the Xen source package with
> qemu-xen-traditional
>    in such a way that it meets all the requirements for inclusion in
> the main
>    area of Debian's archive of packages. Specifically, the build must
> access
>    remote repositories to download qemu-xen-traditional source files from
>    xen.org and ipxe source files from ipxe.org. Later I found it is
> possible to
>    build and configure a working package that supports VGA passthrough
> without
>    ipxe by making a couple of small configuration tweaks that remove
> etherboot
>    and ipxe support from the rombios modules that are statically linked
> into
>    hvmloader. But the requirement that the build not access the
> internet for
>    source files cannot be met unless the xen source package is modified
>    by adding another original source tar archive, something like the
>    xen_4.1.6.lts1.orig-qemu.tar.xz source file for the xen package for
> oldstable
>    Wheezy that does include support for qemu-xen-traditional and VGA
>    passthrough, in order for the build to succeed without needing to access
>    a remote repository.
> 
> With these discoveries, I would like to make the observation that
> although it is
> not trivial, it seems like it would not be too dificult for the Debian
> Xen team
> to add support for qemu-xen-traditional and rombios to the Xen source
> package so
> the resulting binary packages would support VGA passthrough when an HVM
> domain
> is configured to use qemu-xen-traditional.
> 
> After having said all this, the stage is set for my question: Why
hasn't the
> Debian Xen team added support for VGA passthrough in Debian Jessie?
> 
> From Archived Bug #688311, Bastian Blank asserted that qemu-dm will not
> come
> back and that you have to use the normal (now called upstream) qemu.
> 
> Ian Campbell responded: "Just FYI position of upstream is that the
> 'traditional'
> qemu-dm will continue to be maintained for the foreseeable future for the
> benefit of users who are running guest OSes which are not as happy to
> have the
> "motherboard" changed from under them as Linux is."
> 
> That was over four years ago, and Debian still has not restored qemu-dm so
> users of a stable Debian system can have a working VGA Passthrough for any
> guest OS, not even Linux, much less users of those Oses that don't like
the
> motherboard changed from other them.
> 
> As far as I can tell, Xen still maintains the traditional qemu-dm, and I
was
> able to recently rebuild the xen package for Jessie and get VGA Passthrough
> working on Jessie with the most recent version of the traditional device
> model
> that is available from Xen for the stable 4.4 release of Xen.
> 
> So I ask again, why can't the Debian Xen team restore qemu-dm to its
> official
> Xen package for Jessie?
> 
> The only reasonable reasons I can think of is that there is some free
> software
> licensing issue with the rombios modules that are statically linked to
> hvmloader
> or with some necessary component of qemu-dm, or the Debian Xen team has
> too few
> resources and is devoting its efforts to developing Xen for stretch
> rather than
> adding feaatures that did not make it into Jessie when it was released.
> But why
> should oldstable Wheezy have a feature that stable Jessie does not have?
> 
> In any case, I hope the Debian Xen team can explain why qemu-dm cannot be
> restored to Debian Jessie's offical xen package.
> 
> Thank you for your consideration of my question.
> 
> Sincerely,
> 
> Chuck Zmudzinski

Here are my hardware specs:

Motherboard: ASRock B85M Pro4 LGA 1150 Intel B85 HDMI SATA 6Gb/s USB 3.0 
Micro
ATX Intel (About 2 and a half years old)

CPU: Intel core i5-4590S (Haswell 4th Generation)

Chipset: Intel B85

Intel Integrated Graphics: HD4600, No external video card, I pass 
through the
Intel integrated graphics card to my DomUs

Sound: Realtek integrated on the motherboard, this can also be passed 
through
to DomUs using PCI passthrough, as well as the USB on the motherboard,
although I could not get USB 3 ports to work with passthrough, the USB 2
ports work fine though in DomUs

I also have 16 GB RAM and a 240 GB SSD, with a 1 TB HD. I bought the system
specifically because it had hardware specs that were known to support VGA
passthrough at that time. I don't know what the best options are now if you
are looking for hardware that supports VGA passthrough.

About Debian stretch:

As I said I have not tried it. But I think it is more likely to support
VGA passthrough out of the box than Jessie because the version of Xen on
stretch is 4.8, and version 4.8 supports VGA passthrough using either
the newer upstream qemu or the traditional qemu, but the version
of Xen on jessie is 4.4, and in that version VGA passthrough is only
supported using the older traditional version of qemu which is included in
Wheezy but is not available out of the box on jessie. Compare the man
page of xl.cfg on jessie with the man page of xl.cfg on stretch. Look at
what each says about supporting gfx passthrough:

On jessie: gfx_passthru is currently only supported with the qemu-xen-
traditional device-model. Upstream qemu-xen device-model currently does
not have support for gfx_passthru.

On stretch: gfx_passthru is currently supported both with the
qemu-xen-traditional device-model and upstream qemu-xen device-model.

The reason gfx passthrough does not work on jessie is that Debian took
the traditional qemu device model out of its xen package, but that
component is required for that version of Xen to function with VGA
passthrough. Unless Debian backports version 4.7 or 4.8 to jessie
so the upstream qemu device model will work with VGA passthrough,
or unless Debian provides a supported way to install the traditional qemu
device model on jessie, I don't think there will ever be a supported
configuration on jessie that supports VGA passthrough. I got VGA
passthrough working on my jessie system by hacking the xen source package
for jessie, but I don't think it is possible to get VGA passthrough 
working with
the current version of Debian's xen package for jessie, no matter what
hardware you have. You should try it on stretch and wheezy to test your
hardware for VGA passthrough functionality. Wheezy also has a better
chance of working and it also works on my system with wheezy, but it is a
little flaky and I had to hold back upgrades of the hypervisor for it to
continue to work on wheezy. But wheezy has the traditional qemu device
model, but jessie doesn't. For these reasons, you are better off trying 
wheezy
or stretch for VGA passthrough until Debian provides a solution for jessie.

Chuck

On 02/12/2017 01:37 PM, Juergen Schinker wrote:
> can you add your hw specs and also what about Debian-Stretch?
> J
>
> ----- On 12 Feb, 2017, at 18:05, Chuck Zmudzinski brchuckz at netscape.net
wrote:
>
>> This bug, at its core, is that currently there is no supported solution
>> for VGA
>> passthrough on Xen for stable version Jessie from Debian.
>>
>> After browsing Xen's repositories, I found out that Xen did not
claim to
>> support
>> VGA passthrough with the upstream qemu-xen device model until Sep 25,
>> 2015, the
>> date the xl.cfg man page was updated to indicate support for VGA
passthrough
>> with upstream qemu-xen. This change to the xl.cfg man page was only
made
>> on the
>> Xen version 4.7 and 4.8 branches, so if you want to use VGA passthrough
>> without
>> the traditional qemu-dm binary, you must upgrade to at least Xen
version
>> 4.7.
>> Debian testing (currently stretch) uses Xen 4.8 and it presumably
>> supports VGA
>> passthrough without qemu-xen-traditional but I have not tried it.
>>
>> This situation leaves users of Debian stable (currently Jessie) with no
>> supported solution from Debian for VGA passthrough on Xen. Obviously
>> there are
>> two solutions. Backport Xen 4.7 or greater to Jessie, or restore the
>> traditional qemu-dm binary to the Xen 4.4.x package for Jessie.
>>
>> A couple of months ago I decided to try and rebuild the Xen source
package
>> for Jessie with support for qemu-xen-traditional from upstream
included.
>> It did
>> not take long to get a working package that solves this bug. I
>> discovered the
>> following facts:
>>
>> 1. Adding qemu-xen-traditional in a way supported by Xen also requires
>> rombios
>>     which, like qemu-xen-traditional, is disabled in Debian's
official
>> build of
>>     Xen for Jessie.
>>
>> 2. After configuring the build for qemu-xen-traditional and rombios,
the
>> only
>>     binary package that is modified significantly is xen-utils-4.4,
which is
>>     where the qemu-dm binary is installed, and where a modified
>> hvmloader binary
>>     is installed so it includes the statically linked rombios modules.
>>
>> 3. With these changes to xen-utils-4.4, together with a few simple
>> configuration
>>     tweaks such as the location of the qemu keymaps directory, VGA
>> passthrough
>>     on Debian Jessie works with HVM domains that use
qemu-xen-traditional,
>>     and this feature can be enabled simply by adding 2 lines to the
config
>>     file: device_model_version = 'qemu-xen-traditional'
>>           gfx_passthru = '1'
>>     I think the second option is only needed if you want the passed
through
>>     video card to be the primary video adapter, but I have never tried
VGA
>>     passthrough as a secondary adapter in the HVM. Your mileage may
vary.
>>
>> 4. Also, with these changes to xen-utils, the qemu-xen upstream device
model
>>     and the seabios bios remain the defaults, so these changes will not
do
>>     violence to existing installations that use upstream qemu-xen and
>> seabios.
>>
>> 5. It is not possible to build the Xen source package with
>> qemu-xen-traditional
>>     in such a way that it meets all the requirements for inclusion in
>> the main
>>     area of Debian's archive of packages. Specifically, the build
must
>> access
>>     remote repositories to download qemu-xen-traditional source files
from
>>     xen.org and ipxe source files from ipxe.org. Later I found it is
>> possible to
>>     build and configure a working package that supports VGA passthrough
>> without
>>     ipxe by making a couple of small configuration tweaks that remove
>> etherboot
>>     and ipxe support from the rombios modules that are statically
linked
>> into
>>     hvmloader. But the requirement that the build not access the
>> internet for
>>     source files cannot be met unless the xen source package is
modified
>>     by adding another original source tar archive, something like the
>>     xen_4.1.6.lts1.orig-qemu.tar.xz source file for the xen package for
>> oldstable
>>     Wheezy that does include support for qemu-xen-traditional and VGA
>>     passthrough, in order for the build to succeed without needing to
access
>>     a remote repository.
>>
>> With these discoveries, I would like to make the observation that
>> although it is
>> not trivial, it seems like it would not be too dificult for the Debian
>> Xen team
>> to add support for qemu-xen-traditional and rombios to the Xen source
>> package so
>> the resulting binary packages would support VGA passthrough when an HVM
>> domain
>> is configured to use qemu-xen-traditional.
>>
>> After having said all this, the stage is set for my question: Why
hasn't the
>> Debian Xen team added support for VGA passthrough in Debian Jessie?
>>
>>  From Archived Bug #688311, Bastian Blank asserted that qemu-dm will
not
>> come
>> back and that you have to use the normal (now called upstream) qemu.
>>
>> Ian Campbell responded: "Just FYI position of upstream is that the
>> 'traditional'
>> qemu-dm will continue to be maintained for the foreseeable future for
the
>> benefit of users who are running guest OSes which are not as happy to
>> have the
>> "motherboard" changed from under them as Linux is."
>>
>> That was over four years ago, and Debian still has not restored qemu-dm
so
>> users of a stable Debian system can have a working VGA Passthrough for
any
>> guest OS, not even Linux, much less users of those Oses that don't
like the
>> motherboard changed from other them.
>>
>> As far as I can tell, Xen still maintains the traditional qemu-dm, and
I was
>> able to recently rebuild the xen package for Jessie and get VGA
Passthrough
>> working on Jessie with the most recent version of the traditional
device
>> model
>> that is available from Xen for the stable 4.4 release of Xen.
>>
>> So I ask again, why can't the Debian Xen team restore qemu-dm to
its
>> official
>> Xen package for Jessie?
>>
>> The only reasonable reasons I can think of is that there is some free
>> software
>> licensing issue with the rombios modules that are statically linked to
>> hvmloader
>> or with some necessary component of qemu-dm, or the Debian Xen team has
>> too few
>> resources and is devoting its efforts to developing Xen for stretch
>> rather than
>> adding feaatures that did not make it into Jessie when it was released.
>> But why
>> should oldstable Wheezy have a feature that stable Jessie does not
have?
>>
>> In any case, I hope the Debian Xen team can explain why qemu-dm cannot
be
>> restored to Debian Jessie's offical xen package.
>>
>> Thank you for your consideration of my question.
>>
>> Sincerely,
>>
>> Chuck Zmudzinski