Xen devel - Apr 2010 - Xen VGA graphics passthrough for secondary adapters, vBAR=pBAR etc

Hello,

Now that Xen 4.0.0 is out and has support for primary graphics passthru
and many people have been using it already and asking for a status update
for secondary adapter passthru I thought of opening a new thread about this.

What are the actual issues that need to be resolved to get the 
secondary graphics passthru working? Probably related to VGA BIOS execution
in the guest? 

How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
How do we get that functionality integrated so people don''t need
additional patches?

Was there also a problem that a working VGA BIOS cannot be copied from some
cards
and they need to use/load it from a file? 

Thanks!

-- Pasi

btw. I''ve created a wiki page where we can list all the tested/working
graphics adapters:
http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> Hello,
> 
> Now that Xen 4.0.0 is out and has support for primary graphics passthru
> and many people have been using it already and asking for a status update
> for secondary adapter passthru I thought of opening a new thread about
this.
> 
> What are the actual issues that need to be resolved to get the 
> secondary graphics passthru working? Probably related to VGA BIOS execution
> in the guest? 
> 
> How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> How do we get that functionality integrated so people don''t need
additional patches?
> 
> Was there also a problem that a working VGA BIOS cannot be copied from some
cards
> and they need to use/load it from a file? 
> 
> Thanks!
> 
I think is worth letting people know that the only devices supported by
Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800 and
5800 at the moment.

This doesn''t mean we have to stop trying making the other devices work
too ;)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

12.04.2010 09:21, Pasi Kärkkäinen wrote:
> Hello,
>
> Now that Xen 4.0.0 is out and has support for primary graphics passthru
> and many people have been using it already and asking for a status update
> for secondary adapter passthru I thought of opening a new thread about
this.
>
> What are the actual issues that need to be resolved to get the
> secondary graphics passthru working? Probably related to VGA BIOS execution
> in the guest?
>
> How about the vBAR=pBAR stuff required for some (Nvidia) cards?
> How do we get that functionality integrated so people don''t need
additional patches?
>
> Was there also a problem that a working VGA BIOS cannot be copied from some
cards
> and they need to use/load it from a file?
>
> Thanks!
>
> -- Pasi
>
> btw. I''ve created a wiki page where we can list all the
tested/working graphics adapters:
> http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters

I already added that suggestion to the 4.1 suggested features some days 
ago and planned to rebase the patches to 4.0 if they need any adjustments.
One major change I''d like to see is dynamic (and thus optional) 
configuration of the mem space to be mapped and the BIOS file to be used 
(instead of configuration at compile time).
Also afair the previous patch set also had a maximum size (correct me if 
I''m wrong) for the BIOS size that was to small for recent (HD4800+) ATI
devices.


Best regards,
    Christian


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Pasi Kärkkäinen wrote:
> Hello,
>
> Now that Xen 4.0.0 is out and has support for primary graphics passthru
> and many people have been using it already and asking for a status update
> for secondary adapter passthru I thought of opening a new thread about
this.
>
> What are the actual issues that need to be resolved to get the 
> secondary graphics passthru working? Probably related to VGA BIOS execution
> in the guest? 
>   
I think of two things need to be solved for secondary gfx assigned:
 1) claim VGA cycle for it to display in VGA for guest, VGA is owned by 
primary gfx.
 2) load VGA bios from file, because you cannot get it from memory like 
primary gfx.
> How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> How do we get that functionality integrated so people don''t need
additional patches?
>   
vBAR=pBAR patch is a bit tricky. It needs to cook a new clean vBAR=pBAR 
patch if integrate it in upstream.
> Was there also a problem that a working VGA BIOS cannot be copied from some
cards
> and they need to use/load it from a file? 
>   
Load it from a file should work for all cards, but it''s not user 
friendly  (involving manual steps). It''s better to implement a generic 
approach to get VGA bios for various gfx cards including primary and 
secondary gfx. The direct way is to load it from expansion rom, but it 
doesn''t always work because expansion rom address may be invalid (the 
bridge may doesn''t have enough memory for gfx card).

Regards,
Weidong
> Thanks!
>
> -- Pasi
>
> btw. I''ve created a wiki page where we can list all the
tested/working graphics adapters:
> http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters
>
>
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Christian Tramnitz wrote:
> 12.04.2010 09:21, Pasi Kärkkäinen wrote:
>   
>> Hello,
>>
>> Now that Xen 4.0.0 is out and has support for primary graphics passthru
>> and many people have been using it already and asking for a status
update
>> for secondary adapter passthru I thought of opening a new thread about
this.
>>
>> What are the actual issues that need to be resolved to get the
>> secondary graphics passthru working? Probably related to VGA BIOS
execution
>> in the guest?
>>
>> How about the vBAR=pBAR stuff required for some (Nvidia) cards?
>> How do we get that functionality integrated so people don''t
need additional patches?
>>
>> Was there also a problem that a working VGA BIOS cannot be copied from
some cards
>> and they need to use/load it from a file?
>>
>> Thanks!
>>
>> -- Pasi
>>
>> btw. I''ve created a wiki page where we can list all the
tested/working graphics adapters:
>> http://wiki.xensource.com/xenwiki/XenVGAPassthroughTestedAdapters
>>     
>
>
> I already added that suggestion to the 4.1 suggested features some days 
> ago and planned to rebase the patches to 4.0 if they need any adjustments.
> One major change I''d like to see is dynamic (and thus optional) 
> configuration of the mem space to be mapped and the BIOS file to be used 
>   
Could you explain "dynamic configuration of the mem space to be
mapped"?
do you mean MMIO memories?
> (instead of configuration at compile time).
> Also afair the previous patch set also had a maximum size (correct me if 
> I''m wrong) for the BIOS size that was to small for recent
(HD4800+) ATI
> devices.
>   
What is the size of HD4800 bios? It can adjust memory map address 
definitions in hvmloader to solve it.

Regards,
Weidong
>
> Best regards,
>     Christian
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

12.04.2010 12:08, Weidong Han wrote:
>> I already added that suggestion to the 4.1 suggested features some
>> days ago and planned to rebase the patches to 4.0 if they need any
>> adjustments.
>> One major change I''d like to see is dynamic (and thus
optional)
>> configuration of the mem space to be mapped and the BIOS file to be
used
> Could you explain "dynamic configuration of the mem space to be
mapped"?
> do you mean MMIO memories?
Yes, the MMIO BARs that currently need manual changes to dsdt.asl before 
compilation. If we could make that a runtime configuration option or 
even allow dynamic detection (via libpci?) this would help a lot. It 
would also make the changes less intrusive (and increase the chances of 
being accepted I hope).
>> Also afair the previous patch set also had a maximum size (correct me
>> if I''m wrong) for the BIOS size that was to small for recent
(HD4800+)
>> ATI devices.
> What is the size of HD4800 bios? It can adjust memory map address
> definitions in hvmloader to solve it.
It''s 128kB (current limitation is 64k), but that should be easy enough 
to fix, just wanted to mention it while we''re discussing the topic ;-)


Are you still actively working on this topic? Haven''t seen a reply to
my
previous mails to list if work is still being done. Just want to avoid, 
that we''re both working on the same thing...


Best regards,
    Christian


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Apr 12, 2010 at 10:34:14AM +0100, Stefano Stabellini
wrote:
> On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > Hello,
> > 
> > Now that Xen 4.0.0 is out and has support for primary graphics
passthru
> > and many people have been using it already and asking for a status
update
> > for secondary adapter passthru I thought of opening a new thread about
this.
> > 
> > What are the actual issues that need to be resolved to get the 
> > secondary graphics passthru working? Probably related to VGA BIOS
execution
> > in the guest? 
> > 
> > How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> > How do we get that functionality integrated so people don''t
need additional patches?
> > 
> > Was there also a problem that a working VGA BIOS cannot be copied from
some cards
> > and they need to use/load it from a file? 
> > 
> > Thanks!
> > 
> 
> I think is worth letting people know that the only devices supported by
> Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800 and
> 5800 at the moment.
> 
Yeah.. I guess you mean ''official support'' here? What does
that mean practically?
FLR support? 
> This doesn''t mean we have to stop trying making the other devices
work
> too ;)
>
01:00.0 VGA compatible controller: nVidia Corporation G92 [GeForce 8800 GTS 512]
(rev a2)

That card was reported to work.. :)

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> On Mon, Apr 12, 2010 at 10:34:14AM +0100, Stefano Stabellini wrote:
> > On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > > Hello,
> > > 
> > > Now that Xen 4.0.0 is out and has support for primary graphics
passthru
> > > and many people have been using it already and asking for a
status update
> > > for secondary adapter passthru I thought of opening a new thread
about this.
> > > 
> > > What are the actual issues that need to be resolved to get the 
> > > secondary graphics passthru working? Probably related to VGA BIOS
execution
> > > in the guest? 
> > > 
> > > How about the vBAR=pBAR stuff required for some (Nvidia) cards? 
> > > How do we get that functionality integrated so people
don''t need additional patches?
> > > 
> > > Was there also a problem that a working VGA BIOS cannot be copied
from some cards
> > > and they need to use/load it from a file? 
> > > 
> > > Thanks!
> > > 
> > 
> > I think is worth letting people know that the only devices supported
by
> > Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800 and
> > 5800 at the moment.
> > 
> 
> Yeah.. I guess you mean ''official support'' here? What
does that mean practically?
> FLR support? 
> 
It means that the retail Nvidia drivers have been tested in this use
case and probably contain some changes to make sure it works properly.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Apr 12, 2010 at 12:03:26PM +0100, Stefano Stabellini
wrote:
> On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > On Mon, Apr 12, 2010 at 10:34:14AM +0100, Stefano Stabellini wrote:
> > > On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> > > > Hello,
> > > > 
> > > > Now that Xen 4.0.0 is out and has support for primary
graphics passthru
> > > > and many people have been using it already and asking for a
status update
> > > > for secondary adapter passthru I thought of opening a new
thread about this.
> > > > 
> > > > What are the actual issues that need to be resolved to get
the
> > > > secondary graphics passthru working? Probably related to VGA
BIOS execution
> > > > in the guest? 
> > > > 
> > > > How about the vBAR=pBAR stuff required for some (Nvidia)
cards?
> > > > How do we get that functionality integrated so people
don''t need additional patches?
> > > > 
> > > > Was there also a problem that a working VGA BIOS cannot be
copied from some cards
> > > > and they need to use/load it from a file? 
> > > > 
> > > > Thanks!
> > > > 
> > > 
> > > I think is worth letting people know that the only devices
supported by
> > > Nvidia in the pci passthrough scenario are Quadro FX 3800, 4800
and
> > > 5800 at the moment.
> > > 
> > 
> > Yeah.. I guess you mean ''official support'' here?
What does that mean practically?
> > FLR support? 
> > 
> 
> It means that the retail Nvidia drivers have been tested in this use
> case and probably contain some changes to make sure it works properly.
>
Ok. I''ll add a note about this to the wiki page.

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 12 Apr 2010, Pasi Kärkkäinen wrote:
> 
> Ok. I''ll add a note about this to the wiki page.
 
Thanks :)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, Apr 12, 2010 at 12:18:50PM +0200, Christian Tramnitz
wrote:
> 12.04.2010 12:08, Weidong Han wrote:
>>> I already added that suggestion to the 4.1 suggested features some
>>> days ago and planned to rebase the patches to 4.0 if they need any
>>> adjustments.
>>> One major change I''d like to see is dynamic (and thus
optional)
>>> configuration of the mem space to be mapped and the BIOS file to be
used
>> Could you explain "dynamic configuration of the mem space to be
mapped"?
>> do you mean MMIO memories?
> Yes, the MMIO BARs that currently need manual changes to dsdt.asl before  
> compilation. If we could make that a runtime configuration option or  
> even allow dynamic detection (via libpci?) this would help a lot. It  
> would also make the changes less intrusive (and increase the chances of  
> being accepted I hope).
>
>>> Also afair the previous patch set also had a maximum size (correct
me
>>> if I''m wrong) for the BIOS size that was to small for
recent (HD4800+)
>>> ATI devices.
>> What is the size of HD4800 bios? It can adjust memory map address
>> definitions in hvmloader to solve it.
> It''s 128kB (current limitation is 64k), but that should be easy
enough
> to fix, just wanted to mention it while we''re discussing the topic
;-)
Oh, that might also fix the QLogic BIOS issue that some folks
reported.
>
>
> Are you still actively working on this topic? Haven''t seen a reply
to my
> previous mails to list if work is still being done. Just want to avoid,  
> that we''re both working on the same thing...
I wasn''t sure on the status of this and wrote up a task item for this
exact work on the Google Summer Code Code:

http://wiki.xensource.com/xenwiki/XAPI_project_suggestions

It is great to see that there is interest in this. Christian, since you
are taking a lead in this, would you like me to nuke the entry from the
GSOC and start hacking at this yourself? Or would you like to be a mentor
for this project?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel