Xen devel - Jan 2008 - Source of guest-physical address in PCI BAR for HVM domain?

I''m seeing a seemingly bad guest-physical address get used for a PCI
BAR, so I''m trying to understand who is coming up with this address.
The domain is HVM (Windows 2003).  I''m debugging the qemu-dm process
for the domain and I see where the bad value gets written to the
guest''s (virtual) PCI config space for the PCI device in question.
The call stack at that point shows:
  main
  main_loop
  main_loop_wait
  cpu_handle_ioreq
  etc.
So I guess this the PCI config-space write to set the BAR is passed to
qemu-dm as a command from the hypervisor or some other process?

I understand that eventually it would be the HVM guest OS that would
write to the PCI configuration IO port which would get caught be Xen
and passed along to qemu.  But, this is happending early in the boot
process before the guest OS proper is even running.  My understanding
of how PCI systems work is that the BIOS first configures (a subset
of) the PCI devices, and then the once the real OS is initializing it
can re-configure any PCI devices it wants to.  Can someone tell me if
this is correct?

If so, shouldn''t the early PCI configuration from the BIOS be coming
from qemu itself?  My understanding is that qemu emulates a BIOS for
HVM domains.

Thanks,
Dave

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

David Stone, le Fri 04 Jan 2008 11:35:04 -0500, a écrit
:
>   main_loop_wait
>   cpu_handle_ioreq
>   etc.
> So I guess this the PCI config-space write to set the BAR is passed to
> qemu-dm as a command from the hypervisor
Yes.
> this is happending early in the boot process before the guest OS
> proper is even running.  My understanding of how PCI systems work is
> that the BIOS first configures (a subset of) the PCI devices, and
> then the once the real OS is initializing it can re-configure any PCI
> devices it wants to.  Can someone tell me if this is correct?
That''s what I remember too.
> If so, shouldn''t the early PCI configuration from the BIOS be
coming
> from qemu itself?
It is. It''s odd you have troubles so early in the boot, is your
configuration a bit special in any way?

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks for your response.
> > So I guess this the PCI config-space write to set the BAR is passed to
> > qemu-dm as a command from the hypervisor
>
> Yes.
But doesn''t qemu act as the BIOS for HVM domains?  Why would the
hypervisor be doing the initial PCI BAR setup?
> > If so, shouldn''t the early PCI configuration from the BIOS be
coming
> > from qemu itself?
>
> It is. It''s odd you have troubles so early in the boot, is your
> configuration a bit special in any way?
Yes, I didn''t mention the most important part: the device in question
is a physical PCI device (a PCI Express graphics card) that I am
passing through to the Windows 2003 guest domain via VT-d.  (The VT-d
support generally works for me because I can pass a PCI NIC through no
problem.)  (I realize VT-d''ing a PCI-XP graphics card is
experimental...but that''s what I''m doing, experimenting...).

Thanks,
Dave

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

David Stone, le Fri 04 Jan 2008 13:25:47 -0500, a écrit
:
> > > So I guess this the PCI config-space write to set the BAR is
passed to
> > > qemu-dm as a command from the hypervisor
> >
> > Yes.
> 
> But doesn''t qemu act as the BIOS for HVM domains?
Yes.
> Why would the hypervisor be doing the initial PCI BAR setup?
The hypervisor does nothing but retransmit what the HVM domain performs.
Remember that instructions of the qemu BIOS are run in the HVM domain,
not in qemu, which only gets triggered when the BIOS actually I/O ports
or memory.
> > > If so, shouldn''t the early PCI configuration from the
BIOS be coming
> > > from qemu itself?
> >
> > It is. It''s odd you have troubles so early in the boot, is
your
> > configuration a bit special in any way?
> 
> Yes, I didn''t mention the most important part: the device in
question
> is a physical PCI device (a PCI Express graphics card) that I am
> passing through to the Windows 2003 guest domain via VT-d.  (The VT-d
> support generally works for me because I can pass a PCI NIC through no
> problem.)  (I realize VT-d''ing a PCI-XP graphics card is
> experimental...but that''s what I''m doing,
experimenting...).
Then maybe the qemu BIOS has troubles with that device?

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > Why would the hypervisor be doing the initial PCI BAR setup?
>
> The hypervisor does nothing but retransmit what the HVM domain performs.
> Remember that instructions of the qemu BIOS are run in the HVM domain,
> not in qemu, which only gets triggered when the BIOS actually I/O ports
> or memory.
OK now I understand, thanks.
> > Yes, I didn''t mention the most important part: the device in
question
> > is a physical PCI device (a PCI Express graphics card) that I am
> > passing through to the Windows 2003 guest domain via VT-d.  (The VT-d
> > support generally works for me because I can pass a PCI NIC through no
> > problem.)  (I realize VT-d''ing a PCI-XP graphics card is
> > experimental...but that''s what I''m doing,
experimenting...).
>
> Then maybe the qemu BIOS has troubles with that device?
Maybe, I guess the next step is to start troubleshooting/debugging the
qemu BIOS to see what it is doing.  BTW, the specific thing that the
qemu BIOS is doing that I think is wrong and may be causing a
subsequent machine check (?) is that it is assigning a guest-physical
address of 0x10000000 to one of the device''s BARs.  The problem with
this is that I am assigning 1GB of memory to this domain, which means
that the addresses 0x00000000-0x40000000 should be assigned to the
domain''s system memory...and indeed that is what the domain''s
MTRRs
seem to show.  I know that some low physical addresses correspond to
system resources but doesn''t assigning to a device''s BAR an
address
that is right in the middle of system memory seem wrong?  I don''t see
this ever happen for any other device.

One more question: I actually tried using a PCI graphics card to pass
through via VT-d instead of a PCI-XP graphics card and it
worked...sort of.  The domain doesn''t cause a machine check (which is
always nice).  When I look at Device Manager in Windows I see both the
emulated Cirrus card and the passed-through physical PCI graphics card
as hoped for.  However, the driver for the Cirrus card isn''t
loaded...it says there is a resource conflict.  I haven''t been able to
figure out what resource is being contended for.  But anyway, from the
guest Windows OS perspective card, the Cirrus card is _not_ in use,
but the physical card is.  What surprises me is that I still can
interact with the guest OS through a loopback VNC session to qemu-dm
as normal.  I thought qemu exposed the framebuffer by acting as the
Cirrus card...but in this case it seems to be working even when the
physical card is being used?

Thanks,
Dave

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I see there''s been a small email thread on this topic now, but I want
to
point out that the code that configures the PCI BARs initially is not in the
rombios code but is in the hvmloader code. See
tools/formware/hvmloader/hvmloader.c:pci_setup().

 -- Keir

On 4/1/08 16:35, "David Stone" <unclestoner@gmail.com> wrote:
> I understand that eventually it would be the HVM guest OS that would
> write to the PCI configuration IO port which would get caught be Xen
> and passed along to qemu.  But, this is happending early in the boot
> process before the guest OS proper is even running.  My understanding
> of how PCI systems work is that the BIOS first configures (a subset
> of) the PCI devices, and then the once the real OS is initializing it
> can re-configure any PCI devices it wants to.  Can someone tell me if
> this is correct?
> 
> If so, shouldn''t the early PCI configuration from the BIOS be
coming
> from qemu itself?  My understanding is that qemu emulates a BIOS for
> HVM domains.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks for the pointer...I believe I see the problem.  The code in
pci_setup starts granting guest-physical address space to PCI devices
at 0xF0000000.  The problem in my case is, my PCI-XP graphics card
wants 256MB of guest-phyical address space for its video RAM.  That
wraps around to 0x00000000.  It doesn''t look like the code handles
this, so it just merrily wraps around and ends up assigning memory
ranges that are already in use for system RAM!

I''ll have a look at improving this...

Dave

On Jan 6, 2008 6:15 PM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
wrote:
> I see there''s been a small email thread on this topic now, but I
want to
> point out that the code that configures the PCI BARs initially is not in
the
> rombios code but is in the hvmloader code. See
> tools/formware/hvmloader/hvmloader.c:pci_setup().
>
>  -- Keir
>
>
> On 4/1/08 16:35, "David Stone" <unclestoner@gmail.com>
wrote:
>
> > I understand that eventually it would be the HVM guest OS that would
> > write to the PCI configuration IO port which would get caught be Xen
> > and passed along to qemu.  But, this is happending early in the boot
> > process before the guest OS proper is even running.  My understanding
> > of how PCI systems work is that the BIOS first configures (a subset
> > of) the PCI devices, and then the once the real OS is initializing it
> > can re-configure any PCI devices it wants to.  Can someone tell me if
> > this is correct?
> >
> > If so, shouldn''t the early PCI configuration from the BIOS be
coming
> > from qemu itself?  My understanding is that qemu emulates a BIOS for
> > HVM domains.
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

hvmloader should expand the mmio hole if it detects that there is not enough
space to map all PCI resources.

 -- Keir

On 7/1/08 21:39, "David Stone" <unclestoner@gmail.com> wrote:
> Thanks for the pointer...I believe I see the problem.  The code in
> pci_setup starts granting guest-physical address space to PCI devices
> at 0xF0000000.  The problem in my case is, my PCI-XP graphics card
> wants 256MB of guest-phyical address space for its video RAM.  That
> wraps around to 0x00000000.  It doesn''t look like the code handles
> this, so it just merrily wraps around and ends up assigning memory
> ranges that are already in use for system RAM!
> 
> I''ll have a look at improving this...
> 
> Dave
> 
> On Jan 6, 2008 6:15 PM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>> I see there''s been a small email thread on this topic now, but
I want to
>> point out that the code that configures the PCI BARs initially is not
in the
>> rombios code but is in the hvmloader code. See
>> tools/formware/hvmloader/hvmloader.c:pci_setup().
>> 
>>  -- Keir
>> 
>> 
>> On 4/1/08 16:35, "David Stone" <unclestoner@gmail.com>
wrote:
>> 
>>> I understand that eventually it would be the HVM guest OS that
would
>>> write to the PCI configuration IO port which would get caught be
Xen
>>> and passed along to qemu.  But, this is happending early in the
boot
>>> process before the guest OS proper is even running.  My
understanding
>>> of how PCI systems work is that the BIOS first configures (a subset
>>> of) the PCI devices, and then the once the real OS is initializing
it
>>> can re-configure any PCI devices it wants to.  Can someone tell me
if
>>> this is correct?
>>> 
>>> If so, shouldn''t the early PCI configuration from the BIOS
be coming
>>> from qemu itself?  My understanding is that qemu emulates a BIOS
for
>>> HVM domains.
>> 
>> 
>> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

You can change that starting address to from 0xF000000 to something more
sensible for large BAR devices (like 0xC0000000) in
tools/firmware/acpi/dsdt.asl at line 123, and the change 0x05000000 to
0x3500000 on 126 (that''s a size), and recompiling the asl file.

You need large regions for large bars as bars are aligned at their size
(that 256MB BAR will be aligned on a 256MB boundary)

Now of course you''ve limited memory < 4GB to 3GB, so YMMV.  I
haven''t
experiment as to whether the Xen BIOS recovers the PCI "hole" memory
above 4GB.

Also, beware that hvmloader doesn''t allocate resources behind bridges.
(It doesn''t even configure bridge bars correctly.) 

It is interesting to note that Linux behaves differently that Windows in
this regard.  Windows pays attention to the ASL limits and will not
exceed them even if less RAM is allocated to the domain.  Linux (at
least as of my last test of this) will reallocate BARs anywhere
unclaimed.

John Zulauf

*** Opinions expressed are only those of the author and do not
necessarily reflect the views of his employer.

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of David Stone
Sent: Monday, January 07, 2008 1:39 PM
To: Keir Fraser
Cc: xen-devel@lists.xensource.com; Philip Kufeldt; Samuel Thibault
Subject: Re: [Xen-devel] Source of guest-physical address in PCI BAR for
HVMdomain?

Thanks for the pointer...I believe I see the problem.  The code in
pci_setup starts granting guest-physical address space to PCI devices
at 0xF0000000.  The problem in my case is, my PCI-XP graphics card
wants 256MB of guest-phyical address space for its video RAM.  That
wraps around to 0x00000000.  It doesn''t look like the code handles
this, so it just merrily wraps around and ends up assigning memory
ranges that are already in use for system RAM!

I''ll have a look at improving this...

Dave

On Jan 6, 2008 6:15 PM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
wrote:
> I see there''s been a small email thread on this topic now, but I
want
to
> point out that the code that configures the PCI BARs initially is not
in the
> rombios code but is in the hvmloader code. See
> tools/formware/hvmloader/hvmloader.c:pci_setup().
>
>  -- Keir
>
>
> On 4/1/08 16:35, "David Stone" <unclestoner@gmail.com>
wrote:
>
> > I understand that eventually it would be the HVM guest OS that would
> > write to the PCI configuration IO port which would get caught be Xen
> > and passed along to qemu.  But, this is happending early in the boot
> > process before the guest OS proper is even running.  My
understanding
> > of how PCI systems work is that the BIOS first configures (a subset
> > of) the PCI devices, and then the once the real OS is initializing
it
> > can re-configure any PCI devices it wants to.  Can someone tell me
if
> > this is correct?
> >
> > If so, shouldn''t the early PCI configuration from the BIOS be
coming
> > from qemu itself?  My understanding is that qemu emulates a BIOS for
> > HVM domains.
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> It is interesting to note that Linux behaves differently that Windows in
> this regard.  Windows pays attention to the ASL limits and will not
> exceed them even if less RAM is allocated to the domain.  Linux (at
> least as of my last test of this) will reallocate BARs anywhere
> unclaimed.
That may be something that should be looked at on the Linux side rather
than the Xen side ?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ah, I forgot the io hole is also coded into the DSDT. That''s a pain.
Either
that range specifier needs to be dynamically generated by hvmloader into a
SSDT, or we need two statically generated SSDTs to hand (0xc0000000- and
0xf0000000-) that hvmloader picks between dependent on size of PCI resources
to be mapped, or we just make all guests have a 1GB hole.

As for hvmloader not configuring bridges, without pci passthru there are
never any bridges to be configured. It''s a job for whoever is working
on
passthru of devices behind bridges to fix that.

 -- Keir

On 7/1/08 23:46, "Zulauf, John" <john.zulauf@intel.com> wrote:
> You can change that starting address to from 0xF000000 to something more
> sensible for large BAR devices (like 0xC0000000) in
> tools/firmware/acpi/dsdt.asl at line 123, and the change 0x05000000 to
> 0x3500000 on 126 (that''s a size), and recompiling the asl file.
> 
> You need large regions for large bars as bars are aligned at their size
> (that 256MB BAR will be aligned on a 256MB boundary)
> 
> Now of course you''ve limited memory < 4GB to 3GB, so YMMV.  I
haven''t
> experiment as to whether the Xen BIOS recovers the PCI "hole"
memory
> above 4GB.
> 
> Also, beware that hvmloader doesn''t allocate resources behind
bridges.
> (It doesn''t even configure bridge bars correctly.)
> 
> It is interesting to note that Linux behaves differently that Windows in
> this regard.  Windows pays attention to the ASL limits and will not
> exceed them even if less RAM is allocated to the domain.  Linux (at
> least as of my last test of this) will reallocate BARs anywhere
> unclaimed.
> 
> John Zulauf
> 
> *** Opinions expressed are only those of the author and do not
> necessarily reflect the views of his employer.
> 
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of David Stone
> Sent: Monday, January 07, 2008 1:39 PM
> To: Keir Fraser
> Cc: xen-devel@lists.xensource.com; Philip Kufeldt; Samuel Thibault
> Subject: Re: [Xen-devel] Source of guest-physical address in PCI BAR for
> HVMdomain?
> 
> Thanks for the pointer...I believe I see the problem.  The code in
> pci_setup starts granting guest-physical address space to PCI devices
> at 0xF0000000.  The problem in my case is, my PCI-XP graphics card
> wants 256MB of guest-phyical address space for its video RAM.  That
> wraps around to 0x00000000.  It doesn''t look like the code handles
> this, so it just merrily wraps around and ends up assigning memory
> ranges that are already in use for system RAM!
> 
> I''ll have a look at improving this...
> 
> Dave
> 
> On Jan 6, 2008 6:15 PM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>> I see there''s been a small email thread on this topic now, but
I want
> to
>> point out that the code that configures the PCI BARs initially is not
> in the
>> rombios code but is in the hvmloader code. See
>> tools/formware/hvmloader/hvmloader.c:pci_setup().
>> 
>>  -- Keir
>> 
>> 
>> On 4/1/08 16:35, "David Stone" <unclestoner@gmail.com>
wrote:
>> 
>>> I understand that eventually it would be the HVM guest OS that
would
>>> write to the PCI configuration IO port which would get caught be
Xen
>>> and passed along to qemu.  But, this is happending early in the
boot
>>> process before the guest OS proper is even running.  My
> understanding
>>> of how PCI systems work is that the BIOS first configures (a subset
>>> of) the PCI devices, and then the once the real OS is initializing
> it
>>> can re-configure any PCI devices it wants to.  Can someone tell me
> if
>>> this is correct?
>>> 
>>> If so, shouldn''t the early PCI configuration from the BIOS
be coming
>>> from qemu itself?  My understanding is that qemu emulates a BIOS
for
>>> HVM domains.
>> 
>> 
>> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

So not only must we not wrap around to 0x00000000, we must also not
assign memory ranges to PCI devices that are used by other components
like for example the IOAPIC (0xFEC00000)?  I don''t know anything about
ACPI but in dstd.asl it looks like up to 0xF4FFFFFF is being reserved
for PCI devices...so is 0xF4FFFFFF the upper limit that should be
used?

Also, can someone explain how this all works on 64-bit?

Thanks,
Dave


On Jan 8, 2008 3:19 AM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
wrote:
> Ah, I forgot the io hole is also coded into the DSDT. That''s a
pain. Either
> that range specifier needs to be dynamically generated by hvmloader into a
> SSDT, or we need two statically generated SSDTs to hand (0xc0000000- and
> 0xf0000000-) that hvmloader picks between dependent on size of PCI
resources
> to be mapped, or we just make all guests have a 1GB hole.
>
> As for hvmloader not configuring bridges, without pci passthru there are
> never any bridges to be configured. It''s a job for whoever is
working on
> passthru of devices behind bridges to fix that.
>
>  -- Keir
>
>
> On 7/1/08 23:46, "Zulauf, John" <john.zulauf@intel.com>
wrote:
>
> > You can change that starting address to from 0xF000000 to something
more
> > sensible for large BAR devices (like 0xC0000000) in
> > tools/firmware/acpi/dsdt.asl at line 123, and the change 0x05000000 to
> > 0x3500000 on 126 (that''s a size), and recompiling the asl
file.
> >
> > You need large regions for large bars as bars are aligned at their
size
> > (that 256MB BAR will be aligned on a 256MB boundary)
> >
> > Now of course you''ve limited memory < 4GB to 3GB, so YMMV.
I haven''t
> > experiment as to whether the Xen BIOS recovers the PCI
"hole" memory
> > above 4GB.
> >
> > Also, beware that hvmloader doesn''t allocate resources behind
bridges.
> > (It doesn''t even configure bridge bars correctly.)
> >
> > It is interesting to note that Linux behaves differently that Windows
in
> > this regard.  Windows pays attention to the ASL limits and will not
> > exceed them even if less RAM is allocated to the domain.  Linux (at
> > least as of my last test of this) will reallocate BARs anywhere
> > unclaimed.
> >
> > John Zulauf
> >
> > *** Opinions expressed are only those of the author and do not
> > necessarily reflect the views of his employer.
> >
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com
> > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of David
Stone
> > Sent: Monday, January 07, 2008 1:39 PM
> > To: Keir Fraser
> > Cc: xen-devel@lists.xensource.com; Philip Kufeldt; Samuel Thibault
> > Subject: Re: [Xen-devel] Source of guest-physical address in PCI BAR
for
> > HVMdomain?
> >
> > Thanks for the pointer...I believe I see the problem.  The code in
> > pci_setup starts granting guest-physical address space to PCI devices
> > at 0xF0000000.  The problem in my case is, my PCI-XP graphics card
> > wants 256MB of guest-phyical address space for its video RAM.  That
> > wraps around to 0x00000000.  It doesn''t look like the code
handles
> > this, so it just merrily wraps around and ends up assigning memory
> > ranges that are already in use for system RAM!
> >
> > I''ll have a look at improving this...
> >
> > Dave
> >
> > On Jan 6, 2008 6:15 PM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk>
wrote:
> >> I see there''s been a small email thread on this topic
now, but I want
> > to
> >> point out that the code that configures the PCI BARs initially is
not
> > in the
> >> rombios code but is in the hvmloader code. See
> >> tools/formware/hvmloader/hvmloader.c:pci_setup().
> >>
> >>  -- Keir
> >>
> >>
> >> On 4/1/08 16:35, "David Stone"
<unclestoner@gmail.com> wrote:
> >>
> >>> I understand that eventually it would be the HVM guest OS that
would
> >>> write to the PCI configuration IO port which would get caught
be Xen
> >>> and passed along to qemu.  But, this is happending early in
the boot
> >>> process before the guest OS proper is even running.  My
> > understanding
> >>> of how PCI systems work is that the BIOS first configures (a
subset
> >>> of) the PCI devices, and then the once the real OS is
initializing
> > it
> >>> can re-configure any PCI devices it wants to.  Can someone
tell me
> > if
> >>> this is correct?
> >>>
> >>> If so, shouldn''t the early PCI configuration from the
BIOS be coming
> >>> from qemu itself?  My understanding is that qemu emulates a
BIOS for
> >>> HVM domains.
> >>
> >>
> >>
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I was reviewing my own changes for this and you might also need to
change the value of 

HVM_BELOW_4G_RAM_END to 0xC000000 (or whatever match the ACPI values) in
xen/include/public/hvm/e820.h

As for 64 bit, as far as I can tell hvmloader allocates everything below
4G for 64 bit guests as well.

John Zulauf

*** Opinions expressed are only those of the author and do not
necessarily reflect the views of his employer.

-----Original Message-----
From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] 
Sent: Tuesday, January 08, 2008 12:19 AM
To: Zulauf, John; David Stone
Cc: xen-devel@lists.xensource.com; Philip Kufeldt; Samuel Thibault
Subject: Re: [Xen-devel] Source of guest-physical address in PCI BAR for
HVMdomain?

Ah, I forgot the io hole is also coded into the DSDT. That''s a pain.
Either
that range specifier needs to be dynamically generated by hvmloader into
a
SSDT, or we need two statically generated SSDTs to hand (0xc0000000- and
0xf0000000-) that hvmloader picks between dependent on size of PCI
resources
to be mapped, or we just make all guests have a 1GB hole.

As for hvmloader not configuring bridges, without pci passthru there are
never any bridges to be configured. It''s a job for whoever is working
on
passthru of devices behind bridges to fix that.

 -- Keir

On 7/1/08 23:46, "Zulauf, John" <john.zulauf@intel.com> wrote:
> You can change that starting address to from 0xF000000 to something
more
> sensible for large BAR devices (like 0xC0000000) in
> tools/firmware/acpi/dsdt.asl at line 123, and the change 0x05000000 to
> 0x3500000 on 126 (that''s a size), and recompiling the asl file.
> 
> You need large regions for large bars as bars are aligned at their
size
> (that 256MB BAR will be aligned on a 256MB boundary)
> 
> Now of course you''ve limited memory < 4GB to 3GB, so YMMV.  I
haven''t
> experiment as to whether the Xen BIOS recovers the PCI "hole"
memory
> above 4GB.
> 
> Also, beware that hvmloader doesn''t allocate resources behind
bridges.
> (It doesn''t even configure bridge bars correctly.)
> 
> It is interesting to note that Linux behaves differently that Windows
in
> this regard.  Windows pays attention to the ASL limits and will not
> exceed them even if less RAM is allocated to the domain.  Linux (at
> least as of my last test of this) will reallocate BARs anywhere
> unclaimed.
> 
> John Zulauf
> 
> *** Opinions expressed are only those of the author and do not
> necessarily reflect the views of his employer.
> 
> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of David
Stone
> Sent: Monday, January 07, 2008 1:39 PM
> To: Keir Fraser
> Cc: xen-devel@lists.xensource.com; Philip Kufeldt; Samuel Thibault
> Subject: Re: [Xen-devel] Source of guest-physical address in PCI BAR
for
> HVMdomain?
> 
> Thanks for the pointer...I believe I see the problem.  The code in
> pci_setup starts granting guest-physical address space to PCI devices
> at 0xF0000000.  The problem in my case is, my PCI-XP graphics card
> wants 256MB of guest-phyical address space for its video RAM.  That
> wraps around to 0x00000000.  It doesn''t look like the code handles
> this, so it just merrily wraps around and ends up assigning memory
> ranges that are already in use for system RAM!
> 
> I''ll have a look at improving this...
> 
> Dave
> 
> On Jan 6, 2008 6:15 PM, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>> I see there''s been a small email thread on this topic now, but
I want
> to
>> point out that the code that configures the PCI BARs initially is not
> in the
>> rombios code but is in the hvmloader code. See
>> tools/formware/hvmloader/hvmloader.c:pci_setup().
>> 
>>  -- Keir
>> 
>> 
>> On 4/1/08 16:35, "David Stone" <unclestoner@gmail.com>
wrote:
>> 
>>> I understand that eventually it would be the HVM guest OS that
would
>>> write to the PCI configuration IO port which would get caught be
Xen
>>> and passed along to qemu.  But, this is happending early in the
boot
>>> process before the guest OS proper is even running.  My
> understanding
>>> of how PCI systems work is that the BIOS first configures (a subset
>>> of) the PCI devices, and then the once the real OS is initializing
> it
>>> can re-configure any PCI devices it wants to.  Can someone tell me
> if
>>> this is correct?
>>> 
>>> If so, shouldn''t the early PCI configuration from the BIOS
be coming
>>> from qemu itself?  My understanding is that qemu emulates a BIOS
for
>>> HVM domains.
>> 
>> 
>> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel