Linux Virtualization - Jun 2013 - [PATCH RFC] virtio-pci: new config layout: using memory BAR

"H. Peter Anvin" <hpa at zytor.com> writes:
> On 06/05/2013 02:50 PM, Anthony Liguori wrote:
>> "H. Peter Anvin" <hpa at zytor.com> writes:
>> 
>>> On 06/05/2013 09:20 AM, Michael S. Tsirkin wrote:
>>>>
>>>> Spec says IO and memory can be enabled/disabled, separately.
>>>> PCI Express spec says devices should work without IO.
>>>>
>>>
>>> For "native endpoints".  Currently virtio would be a
"legacy endpoint"
>>> which is quite correct -- it is compatible with a legacy interface.
>> 
>> Do legacy endpoints also use 4k for BARs?
>
> There are no 4K BARs.  In fact, I/O BARs are restricted by spec (there
> is no technical enforcement, however) to 256 bytes.
>
> The 4K come from the upstream bridge windows, which are only 4K granular
> (historic stuff from when bridges were assumed rare.)  However, there
> can be multiple devices, functions, and BARs inside that window.
Got it.
>
> The issue with PCIe is that each PCIe port is a bridge, so in reality
> there is only one real device per bus number.
>
>> If not, can't we use a new device id for native endpoints and call
it a
>> day?  Legacy endpoints would continue using the existing BAR layout.
>
> Definitely an option.  However, we want to be able to boot from native
> devices, too, so having an I/O BAR (which would not be used by the OS
> driver) should still at the very least be an option.
What makes it so difficult to work with an MMIO bar for PCI-e?

With legacy PCI, tracking allocation of MMIO vs. PIO is pretty straight
forward.  Is there something special about PCI-e here?

Regards,

Anthony Liguori
>
> 	-hpa
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 06/05/2013 03:08 PM, Anthony Liguori wrote:
>>
>> Definitely an option.  However, we want to be able to boot from native
>> devices, too, so having an I/O BAR (which would not be used by the OS
>> driver) should still at the very least be an option.
> 
> What makes it so difficult to work with an MMIO bar for PCI-e?
> 
> With legacy PCI, tracking allocation of MMIO vs. PIO is pretty straight
> forward.  Is there something special about PCI-e here?
> 
It's not tracking allocation.  It is that accessing memory above 1 MiB
is incredibly painful in the BIOS environment, which basically means
MMIO is inaccessible.

	-hpa

"H. Peter Anvin" <hpa at zytor.com> writes:
> On 06/05/2013 03:08 PM, Anthony Liguori wrote:
>>>
>>> Definitely an option.  However, we want to be able to boot from
native
>>> devices, too, so having an I/O BAR (which would not be used by the
OS
>>> driver) should still at the very least be an option.
>> 
>> What makes it so difficult to work with an MMIO bar for PCI-e?
>> 
>> With legacy PCI, tracking allocation of MMIO vs. PIO is pretty straight
>> forward.  Is there something special about PCI-e here?
>> 
>
> It's not tracking allocation.  It is that accessing memory above 1 MiB
> is incredibly painful in the BIOS environment, which basically means
> MMIO is inaccessible.
Oh, you mean in real mode.

SeaBIOS runs the virtio code in 32-bit mode with a flat memory layout.
There are loads of ASSERT32FLAT()s in the code to make sure of this.

Regards,

Anthony Liguori
>
> 	-hpa
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html