Anthony Liguori
2013-Jun-06 00:41 UTC
[PATCH RFC] virtio-pci: new config layout: using memory BAR
"H. Peter Anvin" <hpa at zytor.com> writes:> On 06/05/2013 03:08 PM, Anthony Liguori wrote: >>> >>> Definitely an option. However, we want to be able to boot from native >>> devices, too, so having an I/O BAR (which would not be used by the OS >>> driver) should still at the very least be an option. >> >> What makes it so difficult to work with an MMIO bar for PCI-e? >> >> With legacy PCI, tracking allocation of MMIO vs. PIO is pretty straight >> forward. Is there something special about PCI-e here? >> > > It's not tracking allocation. It is that accessing memory above 1 MiB > is incredibly painful in the BIOS environment, which basically means > MMIO is inaccessible.Oh, you mean in real mode. SeaBIOS runs the virtio code in 32-bit mode with a flat memory layout. There are loads of ASSERT32FLAT()s in the code to make sure of this. Regards, Anthony Liguori> > -hpa > > > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
Gleb Natapov
2013-Jun-06 06:34 UTC
[PATCH RFC] virtio-pci: new config layout: using memory BAR
On Wed, Jun 05, 2013 at 07:41:17PM -0500, Anthony Liguori wrote:> "H. Peter Anvin" <hpa at zytor.com> writes: > > > On 06/05/2013 03:08 PM, Anthony Liguori wrote: > >>> > >>> Definitely an option. However, we want to be able to boot from native > >>> devices, too, so having an I/O BAR (which would not be used by the OS > >>> driver) should still at the very least be an option. > >> > >> What makes it so difficult to work with an MMIO bar for PCI-e? > >> > >> With legacy PCI, tracking allocation of MMIO vs. PIO is pretty straight > >> forward. Is there something special about PCI-e here? > >> > > > > It's not tracking allocation. It is that accessing memory above 1 MiB > > is incredibly painful in the BIOS environment, which basically means > > MMIO is inaccessible. > > Oh, you mean in real mode. > > SeaBIOS runs the virtio code in 32-bit mode with a flat memory layout. > There are loads of ASSERT32FLAT()s in the code to make sure of this. >Well, not exactly. Initialization is done in 32bit, but disk reads/writes are done in 16bit mode since it should work from int13 interrupt handler. The only way I know to access MMIO bars from 16 bit is to use SMM which we do not have in KVM. -- Gleb.
H. Peter Anvin
2013-Jun-06 13:53 UTC
[PATCH RFC] virtio-pci: new config layout: using memory BAR
On 06/05/2013 11:34 PM, Gleb Natapov wrote:>> >> SeaBIOS runs the virtio code in 32-bit mode with a flat memory layout. >> There are loads of ASSERT32FLAT()s in the code to make sure of this. >> > Well, not exactly. Initialization is done in 32bit, but disk > reads/writes are done in 16bit mode since it should work from int13 > interrupt handler. The only way I know to access MMIO bars from 16 bit > is to use SMM which we do not have in KVM. >In some ways it is even worse for PXE, since PXE is defined to work from 16-bit protected mode... but the mechanism to request a segment mapping for the high memory area doesn't actually work. -hpa
Anthony Liguori
2013-Jun-06 15:02 UTC
[PATCH RFC] virtio-pci: new config layout: using memory BAR
Gleb Natapov <gleb at redhat.com> writes:> On Wed, Jun 05, 2013 at 07:41:17PM -0500, Anthony Liguori wrote: >> "H. Peter Anvin" <hpa at zytor.com> writes: >> >> > On 06/05/2013 03:08 PM, Anthony Liguori wrote: >> >>> >> >>> Definitely an option. However, we want to be able to boot from native >> >>> devices, too, so having an I/O BAR (which would not be used by the OS >> >>> driver) should still at the very least be an option. >> >> >> >> What makes it so difficult to work with an MMIO bar for PCI-e? >> >> >> >> With legacy PCI, tracking allocation of MMIO vs. PIO is pretty straight >> >> forward. Is there something special about PCI-e here? >> >> >> > >> > It's not tracking allocation. It is that accessing memory above 1 MiB >> > is incredibly painful in the BIOS environment, which basically means >> > MMIO is inaccessible. >> >> Oh, you mean in real mode. >> >> SeaBIOS runs the virtio code in 32-bit mode with a flat memory layout. >> There are loads of ASSERT32FLAT()s in the code to make sure of this. >> > Well, not exactly. Initialization is done in 32bit, but disk > reads/writes are done in 16bit mode since it should work from int13 > interrupt handler. The only way I know to access MMIO bars from 16 bit > is to use SMM which we do not have in KVM.Ah, if it's just the dataplane operations then there's another solution. We can introduce a virtqueue flag that asks the backend to poll for new requests. Then SeaBIOS can add the request to the queue and not worry about kicking or reading the ISR. SeaBIOS is polling for completion anyway. Regards, Anthony Liguori> > -- > Gleb. > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html
Gerd Hoffmann
2013-Jun-06 15:06 UTC
[PATCH RFC] virtio-pci: new config layout: using memory BAR
On 06/06/13 08:34, Gleb Natapov wrote:> On Wed, Jun 05, 2013 at 07:41:17PM -0500, Anthony Liguori wrote: >> >> Oh, you mean in real mode. >> >> SeaBIOS runs the virtio code in 32-bit mode with a flat memory layout. >> There are loads of ASSERT32FLAT()s in the code to make sure of this. >> > Well, not exactly. Initialization is done in 32bit, but disk > reads/writes are done in 16bit mode since it should work from int13 > interrupt handler.Exactly. It's "only" the initialization code which has ASSERt32FLAT() all over the place. Which actually is the majority of the code in most cases as all the hardware detection and initialization code is there. But kicking I/O requests must work from 16bit mode too.> The only way I know to access MMIO bars from 16 bit > is to use SMM which we do not have in KVM.For seabios itself this isn't a big issue, see pci_{readl,writel} in src/pci.c. When called in 16bit mode it goes into 32bit mode temporarily, just for accessing the mmio register. ahci driver uses it, xhci driver (wip atm) will use that too, and virtio-{blk,scsi} drivers in seabios can do the same. But as hpa mentioned it will be more tricky for option roms (aka virtio-net). cheers, Gerd
Reasonably Related Threads
- [PATCH RFC] virtio-pci: new config layout: using memory BAR
- [PATCH RFC] virtio-pci: new config layout: using memory BAR
- [PATCH RFC] virtio-pci: new config layout: using memory BAR
- [PATCH RFC] virtio-pci: new config layout: using memory BAR
- [PATCH RFC] virtio-pci: new config layout: using memory BAR