veerasena reddy
2011-Jun-13 08:54 UTC
[Xen-devel] Accessing PCI BAR space directly from XEN hypervisor
Hi,
I am doing an experiment to see if I can get more throughput by moving the
Tx ring update and doorbell ring mechanism to hypervisor and provide a
hypercall to HVM Guest. When there is a Tx packet ready, the HVM guest
invokes the hypercall along with the packet address as argument. The
hypercall function will write the packet address to the Tx ring and rings
the doorbell directly without involving dom0.
AS part of this, i need to access IO Memory BAR region of the PCI network
device from XEN hypervisor (hypercall handler). For this, i tried converting
the BAR Physical address (captured from lspci as shown below) of the PCI
device to MFN, but got all F''s (invalid entry). Even
pfn_valid(0xf0904000)
failed on this address. Where as I could successfully map a dom0 page
(allocated using vmalloc()) and access in hypervisor using
map_domain_page(mfn).
Could any one please suggest how to access the BAR regions of a PCI device
from hypervisor? how to do mapping and read/write operations?
Also, Could you please explain the difference between mapping locally
allocated pages and PCI BAR memory of dom0 in hypervisor?
Thanks in advance.
Regards,
VSR.
=========================[root@fc13 xen]# lspci -vvs 09:00.0
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B
PCI Express Gigabit Ethernet controller (rev 03)
Subsystem: Lenovo Device 2131
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 1247
Region 0: I/O ports at 4000 [size=256]
Region 2: Memory at f0904000 (64-bit, prefetchable) [size=4K]
Region 4: Memory at f0900000 (64-bit, prefetchable) [size=16K]
[virtual] Expansion ROM at f0920000 [disabled] [size=128K]
==========================
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
veerasena reddy
2011-Jun-14 03:25 UTC
[Xen-devel] Re: Accessing PCI BAR space directly from XEN hypervisor
Hi, Can someone please throw some light on this Thanks in advance. Regards, VSR. On Mon, Jun 13, 2011 at 2:24 PM, veerasena reddy <veeruyours@gmail.com>wrote:> Hi, > > I am doing an experiment to see if I can get more throughput by moving the > Tx ring update and doorbell ring mechanism to hypervisor and provide a > hypercall to HVM Guest. When there is a Tx packet ready, the HVM guest > invokes the hypercall along with the packet address as argument. The > hypercall function will write the packet address to the Tx ring and rings > the doorbell directly without involving dom0. > > AS part of this, i need to access IO Memory BAR region of the PCI network > device from XEN hypervisor (hypercall handler). For this, i tried converting > the BAR Physical address (captured from lspci as shown below) of the PCI > device to MFN, but got all F''s (invalid entry). Even pfn_valid(0xf0904000) > failed on this address. Where as I could successfully map a dom0 page > (allocated using vmalloc()) and access in hypervisor using > map_domain_page(mfn). > > Could any one please suggest how to access the BAR regions of a PCI device > from hypervisor? how to do mapping and read/write operations? > > Also, Could you please explain the difference between mapping locally > allocated pages and PCI BAR memory of dom0 in hypervisor? > > Thanks in advance. > > Regards, > VSR. > =========================> [root@fc13 xen]# lspci -vvs 09:00.0 > 09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B > PCI Express Gigabit Ethernet controller (rev 03) > Subsystem: Lenovo Device 2131 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- > <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 1247 > Region 0: I/O ports at 4000 [size=256] > Region 2: Memory at f0904000 (64-bit, prefetchable) [size=4K] > Region 4: Memory at f0900000 (64-bit, prefetchable) [size=16K] > [virtual] Expansion ROM at f0920000 [disabled] [size=128K] > ==========================>_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2011-Jun-14 06:38 UTC
Re: [Xen-devel] Re: Accessing PCI BAR space directly from XEN hypervisor
On 14/06/2011 04:25, "veerasena reddy" <veeruyours@gmail.com> wrote:> Hi, > > Can someone please throw some light on this > > Thanks in advance.You''ll have to set up a fixmap slot and use set_fixmap and fix_to_virt. Xen doesn''t have ioremap. -- Keir> Regards, > VSR. > On Mon, Jun 13, 2011 at 2:24 PM, veerasena reddy <veeruyours@gmail.com> wrote: >> Hi, >> >> I am doing an experiment to see if I can get more throughput by moving the Tx >> ring update and doorbell ring mechanism to hypervisor and provide a hypercall >> to HVM Guest. When there is a Tx packet ready, the HVM guest invokes the >> hypercall along with the packet address as argument. The hypercall function >> will write the packet address to the Tx ring and rings the doorbell directly >> without involving dom0. >> >> AS part of this, i need to access IO Memory BAR region of the PCI network >> device from XEN hypervisor (hypercall handler). For this, i tried converting >> the BAR Physical address (captured from lspci as shown below) of the PCI >> device to MFN, but got all F''s (invalid entry). Even pfn_valid(0xf0904000) >> failed on this address. Where as I could successfully map a dom0 page >> (allocated using vmalloc()) and access in hypervisor using >> map_domain_page(mfn). >> >> Could any one please suggest how to access the BAR regions of a PCI device >> from hypervisor? how to do mapping and read/write operations? >> >> Also, Could you please explain the difference between mapping locally >> allocated pages and PCI BAR memory of dom0 in hypervisor? >> >> Thanks in advance. >> >> Regards, >> VSR. >> =========================>> [root@fc13 xen]# lspci -vvs 09:00.0 >> 09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B >> PCI Express Gigabit Ethernet controller (rev 03) >> Subsystem: Lenovo Device 2131 >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- >> Stepping- SERR+ FastB2B- DisINTx+ >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- >> <MAbort- >SERR- <PERR- INTx- >> Latency: 0, Cache Line Size: 64 bytes >> Interrupt: pin A routed to IRQ 1247 >> Region 0: I/O ports at 4000 [size=256] >> Region 2: Memory at f0904000 (64-bit, prefetchable) [size=4K] >> Region 4: Memory at f0900000 (64-bit, prefetchable) [size=16K] >> [virtual] Expansion ROM at f0920000 [disabled] [size=128K] >> ==========================> > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel