Guillaume Robin
2009-Mar-05 14:16 UTC
Re[Xen-devel] ceiving NMI, when writing on mmaped PCI
Hi,
I''m using xen 3.3.1 and running a 2.6.18.8-xen Dom0 (compiled from
xen.org).
My hardware is a HP Dc7800 with VT and VT-D activated.
I assigned a acquisition card (NI6023E) to a HVM guest from port 07:09.0
On Dom0 lspci -v give me :
#07:09.0 Class ff00: National Instruments PCI-6023E
# Flags: bus master, medium devsel, latency 32, IRQ 16
# Memory at f0200000 (32-bit, non-prefetchable) [size=4K]
# Memory at f0201000 (32-bit, non-prefetchable) [size=4K]
When I load my VM I get these debug messages so I think memory mapping is OK
for this card.
(from f0200000 on Dom0 to f3001000 on Dom1 and from f0201000 on Dom0 to
f3002000 on Dom1)
#(XEN) [VT-D]iommu.c:1263: domain_context_mapping:PCI: bdf = 7:9.0
#(XEN) [VT-D]iommu.c:1278:d0 domain_context_mapping:map: bdf = 7:9.0 ->
0:1e.0
#(XEN) [VT-D]io.c:140: VT-d irq bind: m_irq = 10 device = 6 intx = 0 //
device 6 on HVM is my card
...
#(XEN) HVM1: pci dev 06:0 bar 10 size 00001000: f3001000
#(XEN) domctl.c:776:d0 memory_map:add: gfn=f3001 mfn=f0200 nr_mfns=1
#(XEN) HVM1: pci dev 06:0 bar 14 size 00001000: f3002000
#(XEN) domctl.c:776:d0 memory_map:add: gfn=f3002 mfn=f0201 nr_mfns=1
I tried some test programs witch all resulted in NMI error on Dom0, so i
tried a simple test to check this mapping.
When I try to write on Bar0 from this card no error appear, but if I write
on Bar1 I get NMI errors.
test program (on HVM):
-------------------------------------------
...
printf("memory mapping bar0 : 0x%X, and 1 : 0x%X...\n", physicalBar0,
physicalBar1); // physicalBar0 = 0xF3001000 and physicalBar1=0xF3002000
mem0 = mmap (NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd,
physicalBar0);
mem1 = mmap (NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd,
physicalBar1);
printf("Bar0 mapped to : 0x%X, Bar1 to : 0x%X \n", mem0, mem1);
// Ex : Bar0 mapped to : 0x14CDB000, Bar1 to : 0x14CDC000 (these address
doesn''t realy matter)
if ( MAP_FAILED == mem0 || MAP_FAILED == mem1)
{
close (fd);
printf("map failed \n");
return NULL;
}
printf("testing write on mem0\n");
sleep(1); // Sleep to give time to printf before NMI appear on Dom0 (using
SSH from dom0 to guest to see both output synchronously)
int* titi=(int*)mem0;
*titi=0x1; // Look to work
printf("testing write on mem1\n");
sleep(1);
titi=(int*)mem1;
*titi=0x1; // ==> receiving a NMI on Dom0 while executing this line
...
-------------------------------------------
NMI message :
#Mar 2 14:53:38 XenVT kernel: Uhhuh. NMI received. Dazed and confused, but
trying to continue
#Mar 2 14:53:38 XenVT kernel: You probably have a hardware problem with
your RAM chips
Why this code working on simple kernel (tested on 2.6.18-amd64), can''t
work
in my HVM with this card assigned ?
btw a message in xm dmesg look strange :
(XEN) [VT-D]iommu.c:1736: Queued Invalidation hardware not found
(XEN) [VT-D]iommu.c:1745: Interrupt Remapping hardware not found
Could this be linked ?
Is there a problem in xen with multiple bar remapping ?
Thank you for your help.
--
View this message in context:
http://www.nabble.com/Receiving-NMI%2C-when-writing-on-mmaped-PCI-tp22352457p22352457.html
Sent from the Xen - Dev mailing list archive at Nabble.com.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Guillaume Robin
2009-Mar-10 16:32 UTC
Re: Re[Xen-devel] ceiving NMI, when writing on mmaped PCI
Hi, This problem comes from initialization of NI cards: they write BAR1 physical address inside the BAR0 , but in our case this address is a virtual one so then, card try to write inside this unknown address witch lead to a NMI or host freeze/reboot. I see 2 solutions : - be able to force a virtual address for a PCI card attached to a HVM (inside the HVM) eg : today I have 0xF3001000(VM) mapped to 0xF0200000(phys) best would be to force VM PCI card address to 0xF0200000 Is it necessary to have a VM PCI address different from real physical address for Xen and VT-D to work ? - Be able to "catch" writing on specifics register (like the one needed by NI to configure this card) and replace data with the real physical address (witch is suposed to be the data) ? Could anyone tell me which one of these solutions (or both) can be done? Thank you. -- View this message in context: http://www.nabble.com/Receiving-NMI%2C-when-writing-on-mmaped-PCI-tp22352457p22438199.html Sent from the Xen - Dev mailing list archive at Nabble.com. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel