I am having problems getting an SR-IOV device to run on the latest Xen pv-ops Dom0 kernel with Xen 4.0.0-testing tree. Note that this actually all works when I run that same pv-ops Dom0 kernel *without* the Xen hypervisor, so the problem must be Xen specific. The Dom0 kernel is from the latest pv-ops git tree. Xen/Dom0 boot fine, then I load the driver on the SR-IOV device, and I get the following kernel crash: [ 548.040328] ixgbe 0000:04:00.0: enabling device (0005 -> 0007) [ 548.040337] xen: registering gsi 16 triggering 0 polarity 1 [ 548.040340] xen_allocate_pirq: returning irq 16 for gsi 16 [ 548.040342] xen: --> irq=16 [ 548.040345] xen_set_ioapic_routing: irq 16 gsi 16 vector 16 ioapic 0 pin 16 triggering 1 polarity 1 [ 548.040352] ixgbe 0000:04:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 [ 548.040407] ixgbe 0000:04:00.0: setting latency timer to 64 [ 561.316277] BUG: unable to handle kernel paging request at ffffc90020e10010 [ 561.316277] IP: [<ffffffffa00d14c2>] ixgbe_init_ops_generic+0x12/0x1c0 [ixgbe] [ 561.316277] PGD 210b067 PUD 210c067 PMD 3d5d7067 PTE 0 [ 561.316277] Oops: 0000 [#1] SMP [ 565.470069] last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.1/irq [ 565.470069] CPU 1 [ 565.470069] Modules linked in: ixgbe(+) sg ata_generic e1000e usb_storage ahci libata sd_mod scsi_mod [last unloaded: ehci_hcd] [ 565.470069] Pid: 4575, comm: modprobe Not tainted 2.6.31.13-sriov #12 Montevina Platform [ 565.470069] RIP: e030:[<ffffffffa00d14c2>] [<ffffffffa00d14c2>] ixgbe_init_ops_generic+0x12/0x1c0 [ixgbe] [ 565.470069] RSP: e02b:ffff88002e4e1c78 EFLAGS: 00010246 [ 565.470069] RAX: ffff880026099550 RBX: ffff880026099550 RCX: 0000000080040008 [ 565.470069] RDX: ffffc90020e00000 RSI: ffff8800260997f0 RDI: ffff880026099540 [ 565.470069] RBP: ffff88002e4e1c78 R08: 0000000000080000 R09: ffff88002e4e1c9c [ 565.470069] R10: 0000000000000000 R11: ffffffffa00ef120 R12: ffff880026099540 [ 565.470069] R13: ffff880026099740 R14: 00000000fffffffb R15: ffff880026098000 [ 565.470069] FS: 00007fc01f28b6e0(0000) GS:ffffc90000018000(0000) knlGS:0000000000000000 [ 565.470069] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 565.470069] CR2: ffffc90020e10010 CR3: 000000002e49e000 CR4: 0000000000002660 [ 565.470069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 565.470069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 565.470069] Process modprobe (pid: 4575, threadinfo ffff88002e4e0000, task ffff88003e4a6b00) [ 565.470069] Stack: [ 565.470069] ffff88002e4e1ca8 ffffffffa00da676 ffff880026099540 ffff880026098580 [ 565.470069] <0> ffff880026099540 00000000fffffffb ffff88002e4e1cc8 ffffffffa00d3698 [ 565.470069] <0> 00000000fffffffb ffff88003fa9d000 ffff88002e4e1d48 ffffffffa00e0687 [ 565.470069] Call Trace: [ 565.470069] [<ffffffffa00da676>] ixgbe_init_ops_82599+0x36/0x1a0 [ixgbe] On the Xen console (xm dmesg) I get this at the same time: (XEN) mm.c:859:d0 Error getting mfn 40300 (pfn 5555555555555555) from L1 entry 8000000040300473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40301 (pfn 5555555555555555) from L1 entry 8000000040301473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40302 (pfn 5555555555555555) from L1 entry 8000000040302473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40303 (pfn 5555555555555555) from L1 entry 8000000040303473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40304 (pfn 5555555555555555) from L1 entry 8000000040304473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40305 (pfn 5555555555555555) from L1 entry 8000000040305473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40306 (pfn 5555555555555555) from L1 entry 8000000040306473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40307 (pfn 5555555555555555) from L1 entry 8000000040307473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40308 (pfn 5555555555555555) from L1 entry 8000000040308473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40309 (pfn 5555555555555555) from L1 entry 8000000040309473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4030a (pfn 5555555555555555) from L1 entry 800000004030a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4030b (pfn 5555555555555555) from L1 entry 800000004030b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4030c (pfn 5555555555555555) from L1 entry 800000004030c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4030d (pfn 5555555555555555) from L1 entry 800000004030d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4030e (pfn 5555555555555555) from L1 entry 800000004030e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4030f (pfn 5555555555555555) from L1 entry 800000004030f473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40310 (pfn 5555555555555555) from L1 entry 8000000040310473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40311 (pfn 5555555555555555) from L1 entry 8000000040311473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40312 (pfn 5555555555555555) from L1 entry 8000000040312473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40313 (pfn 5555555555555555) from L1 entry 8000000040313473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40314 (pfn 5555555555555555) from L1 entry 8000000040314473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40315 (pfn 5555555555555555) from L1 entry 8000000040315473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40316 (pfn 5555555555555555) from L1 entry 8000000040316473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40317 (pfn 5555555555555555) from L1 entry 8000000040317473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40318 (pfn 5555555555555555) from L1 entry 8000000040318473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40319 (pfn 5555555555555555) from L1 entry 8000000040319473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4031a (pfn 5555555555555555) from L1 entry 800000004031a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4031b (pfn 5555555555555555) from L1 entry 800000004031b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4031c (pfn 5555555555555555) from L1 entry 800000004031c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4031d (pfn 5555555555555555) from L1 entry 800000004031d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4031e (pfn 5555555555555555) from L1 entry 800000004031e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4031f (pfn 5555555555555555) from L1 entry 800000004031f473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40320 (pfn 5555555555555555) from L1 entry 8000000040320473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40321 (pfn 5555555555555555) from L1 entry 8000000040321473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40322 (pfn 5555555555555555) from L1 entry 8000000040322473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40323 (pfn 5555555555555555) from L1 entry 8000000040323473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40324 (pfn 5555555555555555) from L1 entry 8000000040324473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40325 (pfn 5555555555555555) from L1 entry 8000000040325473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40326 (pfn 5555555555555555) from L1 entry 8000000040326473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40327 (pfn 5555555555555555) from L1 entry 8000000040327473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40328 (pfn 5555555555555555) from L1 entry 8000000040328473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40329 (pfn 5555555555555555) from L1 entry 8000000040329473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4032a (pfn 5555555555555555) from L1 entry 800000004032a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4032b (pfn 5555555555555555) from L1 entry 800000004032b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4032c (pfn 5555555555555555) from L1 entry 800000004032c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4032d (pfn 5555555555555555) from L1 entry 800000004032d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4032e (pfn 5555555555555555) from L1 entry 800000004032e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4032f (pfn 5555555555555555) from L1 entry 800000004032f473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40330 (pfn 5555555555555555) from L1 entry 8000000040330473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40331 (pfn 5555555555555555) from L1 entry 8000000040331473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40332 (pfn 5555555555555555) from L1 entry 8000000040332473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40333 (pfn 5555555555555555) from L1 entry 8000000040333473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40334 (pfn 5555555555555555) from L1 entry 8000000040334473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40335 (pfn 5555555555555555) from L1 entry 8000000040335473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40336 (pfn 5555555555555555) from L1 entry 8000000040336473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40337 (pfn 5555555555555555) from L1 entry 8000000040337473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40338 (pfn 5555555555555555) from L1 entry 8000000040338473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40339 (pfn 5555555555555555) from L1 entry 8000000040339473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4033a (pfn 5555555555555555) from L1 entry 800000004033a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4033b (pfn 5555555555555555) from L1 entry 800000004033b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4033c (pfn 5555555555555555) from L1 entry 800000004033c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4033d (pfn 5555555555555555) from L1 entry 800000004033d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4033e (pfn 5555555555555555) from L1 entry 800000004033e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4033f (pfn 5555555555555555) from L1 entry 800000004033f473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40340 (pfn 5555555555555555) from L1 entry 8000000040340473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40341 (pfn 5555555555555555) from L1 entry 8000000040341473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40342 (pfn 5555555555555555) from L1 entry 8000000040342473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40343 (pfn 5555555555555555) from L1 entry 8000000040343473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40344 (pfn 5555555555555555) from L1 entry 8000000040344473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40345 (pfn 5555555555555555) from L1 entry 8000000040345473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40346 (pfn 5555555555555555) from L1 entry 8000000040346473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40347 (pfn 5555555555555555) from L1 entry 8000000040347473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40348 (pfn 5555555555555555) from L1 entry 8000000040348473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40349 (pfn 5555555555555555) from L1 entry 8000000040349473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4034a (pfn 5555555555555555) from L1 entry 800000004034a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4034b (pfn 5555555555555555) from L1 entry 800000004034b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4034c (pfn 5555555555555555) from L1 entry 800000004034c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4034d (pfn 5555555555555555) from L1 entry 800000004034d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4034e (pfn 5555555555555555) from L1 entry 800000004034e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4034f (pfn 5555555555555555) from L1 entry 800000004034f473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40350 (pfn 5555555555555555) from L1 entry 8000000040350473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40351 (pfn 5555555555555555) from L1 entry 8000000040351473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40352 (pfn 5555555555555555) from L1 entry 8000000040352473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40353 (pfn 5555555555555555) from L1 entry 8000000040353473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40354 (pfn 5555555555555555) from L1 entry 8000000040354473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40355 (pfn 5555555555555555) from L1 entry 8000000040355473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40356 (pfn 5555555555555555) from L1 entry 8000000040356473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40357 (pfn 5555555555555555) from L1 entry 8000000040357473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40358 (pfn 5555555555555555) from L1 entry 8000000040358473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40359 (pfn 5555555555555555) from L1 entry 8000000040359473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4035a (pfn 5555555555555555) from L1 entry 800000004035a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4035b (pfn 5555555555555555) from L1 entry 800000004035b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4035c (pfn 5555555555555555) from L1 entry 800000004035c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4035d (pfn 5555555555555555) from L1 entry 800000004035d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4035e (pfn 5555555555555555) from L1 entry 800000004035e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4035f (pfn 5555555555555555) from L1 entry 800000004035f473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40360 (pfn 5555555555555555) from L1 entry 8000000040360473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40361 (pfn 5555555555555555) from L1 entry 8000000040361473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40362 (pfn 5555555555555555) from L1 entry 8000000040362473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40363 (pfn 5555555555555555) from L1 entry 8000000040363473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40364 (pfn 5555555555555555) from L1 entry 8000000040364473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40365 (pfn 5555555555555555) from L1 entry 8000000040365473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40366 (pfn 5555555555555555) from L1 entry 8000000040366473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40367 (pfn 5555555555555555) from L1 entry 8000000040367473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40368 (pfn 5555555555555555) from L1 entry 8000000040368473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40369 (pfn 5555555555555555) from L1 entry 8000000040369473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4036a (pfn 5555555555555555) from L1 entry 800000004036a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4036b (pfn 5555555555555555) from L1 entry 800000004036b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4036c (pfn 5555555555555555) from L1 entry 800000004036c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4036d (pfn 5555555555555555) from L1 entry 800000004036d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4036e (pfn 5555555555555555) from L1 entry 800000004036e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4036f (pfn 5555555555555555) from L1 entry 800000004036f473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40370 (pfn 5555555555555555) from L1 entry 8000000040370473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40371 (pfn 5555555555555555) from L1 entry 8000000040371473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40372 (pfn 5555555555555555) from L1 entry 8000000040372473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40373 (pfn 5555555555555555) from L1 entry 8000000040373473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40374 (pfn 5555555555555555) from L1 entry 8000000040374473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40375 (pfn 5555555555555555) from L1 entry 8000000040375473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40376 (pfn 5555555555555555) from L1 entry 8000000040376473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40377 (pfn 5555555555555555) from L1 entry 8000000040377473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40378 (pfn 5555555555555555) from L1 entry 8000000040378473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 40379 (pfn 5555555555555555) from L1 entry 8000000040379473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4037a (pfn 5555555555555555) from L1 entry 800000004037a473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4037b (pfn 5555555555555555) from L1 entry 800000004037b473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4037c (pfn 5555555555555555) from L1 entry 800000004037c473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4037d (pfn 5555555555555555) from L1 entry 800000004037d473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4037e (pfn 5555555555555555) from L1 entry 800000004037e473 for l1e_owner=0, pg_owner=32753 (XEN) mm.c:859:d0 Error getting mfn 4037f (pfn 5555555555555555) from L1 entry 800000004037f473 for l1e_owner=0, pg_owner=32753 [ 561.316277] BUG: unable to handle kernel paging request at ffffc90020e10010 [ 561.316277] IP: [<ffffffffa00d14c2>] ixgbe_init_ops_generic+0x12/0x1c0 [ixgbe] [ 561.316277] PGD 210b067 PUD 210c067 PMD 3d5d7067 PTE 0 [ 561.316277] Oops: 0000 [#1] SMP [ 565.470069] last sysfs file: /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.1/irq [ 565.470069] CPU 1 [ 565.470069] Modules linked in: ixgbe(+) sg ata_generic e1000e usb_storage ahci libata sd_mod scsi_mod [last unloaded: ehci_hcd] [ 565.470069] Pid: 4575, comm: modprobe Not tainted 2.6.31.13-sriov #12 Montevina Platform [ 565.470069] RIP: e030:[<ffffffffa00d14c2>] [<ffffffffa00d14c2>] ixgbe_init_ops_generic+0x12/0x1c0 [ixgbe] [ 565.470069] RSP: e02b:ffff88002e4e1c78 EFLAGS: 00010246 [ 565.470069] RAX: ffff880026099550 RBX: ffff880026099550 RCX: 0000000080040008 [ 565.470069] RDX: ffffc90020e00000 RSI: ffff8800260997f0 RDI: ffff880026099540 [ 565.470069] RBP: ffff88002e4e1c78 R08: 0000000000080000 R09: ffff88002e4e1c9c [ 565.470069] R10: 0000000000000000 R11: ffffffffa00ef120 R12: ffff880026099540 [ 565.470069] R13: ffff880026099740 R14: 00000000fffffffb R15: ffff880026098000 [ 565.470069] FS: 00007fc01f28b6e0(0000) GS:ffffc90000018000(0000) knlGS:0000000000000000 [ 565.470069] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b [ 565.470069] CR2: ffffc90020e10010 CR3: 000000002e49e000 CR4: 0000000000002660 [ 565.470069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 565.470069] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 565.470069] Process modprobe (pid: 4575, threadinfo ffff88002e4e0000, task ffff88003e4a6b00) [ 565.470069] Stack: [ 565.470069] ffff88002e4e1ca8 ffffffffa00da676 ffff880026099540 ffff880026098580 [ 565.470069] <0> ffff880026099540 00000000fffffffb ffff88002e4e1cc8 ffffffffa00d3698 [ 565.470069] <0> 00000000fffffffb ffff88003fa9d000 ffff88002e4e1d48 ffffffffa00e0687 [ 565.470069] Call Trace: [ 565.470069] [<ffffffffa00da676>] ixgbe_init_ops_82599+0x36/0x1a0 [ixgbe] [ 565.470069] [<ffffffffa00d3698>] ixgbe_init_shared_code+0x48/0x50 [ixgbe] [ 565.470069] [<ffffffffa00e0687>] ixgbe_probe+0x317/0x1470 [ixgbe] [ 565.470069] [<ffffffff811e73dc>] ? pci_match_id+0x1f/0x43 [ 565.470069] [<ffffffff811e7412>] local_pci_probe+0x12/0x16 [ 565.470069] [<ffffffff811e7efc>] pci_device_probe+0x5c/0x88 [ 565.470069] [<ffffffff812acd38>] ? driver_sysfs_add+0x4d/0x73 [ 565.470069] [<ffffffff812ace89>] driver_probe_device+0xb2/0x136 [ 565.470069] [<ffffffff812acf61>] __driver_attach+0x54/0x77 [ 567.919230] [<ffffffff812acf0d>] ? __driver_attach+0x0/0x77 [ 567.919230] [<ffffffff812acf0d>] ? __driver_attach+0x0/0x77 [ 567.919230] [<ffffffff812ac518>] bus_for_each_dev+0x49/0x78 [ 567.919230] [<ffffffff812acce9>] driver_attach+0x1c/0x1e [ 567.919230] [<ffffffff812ac027>] bus_add_driver+0xba/0x21f [ 567.919230] [<ffffffff812ad2c6>] driver_register+0x9e/0x115 [ 567.919230] [<ffffffff811e8148>] __pci_register_driver+0x50/0xad [ 567.919230] [<ffffffffa0000000>] ? ixgbe_init_module+0x0/0x51 [ixgbe] [ 567.919230] [<ffffffffa000004f>] ixgbe_init_module+0x4f/0x51 [ixgbe] [ 567.919230] [<ffffffff8100a096>] do_one_initcall+0x59/0x15d [ 567.919230] [<ffffffff81071c11>] sys_init_module+0xcb/0x1fe [ 568.693541] [<ffffffff81013bc2>] system_call_fastpath+0x16/0x1b [ 568.693541] Code: 5d c9 c3 48 83 c4 08 b8 ff ff ff ff 5b 41 5c 41 5d c9 c3 66 0f 1f 44 00 00 55 48 8d b7 b0 02 00 00 48 89 e5 48 8d 47 10 48 8b 17 <8b> 8a 10 00 01 00 48 c7 c2 90 1e 0d a0 48 c7 4 [ 568.975209] RIP [<ffffffffa00d14c2>] ixgbe_init_ops_generic+0x12/0x1c0 [ixgbe] [ 569.114713] RSP <ffff88002e4e1c78> [ 569.114713] CR2: ffffc90020e10010 [ 569.192440] ---[ end trace 9fbdac6ca1130b89 ]--- I boot Xen with ''dom0_mem=max:1024M iommu=0'' set in GRUB. The system supports VT-d, but I am disabling it. I am not quite sure where to further look for the problem. Is the a bug in the driver (but then it works when booting Dom0 without Xen)? Or did I mis-configure something? I am not running any guests yet, and I don''t use pciback to hide devices either. This is just trying to boot up Dom0. Any help would be greatly appreciated. Below is also my kernel configuration, if that is of any help. Thanks, Anna [root@211052610-f-cpu src]# cat /lib/modules/`uname -r`/build/.config | grep PCI CONFIG_PCI_QUIRKS=y CONFIG_XEN_DOM0_PCI=y CONFIG_XEN_PCI_PASSTHROUGH=y # CONFIG_ACPI_PCI_SLOT is not set # Bus options (PCI etc.) CONFIG_PCI=y CONFIG_PCI_DIRECT=y CONFIG_PCI_MMCONFIG=y CONFIG_PCI_XEN=y CONFIG_PCI_DOMAINS=y CONFIG_PCIEPORTBUS=y CONFIG_HOTPLUG_PCI_PCIE=y CONFIG_PCIEAER=y # CONFIG_PCIE_ECRC is not set # CONFIG_PCIEAER_INJECT is not set # CONFIG_PCIEASPM is not set CONFIG_PCI_MSI=y # CONFIG_PCI_LEGACY is not set # CONFIG_PCI_DEBUG is not set CONFIG_PCI_STUB=m CONFIG_XEN_PCIDEV_FRONTEND=m CONFIG_PCI_IOV=y CONFIG_HOTPLUG_PCI=y # CONFIG_HOTPLUG_PCI_FAKE is not set # CONFIG_HOTPLUG_PCI_ACPI is not set # CONFIG_HOTPLUG_PCI_CPCI is not set # CONFIG_HOTPLUG_PCI_SHPC is not set # PCI IDE chipsets support CONFIG_BLK_DEV_IDEPCI=y CONFIG_BLK_DEV_IDEDMA_PCI=y # CONFIG_PATA_CMD640_PCI is not set CONFIG_NET_PCI=y CONFIG_NE2K_PCI=y # CONFIG_SERIO_PCIPS2 is not set CONFIG_SERIAL_8250_PCI=y # PCI-based Watchdog Cards # CONFIG_PCIPCWATCHDOG is not set # CONFIG_WDTPCI is not set CONFIG_SND_PCI=y # CONFIG_SND_CMIPCI is not set # CONFIG_SND_YMFPCI is not set CONFIG_XEN_PCIDEV_BACKEND=m # CONFIG_XEN_PCIDEV_BACKEND_VPCI is not set CONFIG_XEN_PCIDEV_BACKEND_PASS=y # CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set # CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set CONFIG_XEN_PCIDEV_BE_DEBUG=y # CONFIG_VIRTIO_PCI is not set [root@211052610-f-cpu src]# cat /lib/modules/`uname -r`/build/.config | grep XEN CONFIG_XEN=y CONFIG_XEN_TIME_VSYSCALL=y CONFIG_XEN_MAX_DOMAIN_MEMORY=32 CONFIG_XEN_SAVE_RESTORE=y CONFIG_XEN_DEBUG_FS=y CONFIG_XEN_DOM0=y CONFIG_XEN_PRIVILEGED_GUEST=y CONFIG_XEN_DOM0_PCI=y CONFIG_XEN_PCI_PASSTHROUGH=y CONFIG_PCI_XEN=y CONFIG_XEN_PCIDEV_FRONTEND=m CONFIG_XEN_BLKDEV_FRONTEND=m # CONFIG_NETXEN_NIC is not set CONFIG_XEN_NETDEV_FRONTEND=m # CONFIG_XEN_NETCHANNEL2 is not set CONFIG_XEN_KBDDEV_FRONTEND=y CONFIG_HVC_XEN=y CONFIG_XEN_FBDEV_FRONTEND=y CONFIG_XEN_BALLOON=y CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_DEV_EVTCHN=y CONFIG_XEN_BACKEND=y CONFIG_XEN_BLKDEV_BACKEND=m CONFIG_XEN_BLKDEV_TAP=m CONFIG_XEN_BLKBACK_PAGEMAP=m CONFIG_XEN_NETDEV_BACKEND=m CONFIG_XEN_PCIDEV_BACKEND=m # CONFIG_XEN_PCIDEV_BACKEND_VPCI is not set CONFIG_XEN_PCIDEV_BACKEND_PASS=y # CONFIG_XEN_PCIDEV_BACKEND_SLOT is not set # CONFIG_XEN_PCIDEV_BACKEND_CONTROLLER is not set CONFIG_XEN_PCIDEV_BE_DEBUG=y CONFIG_XENFS=y CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y CONFIG_XEN_XENBUS_FRONTEND=y CONFIG_XEN_S3=y CONFIG_ACPI_PROCESSOR_XEN=y _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2010-May-26 07:31 UTC
Re: [Xen-devel] Error getting mfn for PCI SR-IOV device?
>>> On 25.05.10 at 15:58, "Fischer, Anna" <anna.fischer@hp.com> wrote: > (XEN) mm.c:859:d0 Error getting mfn 40300 (pfn 5555555555555555) from L1 > entry 8000000040300473 for l1e_owner=0, pg_owner=32753 >... > entry 800000004037e473 for l1e_owner=0, pg_owner=32753 > (XEN) mm.c:859:d0 Error getting mfn 4037f (pfn 5555555555555555) from L1 > entry 800000004037f473 for l1e_owner=0, pg_owner=32753 > > I boot Xen with ''dom0_mem=max:1024M iommu=0'' set in GRUB. The system > supports VT-d, but I am disabling it.Would seem like the kernel assigns I/O memory space to the device, based on the physical (not machine) memory map it gets (making the assumption that there is RAM at 0x40000000 aka 1Gb). Does the behavior change if you remove the dom0_mem= option? Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Fischer, Anna
2010-May-26 09:19 UTC
RE: [Xen-devel] Error getting mfn for PCI SR-IOV device?
> Subject: Re: [Xen-devel] Error getting mfn for PCI SR-IOV device? > > >>> On 25.05.10 at 15:58, "Fischer, Anna" <anna.fischer@hp.com> wrote: > > (XEN) mm.c:859:d0 Error getting mfn 40300 (pfn 5555555555555555) from > L1 > > entry 8000000040300473 for l1e_owner=0, pg_owner=32753 > >... > > entry 800000004037e473 for l1e_owner=0, pg_owner=32753 > > (XEN) mm.c:859:d0 Error getting mfn 4037f (pfn 5555555555555555) from > L1 > > entry 800000004037f473 for l1e_owner=0, pg_owner=32753 > > > > I boot Xen with ''dom0_mem=max:1024M iommu=0'' set in GRUB. The system > > supports VT-d, but I am disabling it. > > Would seem like the kernel assigns I/O memory space to the device, > based on the physical (not machine) memory map it gets (making the > assumption that there is RAM at 0x40000000 aka 1Gb). Does the > behavior change if you remove the dom0_mem= option?Yes, thanks, I was going to post to the list that the problem is solved, and the issue was that the PCI BARs were allocated > 1G while I was only making 1024MB visible to Dom0. Now it works. Thanks, Anna _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich
2010-May-26 09:49 UTC
RE: [Xen-devel] Error getting mfn for PCI SR-IOV device?
>>> On 26.05.10 at 11:19, "Fischer, Anna" <anna.fischer@hp.com> wrote: >> Subject: Re: [Xen-devel] Error getting mfn for PCI SR-IOV device? >> >> >>> On 25.05.10 at 15:58, "Fischer, Anna" <anna.fischer@hp.com> wrote: >> > (XEN) mm.c:859:d0 Error getting mfn 40300 (pfn 5555555555555555) from >> L1 >> > entry 8000000040300473 for l1e_owner=0, pg_owner=32753 >> >... >> > entry 800000004037e473 for l1e_owner=0, pg_owner=32753 >> > (XEN) mm.c:859:d0 Error getting mfn 4037f (pfn 5555555555555555) from >> L1 >> > entry 800000004037f473 for l1e_owner=0, pg_owner=32753 >> > >> > I boot Xen with ''dom0_mem=max:1024M iommu=0'' set in GRUB. The system >> > supports VT-d, but I am disabling it. >> >> Would seem like the kernel assigns I/O memory space to the device, >> based on the physical (not machine) memory map it gets (making the >> assumption that there is RAM at 0x40000000 aka 1Gb). Does the >> behavior change if you remove the dom0_mem= option? > > Yes, thanks, I was going to post to the list that the problem is solved, and > the issue was that the PCI BARs were allocated > 1G while I was only making > 1024MB visible to Dom0. Now it works.But it''s a kernel side bug nevertheless... Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Fischer, Anna
2010-May-26 11:15 UTC
RE: [Xen-devel] Error getting mfn for PCI SR-IOV device?
> Subject: RE: [Xen-devel] Error getting mfn for PCI SR-IOV device? > > >>> On 26.05.10 at 11:19, "Fischer, Anna" <anna.fischer@hp.com> wrote: > >> Subject: Re: [Xen-devel] Error getting mfn for PCI SR-IOV device? > >> > >> >>> On 25.05.10 at 15:58, "Fischer, Anna" <anna.fischer@hp.com> wrote: > >> > (XEN) mm.c:859:d0 Error getting mfn 40300 (pfn 5555555555555555) > from > >> L1 > >> > entry 8000000040300473 for l1e_owner=0, pg_owner=32753 > >> >... > >> > entry 800000004037e473 for l1e_owner=0, pg_owner=32753 > >> > (XEN) mm.c:859:d0 Error getting mfn 4037f (pfn 5555555555555555) > from > >> L1 > >> > entry 800000004037f473 for l1e_owner=0, pg_owner=32753 > >> > > >> > I boot Xen with ''dom0_mem=max:1024M iommu=0'' set in GRUB. The > system > >> > supports VT-d, but I am disabling it. > >> > >> Would seem like the kernel assigns I/O memory space to the device, > >> based on the physical (not machine) memory map it gets (making the > >> assumption that there is RAM at 0x40000000 aka 1Gb). Does the > >> behavior change if you remove the dom0_mem= option? > > > > Yes, thanks, I was going to post to the list that the problem is > solved, and > > the issue was that the PCI BARs were allocated > 1G while I was only > making > > 1024MB visible to Dom0. Now it works. > > But it''s a kernel side bug nevertheless...Yes, it should not allocate > 1G if Dom0 does not have access to that memory space. This runs the latest pv-ops kernel http://git.kernel.org/?p=linux/kernel/git/jeremy/xen.git;a=summary from the xen/master branch. It only happens for the SRIOV device when the kernel does PCI memory reassignment on boot. Anna _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel