Hey,
I just started noticing it today - with qemu-xen (tip is
commit b97307ecaad98360f41ea36cd9674ef810c4f8cf
xen_disk: mark ioreq as mapped before unmapping in error case)
when I try to pass in a PCI device at bootup it blows up with:
char device redirected to /dev/pts/2 (label serial0)
qemu: hardware error: xen: failed to populate ram at 40050000
CPU #0:
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000633
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 00000000 0000ffff
IDT= 00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000
DR6=ffff0ff0 DR7=00000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000
XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000
XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000
XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000
XMM07=00000000000000000000000000000000
CPU #1:
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000633
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=1
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00
...
-bash-4.1# xl dmesg | tail
[ 2788.038463] xen_pciback: vpci(d3) [2013-12-04 19:43:40] System
requested SeaBIOS
: 0000:01:00.1: (d3) [2013-12-04 19:43:40] CPU speed is 3093 MHz
assign to virtua(d3) [2013-12-04 19:43:40] Relocating guest memory for
lowmem MMIO space disabled
l slot 0
[ 2788.076396] device vif3.0 entered promiscuous mode
If I don''t have ''pci='' stanze in my guest config it
boots just fine.
-bash-4.1# more /vm.cfg
builder=''hvm''
disk = [ ''file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r'']
memory=1024
boot="d"
vcpus=2
serial="pty"
vnclisten="0.0.0.0"
name="latest"
vif = [ ''mac=00:0F:4B:00:00:68, bridge=switch'' ]
on_crash="preserve"
pci=["01:00.0"]
Any ideas?
This is with today''s Xen and 3.13-rc2. The device in question is
-bash-4.1# lspci -s 01:00.0 -v
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
Flags: fast devsel, IRQ 16
Memory at fbc20000 (32-bit, non-prefetchable) [disabled]
[size=128K]
Memory at fb800000 (32-bit, non-prefetchable) [disabled]
[size=4M]
I/O ports at e020 [disabled] [size=32]
Memory at fbc44000 (32-bit, non-prefetchable) [disabled]
[size=16K]
Expansion ROM at fb400000 [disabled] [size=4M]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable- Count=10 Masked-
Capabilities: [a0] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
Kernel driver in use: pciback
Kernel modules: igb
Oh and of course it boots with the traditional QEMU.
On Wed, Dec 04, 2013 at 02:51:48PM -0500, Konrad Rzeszutek Wilk wrote:> Hey, > > I just started noticing it today - with qemu-xen (tip is > commit b97307ecaad98360f41ea36cd9674ef810c4f8cf > xen_disk: mark ioreq as mapped before unmapping in error case) > > when I try to pass in a PCI device at bootup it blows up with: > > char device redirected to /dev/pts/2 (label serial0) > qemu: hardware error: xen: failed to populate ram at 40050000There is a good chance that the issue is in QEMU. By default, qemu-xen is not very verbose, so a #define XEN_PT_LOGGING_ENABLED can help identifying what is missing. The error is reported by xen_ram_alloc(), so it might help to enable trace for it. It''s easy: sh$ cat > /tmp/trace-to-enable << EOF xen_ram_alloc EOF then device_model_args_hvm = [ ''-trace'', ''events=/tmp/trace-to-enable'' ] There is more of those trace events in qemu.git/trace-events Does it works if you `xl pci-attach` after the guest has booted ?> ... > -bash-4.1# xl dmesg | tail > [ 2788.038463] xen_pciback: vpci(d3) [2013-12-04 19:43:40] System > requested SeaBIOS > : 0000:01:00.1: (d3) [2013-12-04 19:43:40] CPU speed is 3093 MHz > assign to virtua(d3) [2013-12-04 19:43:40] Relocating guest memory for > lowmem MMIO space disabled > l slot 0 > [ 2788.076396] device vif3.0 entered promiscuous mode > > If I don''t have ''pci='' stanze in my guest config it boots just fine. > > -bash-4.1# more /vm.cfg > builder=''hvm'' > disk = [ ''file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r''] > memory=1024 > boot="d" > vcpus=2 > serial="pty" > vnclisten="0.0.0.0" > name="latest" > vif = [ ''mac=00:0F:4B:00:00:68, bridge=switch'' ] > on_crash="preserve" > pci=["01:00.0"] > > Any ideas? > > This is with today''s Xen and 3.13-rc2. The device in question is > -bash-4.1# lspci -s 01:00.0 -v > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network > Connection (rev 01) > Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter > Flags: fast devsel, IRQ 16 > Memory at fbc20000 (32-bit, non-prefetchable) [disabled] > [size=128K] > Memory at fb800000 (32-bit, non-prefetchable) [disabled] > [size=4M] > I/O ports at e020 [disabled] [size=32] > Memory at fbc44000 (32-bit, non-prefetchable) [disabled] > [size=16K] > Expansion ROM at fb400000 [disabled] [size=4M] > Capabilities: [40] Power Management version 3 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ > Capabilities: [70] MSI-X: Enable- Count=10 Masked- > Capabilities: [a0] Express Endpoint, MSI 00 > Capabilities: [100] Advanced Error Reporting > Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac > Capabilities: [150] Alternative Routing-ID Interpretation (ARI) > Capabilities: [160] Single Root I/O Virtualization (SR-IOV) > Kernel driver in use: pciback > Kernel modules: igb > > > Oh and of course it boots with the traditional QEMU.-- Anthony PERARD
On Thu, Dec 05, 2013 at 12:16:32PM +0000, Anthony PERARD wrote:> On Wed, Dec 04, 2013 at 02:51:48PM -0500, Konrad Rzeszutek Wilk wrote: > > Hey, > > > > I just started noticing it today - with qemu-xen (tip is > > commit b97307ecaad98360f41ea36cd9674ef810c4f8cf > > xen_disk: mark ioreq as mapped before unmapping in error case) > > > > when I try to pass in a PCI device at bootup it blows up with: > > > > char device redirected to /dev/pts/2 (label serial0) > > qemu: hardware error: xen: failed to populate ram at 40050000 > > There is a good chance that the issue is in QEMU. > > By default, qemu-xen is not very verbose, so a > #define XEN_PT_LOGGING_ENABLED > can help identifying what is missing. > > The error is reported by xen_ram_alloc(), so it might help to enable > trace for it. It''s easy: > sh$ cat > /tmp/trace-to-enable << EOF > xen_ram_alloc > EOF > > then > device_model_args_hvm = [ ''-trace'', ''events=/tmp/trace-to-enable'' ] > > There is more of those trace events in qemu.git/trace-eventschar device redirected to /dev/pts/2 (label serial0) xen_ram_alloc requested: 0x3f800000, size 0x800000 xen_ram_alloc requested: 0x40000000, size 0x10000 xen_ram_alloc requested: 0x40010000, size 0x40000 [00:05.0] xen_pt_initfn: Assigning real physical device 01:00.0 to devfn 0x28 [00:05.0] xen_pt_register_regions: IO region 0 registered (size=0x20000lx base_addr=0xfbc20000lx type: 0) [00:05.0] xen_pt_register_regions: IO region 1 registered (size=0x400000lx base_addr=0xfb800000lx type: 0) [00:05.0] xen_pt_register_regions: IO region 2 registered (size=0x20lx base_addr=0xe020lx type: 0x1) [00:05.0] xen_pt_register_regions: IO region 3 registered (size=0x4000lx base_addr=0xfbc44000lx type: 0) xen_ram_alloc requested: 0x40050000, size 0x400000 qemu: hardware error: xen: failed to populate ram at 40050000> > Does it works if you `xl pci-attach` after the guest has booted ?-bash-4.1# xl pci-attach latest 01:00.0 libxl: error: libxl_qmp.c:448:qmp_next: Socket read error: Connection reset by peer -bash-4.1# head -16 /var/log/xen/qemu-dm-latest.log char device redirected to /dev/pts/3 (label serial0) xen_ram_alloc requested: 0x3f800000, size 0x800000 xen_ram_alloc requested: 0x40000000, size 0x10000 xen_ram_alloc requested: 0x40010000, size 0x40000 xen be: vkbd-0: initialise() failed xen be: vkbd-0: initialise() failed xen be: vkbd-0: initialise() failed [00:04.0] xen_pt_initfn: Assigning real physical device 01:00.0 to devfn 0x20 [00:04.0] xen_pt_register_regions: IO region 0 registered (size=0x20000lx base_addr=0xfbc20000lx type: 0) [00:04.0] xen_pt_register_regions: IO region 1 registered (size=0x400000lx base_addr=0xfb800000lx type: 0) [00:04.0] xen_pt_register_regions: IO region 2 registered (size=0x20lx base_addr=0xe020lx type: 0x1) [00:04.0] xen_pt_register_regions: IO region 3 registered (size=0x4000lx base_addr=0xfbc44000lx type: 0) xen_ram_alloc requested: 0x40010000, size 0x400000 qemu: hardware error: xen: failed to populate ram at 40010000 CPU #0: EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000633
On Fri, Dec 06, 2013 at 09:49:36AM -0500, Konrad Rzeszutek Wilk wrote:> On Thu, Dec 05, 2013 at 12:16:32PM +0000, Anthony PERARD wrote: > > On Wed, Dec 04, 2013 at 02:51:48PM -0500, Konrad Rzeszutek Wilk wrote: > > > Hey, > > > > > > I just started noticing it today - with qemu-xen (tip is > > > commit b97307ecaad98360f41ea36cd9674ef810c4f8cf > > > xen_disk: mark ioreq as mapped before unmapping in error case) > > > > > > when I try to pass in a PCI device at bootup it blows up with: > > > > > > char device redirected to /dev/pts/2 (label serial0) > > > qemu: hardware error: xen: failed to populate ram at 40050000 > > > > There is a good chance that the issue is in QEMU. > > > > By default, qemu-xen is not very verbose, so a > > #define XEN_PT_LOGGING_ENABLED > > can help identifying what is missing. > > > > The error is reported by xen_ram_alloc(), so it might help to enable > > trace for it. It''s easy: > > sh$ cat > /tmp/trace-to-enable << EOF > > xen_ram_alloc > > EOF > > > > then > > device_model_args_hvm = [ ''-trace'', ''events=/tmp/trace-to-enable'' ] > > > > There is more of those trace events in qemu.git/trace-events > > char device redirected to /dev/pts/2 (label serial0) > xen_ram_alloc requested: 0x3f800000, size 0x800000 > xen_ram_alloc requested: 0x40000000, size 0x10000 > xen_ram_alloc requested: 0x40010000, size 0x40000 > [00:05.0] xen_pt_initfn: Assigning real physical device 01:00.0 to devfn > 0x28 > [00:05.0] xen_pt_register_regions: IO region 0 registered > (size=0x20000lx base_addr=0xfbc20000lx type: 0) > [00:05.0] xen_pt_register_regions: IO region 1 registered > (size=0x400000lx base_addr=0xfb800000lx type: 0) > [00:05.0] xen_pt_register_regions: IO region 2 registered (size=0x20lx > base_addr=0xe020lx type: 0x1) > [00:05.0] xen_pt_register_regions: IO region 3 registered (size=0x4000lx > base_addr=0xfbc44000lx type: 0) > xen_ram_alloc requested: 0x40050000, size 0x400000 > qemu: hardware error: xen: failed to populate ram at 40050000Those Xen report something like: (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 > 131328 (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46 memflags=0 (62 of 64) ? (I tryied to reproduce the issue by simply add many emulated e1000 in QEMU :) ) -- Anthony PERARD
On Fri, Dec 06, 2013 at 03:35:04PM +0000, Anthony PERARD wrote:> On Fri, Dec 06, 2013 at 09:49:36AM -0500, Konrad Rzeszutek Wilk wrote: > > On Thu, Dec 05, 2013 at 12:16:32PM +0000, Anthony PERARD wrote: > > > On Wed, Dec 04, 2013 at 02:51:48PM -0500, Konrad Rzeszutek Wilk wrote: > > > > Hey, > > > > > > > > I just started noticing it today - with qemu-xen (tip is > > > > commit b97307ecaad98360f41ea36cd9674ef810c4f8cf > > > > xen_disk: mark ioreq as mapped before unmapping in error case) > > > > > > > > when I try to pass in a PCI device at bootup it blows up with: > > > > > > > > char device redirected to /dev/pts/2 (label serial0) > > > > qemu: hardware error: xen: failed to populate ram at 40050000 > > > > > > There is a good chance that the issue is in QEMU. > > > > > > By default, qemu-xen is not very verbose, so a > > > #define XEN_PT_LOGGING_ENABLED > > > can help identifying what is missing. > > > > > > The error is reported by xen_ram_alloc(), so it might help to enable > > > trace for it. It''s easy: > > > sh$ cat > /tmp/trace-to-enable << EOF > > > xen_ram_alloc > > > EOF > > > > > > then > > > device_model_args_hvm = [ ''-trace'', ''events=/tmp/trace-to-enable'' ] > > > > > > There is more of those trace events in qemu.git/trace-events > > > > char device redirected to /dev/pts/2 (label serial0) > > xen_ram_alloc requested: 0x3f800000, size 0x800000 > > xen_ram_alloc requested: 0x40000000, size 0x10000 > > xen_ram_alloc requested: 0x40010000, size 0x40000 > > [00:05.0] xen_pt_initfn: Assigning real physical device 01:00.0 to devfn > > 0x28 > > [00:05.0] xen_pt_register_regions: IO region 0 registered > > (size=0x20000lx base_addr=0xfbc20000lx type: 0) > > [00:05.0] xen_pt_register_regions: IO region 1 registered > > (size=0x400000lx base_addr=0xfb800000lx type: 0) > > [00:05.0] xen_pt_register_regions: IO region 2 registered (size=0x20lx > > base_addr=0xe020lx type: 0x1) > > [00:05.0] xen_pt_register_regions: IO region 3 registered (size=0x4000lx > > base_addr=0xfbc44000lx type: 0) > > xen_ram_alloc requested: 0x40050000, size 0x400000 > > qemu: hardware error: xen: failed to populate ram at 40050000 > > Those Xen report something like: > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 > > 131328 > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46 > memflags=0 (62 of 64) > > ? > > (I tryied to reproduce the issue by simply add many emulated e1000 in > QEMU :) ) >FWIW this probably comes from XENMEM_populate_physmap. Simply speaking your domain is exceeding memory limit... Wei.> -- > Anthony PERARD > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Fri, Dec 06, 2013 at 04:00:18PM +0000, Wei Liu wrote: [...]> > Those Xen report something like: > > (XEN) page_alloc.c:1460:d0 Over-allocation for domain 46: 131329 > > > 131328 > > (XEN) memory.c:132:d0 Could not allocate order=0 extent: id=46 > > memflags=0 (62 of 64) > > > > ? > > > > (I tryied to reproduce the issue by simply add many emulated e1000 in > > QEMU :) ) > > > > FWIW this probably comes from XENMEM_populate_physmap. Simply speaking > your domain is exceeding memory limit... >And, if this comes from hvmloader it is not fatal. It will use lower memory to fill in the holes. Wei.
Reasonably Related Threads
- Linux Stubdom Problem
- [xen-unstable test] 22184: regressions - trouble: broken/fail/pass
- hardware error 0x80000021 on Ubuntu 12.10, need advice
- LSI SAS2008 Option Rom Failure
- RESEND [Xen-unstable][Qemu-xen] HVM Guest reading of Expansion ROM from passthroughed PCI device returns data from emulated VGA rom