David F Barrera
2005-Sep-29 16:12 UTC
[Xen-devel] (XEN) Domain 2 (vcpu#0) crashed on cpu#3: (Resend)
Trace produced while running xm-test on IBM xSeries 335, 4GB RAM, PAE
enabled SLES 9 SP2 box.
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=267
changeset: 7124:805ee053e61f
Previously, same workload was causing the Ouch! messages.
Also, time has virtually stopped on the machine--it is moving forward
but very slowly, like 1 second for every 2 or 3 real minutes.
x335b:~ # xm dmesg
__ __ _____ ___ _ _
\ \/ /___ _ __ |___ / / _ \ __| | _____ _____| |
\ // _ \ ''_ \ |_ \| | | |__ / _` |/ _ \ \ / / _ \ |
/ \ __/ | | | ___) | |_| |__| (_| | __/\ V / __/ |
/_/\_\___|_| |_| |____(_)___/ \__,_|\___| \_/ \___|_|
http://www.cl.cam.ac.uk/netos/xen
University of Cambridge Computer Laboratory
Xen version 3.0-devel (root@ltc.austin.ibm.com) (gcc version 3.3.3
(SuSE Linux) ) Thu Sep 29 08:17:15 CDT 2005
Latest ChangeSet: Thu Sep 29 07:05:43 2005 +0100 805ee053e61f
(XEN) Physical RAM map:
(XEN) 0000000000000000 - 000000000009d400 (usable)
(XEN) 000000000009d400 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000f7fec140 (usable)
(XEN) 00000000f7fec140 - 00000000f7ff0000 (ACPI data)
(XEN) 00000000f7ff0000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 0000000108000000 (usable)
(XEN) System RAM: 4095MB (4193828kB)
(XEN) Xen heap: 10MB (10496kB)
(XEN) PAE enabled, limit: 16 GB
(XEN) found SMP MP-table at 0009d540
(XEN) DMI 2.3 present.
(XEN) Using APIC driver default
(XEN) ACPI: RSDP (v000 IBM ) @
0x000fdfc0
(XEN) ACPI: RSDT (v001 IBM SERONYXP 0x00001000 IBM 0x45444f43) @
0xf7feff80
(XEN) ACPI: FADT (v001 IBM SERONYXP 0x00001000 IBM 0x45444f43) @
0xf7feff00
(XEN) ACPI: MADT (v001 IBM SERONYXP 0x00001000 IBM 0x45444f43) @
0xf7fefe40
(XEN) ACPI: ASF! (v016 IBM SERONYXP 0x00000001 IBM 0x45444f43) @
0xf7fefd80
(XEN) ACPI: DSDT (v001 IBM SERTURQU 0x00001000 MSFT 0x0100000b) @
0x00000000
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 15:2 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x06] enabled)
(XEN) Processor #6 15:2 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 15:2 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled)
(XEN) Processor #7 15:2 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] dfl dfl lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x06] dfl dfl lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x07] dfl dfl lint[0x1])
(XEN) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
(XEN) ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
(XEN) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
(XEN) ACPI: IOAPIC (id[0x0c] address[0xfec02000] gsi_base[32])
(XEN) IOAPIC[2]: apic_id 12, version 17, address 0xfec02000, GSI 32-47
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 3 I/O APICs
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Initializing CPU#0
(XEN) Detected 3189.565 MHz processor.
(XEN) Using scheduler: Simple EDF Scheduler (sedf)
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU0: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Booting processor 1/1 eip 90000
(XEN) Initializing CPU#1
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU1: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Booting processor 2/6 eip 90000
(XEN) Initializing CPU#2
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 3
(XEN) CPU2: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Booting processor 3/7 eip 90000
(XEN) Initializing CPU#3
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 3
(XEN) CPU3: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Total of 4 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
(XEN) checking TSC synchronization across 4 CPUs: passed.
(XEN) Platform timer is 1.193MHz PIT
(XEN) Brought up 4 CPUs
(XEN) mtrr: v2.0 (20020519)
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen-ELF header found:
''GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE0xC0000000,PAE=yes,LOADER=generic''
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000003800000->0000000004000000 (125952 pages
to be all ocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: c0100000->c06396e4
(XEN) Init. ramdisk: c063a000->c063a000
(XEN) Phys-Mach map: c063a000->c06b7000
(XEN) Start info: c06b7000->c06b8000
(XEN) Page tables: c06b8000->c06c1000
(XEN) Boot stack: c06c1000->c06c2000
(XEN) TOTAL: c0000000->c0800000
(XEN) ENTRY ADDRESS: c0100000
(XEN) Scrubbing Free
RAM: ...........................................done.
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch
input to Xen ).
(XEN) microcode: CPU1 updated from revision 0x11 to 0x29, date 08112004
(XEN) microcode: CPU2 updated from revision 0x11 to 0x29, date 08112004
(XEN) microcode: CPU3 updated from revision 0x11 to 0x29, date 08112004
(XEN) microcode: CPU0 updated from revision 0x11 to 0x29, date 08112004
(XEN) mtrr: type mismatch for fd000000,800000 old: write-back new:
write-combini ng
(XEN) mtrr: type mismatch for fd000000,800000 old: write-back new:
write-combini ng
(XEN) Domain 2 (vcpu#0) crashed on cpu#3:
(XEN) CPU: 3
(XEN) EIP: e019:[<c0115a26>]
(XEN) EFLAGS: 00000246 CONTEXT: guest
(XEN) eax: 00000e90 ebx: 00000000 ecx: 00000000 edx: 00000000
(XEN) esi: 00000000 edi: 00000000 ebp: 00000e90 esp: c0355e8c
(XEN) cr0: 8005003b cr3: f7f0d000
(XEN) ds: e021 es: e021 fs: 0000 gs: e021 ss: e021 cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN) c0000e94 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000
(XEN) c0000000 c0361885 c27d2000 00001000 00000000 00000000 c022b798
c0384580
(XEN) 00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74
(XEN) c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000
(XEN) 00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035
(XEN) c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308
(XEN) 0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4
(XEN) c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000
(XEN) ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000
(XEN) 00000000 00000000 c03838c0 0702080b c010007a
(XEN) (file=memory.c, line=57) Could not allocate order=0 extent: id=5
flags=0 ( 36782 of 62338)
(XEN) Domain 6 (vcpu#0) crashed on cpu#3:
(XEN) CPU: 3
(XEN) EIP: e019:[<c0115a26>]
(XEN) EFLAGS: 00000246 CONTEXT: guest
(XEN) eax: 00000898 ebx: 00000000 ecx: 00000000 edx: 00000000
(XEN) esi: 00000000 edi: 00000000 ebp: 00000898 esp: c0355e8c
(XEN) cr0: 8005003b cr3: f6a9c000
(XEN) ds: e021 es: e021 fs: 0000 gs: e021 ss: e021 cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN) c000089c 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000
(XEN) c0000000 c0361885 c3513000 00001000 00000000 00000000 c022b798
c0384580
(XEN) 00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74
(XEN) c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000
(XEN) 00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035
(XEN) c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308
(XEN) 0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4
(XEN) c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000
(XEN) ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000
(XEN) 00000000 00000000 c03838c0 0702080b c010007a
(XEN) Domain 15 (vcpu#0) crashed on cpu#3:
(XEN) CPU: 3
(XEN) EIP: e019:[<c0115a26>]
(XEN) EFLAGS: 00000246 CONTEXT: guest
(XEN) eax: 00000480 ebx: 00000000 ecx: 00000000 edx: 00000000
(XEN) esi: 00000000 edi: 00000000 ebp: 00000480 esp: c0355e8c
(XEN) cr0: 8005003b cr3: f6ac8000
(XEN) ds: e021 es: e021 fs: 0000 gs: e021 ss: e021 cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN) c0000484 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000
(XEN) c0000000 c0361885 c3490000 00001000 00000000 00000000 c022b798
c0384580
(XEN) 00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74
(XEN) c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000
(XEN) 00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035
(XEN) c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308
(XEN) 0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4
(XEN) c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000
(XEN) ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000
(XEN) 00000000 00000000 c03838c0 0702080b c010007a
(XEN) Domain 25 (vcpu#0) crashed on cpu#3:
(XEN) CPU: 3
(XEN) EIP: e019:[<c0115a26>]
(XEN) EFLAGS: 00000246 CONTEXT: guest
(XEN) eax: 00000d28 ebx: 00000000 ecx: 00000000 edx: 00000000
(XEN) esi: 00000000 edi: 00000000 ebp: 00000d28 esp: c0355e8c
(XEN) cr0: 8005003b cr3: f6af3000
(XEN) ds: e021 es: e021 fs: 0000 gs: e021 ss: e021 cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN) c0000d2c 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000
(XEN) c0000000 c0361885 c1fa5000 00001000 00000000 00000000 c022b798
c0384580
(XEN) 00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74
(XEN) c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000
(XEN) 00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035
(XEN) c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308
(XEN) 0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4
(XEN) c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000
(XEN) ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000
(XEN) 00000000 00000000 c03838c0 0702080b c010007a
--
Regards,
David F Barrera
Linux Technology Center
Systems and Technology Group, IBM
"The wisest men follow their own direction. "
Euripides
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel