With changeset 8728, I get the following for x86_64, smp on.
passing maxcpus=1 to dom0 makes dom0 boot fine
x86_32 w/ smp boots fine
hardware:
two physical amd opteron 248''s
Booting ''xen''
root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
kernel /xen.gz com1=115200,8n1
[Multiboot-elf, <0x100000:0x80408:0x31bf8>, shtab=0x1b2078,
entry=0x100000]
module /vmlinuz-2.6-xen0 root=/dev/sda4 ro conswitch=x console=tty0
console=tt
yS1,115200
[Multiboot-module @ 0x1b3000, 0x7956a8 bytes]
savedefault
boot
__ __ _____ ___ ___
\ \/ /___ _ __ |___ / / _ \ / _ \
\ // _ \ ''_ \ |_ \| | | | | | |
/ \ __/ | | | ___) | |_| | |_| |
/_/\_\___|_| |_| |____(_)___(_)___/
http://www.cl.cam.ac.uk/netos/xen
University of Cambridge Computer Laboratory
Xen version 3.0.0 (root@ltc.austin.ibm.com) (gcc version 3.4.2 20041017
(Red H6 Latest ChangeSet: Wed Feb 1 11:06:16 2006 +0000
8728:d609de73b9fa
(XEN) Physical RAM map:
(XEN) 0000000000000000 - 000000000009dc00 (usable)
(XEN) 000000000009dc00 - 00000000000a0000 (reserved)
(XEN) 00000000000d0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000dff60000 (usable)
(XEN) 00000000dff60000 - 00000000dff72000 (ACPI data)
(XEN) 00000000dff72000 - 00000000dff80000 (ACPI NVS)
(XEN) 00000000dff80000 - 00000000e0000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec00400 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fff80000 - 0000000100000000 (reserved)
(XEN) System RAM: 3582MB (3668980kB)
(XEN) Xen heap: 14MB (14536kB)
(XEN) Using scheduler: Simple EDF Scheduler (sedf)
(XEN) found SMP MP-table at 000f7140
(XEN) DMI present.
(XEN) Using APIC driver default
(XEN) ACPI: RSDP (v002 PTLTD ) @
0x00000000000f0(XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP
0x00000000) @ 0x0000000f(XEN) ACPI: FADT (v003 AMD HAMMER
0x06040000 PTEC 0x000f4240) @ 0x00000000f(XEN) ACPI: SRAT (v001 AMD
HAMMER 0x06040000 AMD 0x00000001) @ 0x000000003(XEN) ACPI: MADT (v001
PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x0000000b(XEN) ACPI:
ASF! (v016 MBI CETP 0x06040000 PTL 0x00000001) @
0x000000009(XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT
0x0100000e) @ 0x000000000(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 15:5 APIC version 16
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 15:5 APIC version 16
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x03] address[0xf9220000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 3, version 17, address 0xf9220000, GSI 24-27
(XEN) ACPI: IOAPIC (id[0x04] address[0xf9230000] gsi_base[28])
(XEN) IOAPIC[2]: apic_id 4, version 17, address 0xf9230000, GSI 28-31
(XEN) ACPI: IOAPIC (id[0x05] address[0xf9200000] gsi_base[32])
(XEN) IOAPIC[3]: apic_id 5, version 17, address 0xf9200000, GSI 32-35
(XEN) ACPI: IOAPIC (id[0x06] address[0xf9210000] gsi_base[36])
(XEN) IOAPIC[4]: apic_id 6, version 17, address 0xf9210000, GSI 36-39
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 5 I/O APICs
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Initializing CPU#0
(XEN) Detected 2193.790 MHz processor.
(XEN) CPU0: AMD Flush Filter enabled
(XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
(XEN) CPU: L2 Cache: 1024K (64 bytes/line)
(XEN) CPU0: AMD Opteron(tm) Processor 248 stepping 0a
(XEN) Booting processor 1/1 eip 90000
(XEN) Initializing CPU#1
(XEN) CPU1: AMD Flush Filter enabled
(XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
(XEN) CPU: L2 Cache: 1024K (64 bytes/line)
(XEN) CPU1: AMD Opteron(tm) Processor 248 stepping 08
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 1.193MHz PIT
(XEN) Brought up 2 CPUs
(XEN) mtrr: v2.0 (20020519)
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen-ELF header found:
''GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=xen-3.0,VIRT_''(XEN)
PHYSICAL
MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000007000000->0000000008000000 (865368 pages
to be al)(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80100000->ffffffff80689d18
(XEN) Init. ramdisk: ffffffff8068a000->ffffffff8068a000
(XEN) Phys-Mach map: ffffffff8068a000->ffffffff80d2c2c0
(XEN) Start info: ffffffff80d2d000->ffffffff80d2e000
(XEN) Page tables: ffffffff80d2e000->ffffffff80d39000
(XEN) Boot stack: ffffffff80d39000->ffffffff80d3a000
(XEN) TOTAL: ffffffff80000000->ffffffff81000000
(XEN) ENTRY ADDRESS: ffffffff80100000
(XEN) Scrubbing Free RAM: ....................................done.
(XEN) Xen trace buffers: disabled
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch
input to Xe.Linux version 2.6.14-xen0
(root@quadmonster.ltc.austin.ibm.com) (gcc version 3.6kernel direct
mapping tables upto d4c58000 @ d3b000-13e7000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xf9220000] gsi_base[24])
IOAPIC[1]: apic_id 3, version 17, address 0xf9220000, GSI 24-27
ACPI: IOAPIC (id[0x04] address[0xf9230000] gsi_base[28])
IOAPIC[2]: apic_id 4, version 17, address 0xf9230000, GSI 28-31
ACPI: IOAPIC (id[0x05] address[0xf9200000] gsi_base[32])
IOAPIC[3]: apic_id 5, version 17, address 0xf9200000, GSI 32-35
ACPI: IOAPIC (id[0x06] address[0xf9210000] gsi_base[36])
IOAPIC[4]: apic_id 6, version 17, address 0xf9210000, GSI 36-39
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
Setting APIC routing to xen
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at e2000000 (gap: e0000000:1ec00000)
Built 1 zonelists
Kernel command line: root=/dev/sda4 ro conswitch=x console=tty0
console=ttyS1,10Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
Xen reported: 2193.790 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Software IO TLB enabled:
Aperture: 64 megabytes
Bus range: 0x0000000040000000 - 0x0000000044000000
Kernel range: 0xffff880004f78000 - 0xffff880008f78000
Memory: 3337340k/3486048k available (3499k kernel code, 147968k
reserved, 1310k)Calibrating delay using timer specific routine.. 5485.75
BogoMIPS (lpj=10971515)Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
softlockup thread 0 started up.
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at kernel/softlockup.c:110
invalid operand: 0000 [1] SMP
CPU 0
Modules linked in:
Pid: 1, comm: swapper Not tainted 2.6.14-xen0 #1
RIP: e030:[<ffffffff8015500c>] <ffffffff8015500c>{cpu_callback+76}
RSP: e02b:ffff88000068be98 EFLAGS: 00010286
RAX: ffffffff806890f8 RBX: 0000000000000001 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000003 RDI: ffffffff80507210
RBP: ffff88000068beb8 R08: 00000000fffffffb R09: 0000000000000000
R10: ffff88000068bcc8 R11: 0000000000000000 R12: 0000000000000080
R13: ffffffff806890f8 R14: 0000000000000001 R15: 0000ffffffff8010
FS: 0000000000000000(0000) GS:ffffffff80623800(0000)
knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000
Process swapper (pid: 1, threadinfo ffff88000068a000, task
ffff88000068d4b0)
Stack: ffff88000068bea8 ffffffff80507210 0000000000000001
0000000000000003
ffff88000068bee8 ffffffff80142248 ffffffff80506c00
0000000000000001
0000000000000008 ffffffff80506c00
Call Trace:<ffffffff80142248>{notifier_call_chain+40}
<ffffffff8014de57>{cpu_up} <ffffffff8010e163>{init+275}
<ffffffff80111a9e>{child_rip+8}
<ffffffff8010e050>{init+0} <ffffffff80111a96>{child_rip+0}
Code: 0f 0b 68 ae a2 48 80 c2 6e 00 31 c0 89 d9 48 c7 c2 c2 a2 48
RIP <ffffffff8015500c>{cpu_callback+76} RSP <ffff88000068be98>
<0>Kernel panic - not syncing: Attempted to kill init!
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
I would attribute this to the fact that cpu_possible_mask still appears to net get set early enough for setup_per_cpu_areas() to work properly. A patch (that made things work for us, while it might not be the ideal solution) was sent to the merge list mid December: http://lists.xensource.com/archives/html/xen-merge/2005-12/msg00009.html. Jan>>> Ryan Grimm <grimm@us.ibm.com> 02.02.06 01:18:30 >>>With changeset 8728, I get the following for x86_64, smp on. passing maxcpus=1 to dom0 makes dom0 boot fine x86_32 w/ smp boots fine hardware: two physical amd opteron 248''s Booting ''xen'' root (hd0,0) Filesystem type is ext2fs, partition type 0x83 kernel /xen.gz com1=115200,8n1 [Multiboot-elf, <0x100000:0x80408:0x31bf8>, shtab=0x1b2078, entry=0x100000] module /vmlinuz-2.6-xen0 root=/dev/sda4 ro conswitch=x console=tty0 console=tt yS1,115200 [Multiboot-module @ 0x1b3000, 0x7956a8 bytes] savedefault boot __ __ _____ ___ ___ \ \/ /___ _ __ |___ / / _ \ / _ \ \ // _ \ ''_ \ |_ \| | | | | | | / \ __/ | | | ___) | |_| | |_| | /_/\_\___|_| |_| |____(_)___(_)___/ http://www.cl.cam.ac.uk/netos/xen University of Cambridge Computer Laboratory Xen version 3.0.0 (root@ltc.austin.ibm.com) (gcc version 3.4.2 20041017 (Red H6 Latest ChangeSet: Wed Feb 1 11:06:16 2006 +0000 8728:d609de73b9fa (XEN) Physical RAM map: (XEN) 0000000000000000 - 000000000009dc00 (usable) (XEN) 000000000009dc00 - 00000000000a0000 (reserved) (XEN) 00000000000d0000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000dff60000 (usable) (XEN) 00000000dff60000 - 00000000dff72000 (ACPI data) (XEN) 00000000dff72000 - 00000000dff80000 (ACPI NVS) (XEN) 00000000dff80000 - 00000000e0000000 (reserved) (XEN) 00000000fec00000 - 00000000fec00400 (reserved) (XEN) 00000000fee00000 - 00000000fee01000 (reserved) (XEN) 00000000fff80000 - 0000000100000000 (reserved) (XEN) System RAM: 3582MB (3668980kB) (XEN) Xen heap: 14MB (14536kB) (XEN) Using scheduler: Simple EDF Scheduler (sedf) (XEN) found SMP MP-table at 000f7140 (XEN) DMI present. (XEN) Using APIC driver default (XEN) ACPI: RSDP (v002 PTLTD ) @ 0x00000000000f0(XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP 0x00000000) @ 0x0000000f(XEN) ACPI: FADT (v003 AMD HAMMER 0x06040000 PTEC 0x000f4240) @ 0x00000000f(XEN) ACPI: SRAT (v001 AMD HAMMER 0x06040000 AMD 0x00000001) @ 0x000000003(XEN) ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x0000000b(XEN) ACPI: ASF! (v016 MBI CETP 0x06040000 PTL 0x00000001) @ 0x000000009(XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT 0x0100000e) @ 0x000000000(XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) (XEN) Processor #0 15:5 APIC version 16 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) (XEN) Processor #1 15:5 APIC version 16 (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23 (XEN) ACPI: IOAPIC (id[0x03] address[0xf9220000] gsi_base[24]) (XEN) IOAPIC[1]: apic_id 3, version 17, address 0xf9220000, GSI 24-27 (XEN) ACPI: IOAPIC (id[0x04] address[0xf9230000] gsi_base[28]) (XEN) IOAPIC[2]: apic_id 4, version 17, address 0xf9230000, GSI 28-31 (XEN) ACPI: IOAPIC (id[0x05] address[0xf9200000] gsi_base[32]) (XEN) IOAPIC[3]: apic_id 5, version 17, address 0xf9200000, GSI 32-35 (XEN) ACPI: IOAPIC (id[0x06] address[0xf9210000] gsi_base[36]) (XEN) IOAPIC[4]: apic_id 6, version 17, address 0xf9210000, GSI 36-39 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) Enabling APIC mode: Flat. Using 5 I/O APICs (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Initializing CPU#0 (XEN) Detected 2193.790 MHz processor. (XEN) CPU0: AMD Flush Filter enabled (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) (XEN) CPU: L2 Cache: 1024K (64 bytes/line) (XEN) CPU0: AMD Opteron(tm) Processor 248 stepping 0a (XEN) Booting processor 1/1 eip 90000 (XEN) Initializing CPU#1 (XEN) CPU1: AMD Flush Filter enabled (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) (XEN) CPU: L2 Cache: 1024K (64 bytes/line) (XEN) CPU1: AMD Opteron(tm) Processor 248 stepping 08 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 1.193MHz PIT (XEN) Brought up 2 CPUs (XEN) mtrr: v2.0 (20020519) (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen-ELF header found: ''GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=xen-3.0,VIRT_''(XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000007000000->0000000008000000 (865368 pages to be al)(XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff80100000->ffffffff80689d18 (XEN) Init. ramdisk: ffffffff8068a000->ffffffff8068a000 (XEN) Phys-Mach map: ffffffff8068a000->ffffffff80d2c2c0 (XEN) Start info: ffffffff80d2d000->ffffffff80d2e000 (XEN) Page tables: ffffffff80d2e000->ffffffff80d39000 (XEN) Boot stack: ffffffff80d39000->ffffffff80d3a000 (XEN) TOTAL: ffffffff80000000->ffffffff81000000 (XEN) ENTRY ADDRESS: ffffffff80100000 (XEN) Scrubbing Free RAM: ....................................done. (XEN) Xen trace buffers: disabled (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xe.Linux version 2.6.14-xen0 (root@quadmonster.ltc.austin.ibm.com) (gcc version 3.6kernel direct mapping tables upto d4c58000 @ d3b000-13e7000 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23 ACPI: IOAPIC (id[0x03] address[0xf9220000] gsi_base[24]) IOAPIC[1]: apic_id 3, version 17, address 0xf9220000, GSI 24-27 ACPI: IOAPIC (id[0x04] address[0xf9230000] gsi_base[28]) IOAPIC[2]: apic_id 4, version 17, address 0xf9230000, GSI 28-31 ACPI: IOAPIC (id[0x05] address[0xf9200000] gsi_base[32]) IOAPIC[3]: apic_id 5, version 17, address 0xf9200000, GSI 32-35 ACPI: IOAPIC (id[0x06] address[0xf9210000] gsi_base[36]) IOAPIC[4]: apic_id 6, version 17, address 0xf9210000, GSI 36-39 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) Setting APIC routing to xen Using ACPI (MADT) for SMP configuration information Allocating PCI resources starting at e2000000 (gap: e0000000:1ec00000) Built 1 zonelists Kernel command line: root=/dev/sda4 ro conswitch=x console=tty0 console=ttyS1,10Initializing CPU#0 PID hash table entries: 4096 (order: 12, 131072 bytes) Xen reported: 2193.790 MHz processor. Console: colour VGA+ 80x25 Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes) Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes) Software IO TLB enabled: Aperture: 64 megabytes Bus range: 0x0000000040000000 - 0x0000000044000000 Kernel range: 0xffff880004f78000 - 0xffff880008f78000 Memory: 3337340k/3486048k available (3499k kernel code, 147968k reserved, 1310k)Calibrating delay using timer specific routine.. 5485.75 BogoMIPS (lpj=10971515)Mount-cache hash table entries: 256 CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) CPU: L2 Cache: 1024K (64 bytes/line) softlockup thread 0 started up. ----------- [cut here ] --------- [please bite here ] --------- Kernel BUG at kernel/softlockup.c:110 invalid operand: 0000 [1] SMP CPU 0 Modules linked in: Pid: 1, comm: swapper Not tainted 2.6.14-xen0 #1 RIP: e030:[<ffffffff8015500c>] <ffffffff8015500c>{cpu_callback+76} RSP: e02b:ffff88000068be98 EFLAGS: 00010286 RAX: ffffffff806890f8 RBX: 0000000000000001 RCX: 0000000000000001 RDX: 0000000000000001 RSI: 0000000000000003 RDI: ffffffff80507210 RBP: ffff88000068beb8 R08: 00000000fffffffb R09: 0000000000000000 R10: ffff88000068bcc8 R11: 0000000000000000 R12: 0000000000000080 R13: ffffffff806890f8 R14: 0000000000000001 R15: 0000ffffffff8010 FS: 0000000000000000(0000) GS:ffffffff80623800(0000) knlGS:0000000000000000 CS: e033 DS: 0000 ES: 0000 Process swapper (pid: 1, threadinfo ffff88000068a000, task ffff88000068d4b0) Stack: ffff88000068bea8 ffffffff80507210 0000000000000001 0000000000000003 ffff88000068bee8 ffffffff80142248 ffffffff80506c00 0000000000000001 0000000000000008 ffffffff80506c00 Call Trace:<ffffffff80142248>{notifier_call_chain+40} <ffffffff8014de57>{cpu_up} <ffffffff8010e163>{init+275} <ffffffff80111a9e>{child_rip+8} <ffffffff8010e050>{init+0} <ffffffff80111a96>{child_rip+0} Code: 0f 0b 68 ae a2 48 80 c2 6e 00 31 c0 89 d9 48 c7 c2 c2 a2 48 RIP <ffffffff8015500c>{cpu_callback+76} RSP <ffff88000068be98> <0>Kernel panic - not syncing: Attempted to kill init! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi,
I tried the patch you pointed out but still get the same result. I
turned off ''detect_soft_lockups'', and get the message below.
The same issue is seen in this post:
http://lists.xensource.com/archives/html/xen-devel/2006-02/msg00024.html
The possible and present maps do seem to be used differently than
by the linux kernel when booting SMP. Booting aside, this may have cpu
hotplug implications as well. Is anyone working on cleaning up the use
of the cpu_possible_map and cpu_present_map?
Thanks,
Ryan
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at drivers/xen/core/smpboot.c:99
invalid operand: 0000 [1] SMP
CPU 0
Modules linked in:
Pid: 1, comm: swapper Not tainted 2.6.15-xenU #4
RIP: e030:[<ffffffff8023dc0a>]
<ffffffff8023dc0a>{xen_smp_intr_init+122}
RSP: e02b:ffff88000041bea8 EFLAGS: 00010286
RAX: 00000000fffffff0 RBX: 0000000000000001 RCX: ffffffff803dd6a8
RDX: 00000000fffffff0 RSI: 0000000000000101 RDI: ffffffff8037df60
RBP: ffff88000041bec8 R08: 0000000000000000 R09: 0000000000000000
R10: ffff88000041bdc8 R11: 0000000000000000 R12: 0000000000000080
R13: 0000000000000001 R14: 000000000000000f R15: 0000ffffffff8010
FS: 00002aaaaadf96d0(0000) GS:ffffffff803a5800(0000)
knlGS:ffffffff803a5800
CS: e033 DS: 0000 ES: 0000
Process swapper (pid: 1, threadinfo ffff88000041a000, task
ffff8800004194e0)
Stack: 0000000000000001 0000000000000008 0000000000000001
00000000ffffffff
ffff88000041bee8 ffffffff8023df2a 0000000000000008
0000000000000001
ffff88000041bf18 ffffffff80149420
Call Trace:<ffffffff8023df2a>{__cpu_up+16}
<ffffffff80149420>{cpu_up+118}
<ffffffff8010e158>{init+272} <ffffffff8011188a>{child_rip+8}
<ffffffff8010e048>{init+0} <ffffffff80111882>{child_rip+0}
Code: 0f 0b 68 00 3e 2e 80 c2 63 00 49 8d 9e 20 73 38 80 44 89 ea
RIP <ffffffff8023dc0a>{xen_smp_intr_init+122} RSP <ffff88000041bea8>
<0>Kernel panic - not syncing: Attempted to kill init!
On Thu, Feb 02, 2006 at 09:01:53AM +0100, Jan Beulich
wrote:> I would attribute this to the fact that cpu_possible_mask still appears to
net get set early enough for
> setup_per_cpu_areas() to work properly. A patch (that made things work for
us, while it might not be the ideal solution)
> was sent to the merge list mid December:
http://lists.xensource.com/archives/html/xen-merge/2005-12/msg00009.html. Jan
>
> >>> Ryan Grimm <grimm@us.ibm.com> 02.02.06 01:18:30
>>>
> With changeset 8728, I get the following for x86_64, smp on.
>
> passing maxcpus=1 to dom0 makes dom0 boot fine
> x86_32 w/ smp boots fine
>
> hardware:
> two physical amd opteron 248''s
>
>
> Booting ''xen''
>
> root (hd0,0)
> Filesystem type is ext2fs, partition type 0x83
> kernel /xen.gz com1=115200,8n1
> [Multiboot-elf, <0x100000:0x80408:0x31bf8>, shtab=0x1b2078,
> entry=0x100000]
> module /vmlinuz-2.6-xen0 root=/dev/sda4 ro conswitch=x console=tty0
> console=tt
> yS1,115200
> [Multiboot-module @ 0x1b3000, 0x7956a8 bytes]
> savedefault
> boot
> __ __ _____ ___ ___
> \ \/ /___ _ __ |___ / / _ \ / _ \
> \ // _ \ ''_ \ |_ \| | | | | | |
> / \ __/ | | | ___) | |_| | |_| |
> /_/\_\___|_| |_| |____(_)___(_)___/
>
> http://www.cl.cam.ac.uk/netos/xen
> University of Cambridge Computer Laboratory
>
> Xen version 3.0.0 (root@ltc.austin.ibm.com) (gcc version 3.4.2 20041017
> (Red H6 Latest ChangeSet: Wed Feb 1 11:06:16 2006 +0000
> 8728:d609de73b9fa
>
> (XEN) Physical RAM map:
> (XEN) 0000000000000000 - 000000000009dc00 (usable)
> (XEN) 000000000009dc00 - 00000000000a0000 (reserved)
> (XEN) 00000000000d0000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000dff60000 (usable)
> (XEN) 00000000dff60000 - 00000000dff72000 (ACPI data)
> (XEN) 00000000dff72000 - 00000000dff80000 (ACPI NVS)
> (XEN) 00000000dff80000 - 00000000e0000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec00400 (reserved)
> (XEN) 00000000fee00000 - 00000000fee01000 (reserved)
> (XEN) 00000000fff80000 - 0000000100000000 (reserved)
> (XEN) System RAM: 3582MB (3668980kB)
> (XEN) Xen heap: 14MB (14536kB)
> (XEN) Using scheduler: Simple EDF Scheduler (sedf)
> (XEN) found SMP MP-table at 000f7140
> (XEN) DMI present.
> (XEN) Using APIC driver default
> (XEN) ACPI: RSDP (v002 PTLTD ) @
> 0x00000000000f0(XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP
> 0x00000000) @ 0x0000000f(XEN) ACPI: FADT (v003 AMD HAMMER
> 0x06040000 PTEC 0x000f4240) @ 0x00000000f(XEN) ACPI: SRAT (v001 AMD
> HAMMER 0x06040000 AMD 0x00000001) @ 0x000000003(XEN) ACPI: MADT (v001
> PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x0000000b(XEN) ACPI:
> ASF! (v016 MBI CETP 0x06040000 PTL 0x00000001) @
> 0x000000009(XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT
> 0x0100000e) @ 0x000000000(XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 15:5 APIC version 16
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 15:5 APIC version 16
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: IOAPIC (id[0x03] address[0xf9220000] gsi_base[24])
> (XEN) IOAPIC[1]: apic_id 3, version 17, address 0xf9220000, GSI 24-27
> (XEN) ACPI: IOAPIC (id[0x04] address[0xf9230000] gsi_base[28])
> (XEN) IOAPIC[2]: apic_id 4, version 17, address 0xf9230000, GSI 28-31
> (XEN) ACPI: IOAPIC (id[0x05] address[0xf9200000] gsi_base[32])
> (XEN) IOAPIC[3]: apic_id 5, version 17, address 0xf9200000, GSI 32-35
> (XEN) ACPI: IOAPIC (id[0x06] address[0xf9210000] gsi_base[36])
> (XEN) IOAPIC[4]: apic_id 6, version 17, address 0xf9210000, GSI 36-39
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode: Flat. Using 5 I/O APICs
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Initializing CPU#0
> (XEN) Detected 2193.790 MHz processor.
> (XEN) CPU0: AMD Flush Filter enabled
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU0: AMD Opteron(tm) Processor 248 stepping 0a
> (XEN) Booting processor 1/1 eip 90000
> (XEN) Initializing CPU#1
> (XEN) CPU1: AMD Flush Filter enabled
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU1: AMD Opteron(tm) Processor 248 stepping 08
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 1.193MHz PIT
> (XEN) Brought up 2 CPUs
> (XEN) mtrr: v2.0 (20020519)
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen-ELF header found:
> ''GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=xen-3.0,VIRT_''(XEN)
PHYSICAL
> MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 0000000007000000->0000000008000000 (865368 pages
> to be al)(XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff80100000->ffffffff80689d18
> (XEN) Init. ramdisk: ffffffff8068a000->ffffffff8068a000
> (XEN) Phys-Mach map: ffffffff8068a000->ffffffff80d2c2c0
> (XEN) Start info: ffffffff80d2d000->ffffffff80d2e000
> (XEN) Page tables: ffffffff80d2e000->ffffffff80d39000
> (XEN) Boot stack: ffffffff80d39000->ffffffff80d3a000
> (XEN) TOTAL: ffffffff80000000->ffffffff81000000
> (XEN) ENTRY ADDRESS: ffffffff80100000
> (XEN) Scrubbing Free RAM: ....................................done.
> (XEN) Xen trace buffers: disabled
> (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three
times to switch
> input to Xe.Linux version 2.6.14-xen0
> (root@quadmonster.ltc.austin.ibm.com) (gcc version 3.6kernel direct
> mapping tables upto d4c58000 @ d3b000-13e7000
> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> ACPI: IOAPIC (id[0x03] address[0xf9220000] gsi_base[24])
> IOAPIC[1]: apic_id 3, version 17, address 0xf9220000, GSI 24-27
> ACPI: IOAPIC (id[0x04] address[0xf9230000] gsi_base[28])
> IOAPIC[2]: apic_id 4, version 17, address 0xf9230000, GSI 28-31
> ACPI: IOAPIC (id[0x05] address[0xf9200000] gsi_base[32])
> IOAPIC[3]: apic_id 5, version 17, address 0xf9200000, GSI 32-35
> ACPI: IOAPIC (id[0x06] address[0xf9210000] gsi_base[36])
> IOAPIC[4]: apic_id 6, version 17, address 0xf9210000, GSI 36-39
> ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> Setting APIC routing to xen
> Using ACPI (MADT) for SMP configuration information
> Allocating PCI resources starting at e2000000 (gap: e0000000:1ec00000)
> Built 1 zonelists
> Kernel command line: root=/dev/sda4 ro conswitch=x console=tty0
> console=ttyS1,10Initializing CPU#0
> PID hash table entries: 4096 (order: 12, 131072 bytes)
> Xen reported: 2193.790 MHz processor.
> Console: colour VGA+ 80x25
> Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
> Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
> Software IO TLB enabled:
> Aperture: 64 megabytes
> Bus range: 0x0000000040000000 - 0x0000000044000000
> Kernel range: 0xffff880004f78000 - 0xffff880008f78000
> Memory: 3337340k/3486048k available (3499k kernel code, 147968k
> reserved, 1310k)Calibrating delay using timer specific routine.. 5485.75
> BogoMIPS (lpj=10971515)Mount-cache hash table entries: 256
> CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> CPU: L2 Cache: 1024K (64 bytes/line)
> softlockup thread 0 started up.
> ----------- [cut here ] --------- [please bite here ] ---------
> Kernel BUG at kernel/softlockup.c:110
> invalid operand: 0000 [1] SMP
> CPU 0
> Modules linked in:
> Pid: 1, comm: swapper Not tainted 2.6.14-xen0 #1
> RIP: e030:[<ffffffff8015500c>]
<ffffffff8015500c>{cpu_callback+76}
> RSP: e02b:ffff88000068be98 EFLAGS: 00010286
> RAX: ffffffff806890f8 RBX: 0000000000000001 RCX: 0000000000000001
> RDX: 0000000000000001 RSI: 0000000000000003 RDI: ffffffff80507210
> RBP: ffff88000068beb8 R08: 00000000fffffffb R09: 0000000000000000
> R10: ffff88000068bcc8 R11: 0000000000000000 R12: 0000000000000080
> R13: ffffffff806890f8 R14: 0000000000000001 R15: 0000ffffffff8010
> FS: 0000000000000000(0000) GS:ffffffff80623800(0000)
> knlGS:0000000000000000
> CS: e033 DS: 0000 ES: 0000
> Process swapper (pid: 1, threadinfo ffff88000068a000, task
> ffff88000068d4b0)
> Stack: ffff88000068bea8 ffffffff80507210 0000000000000001
> 0000000000000003
> ffff88000068bee8 ffffffff80142248 ffffffff80506c00
> 0000000000000001
> 0000000000000008 ffffffff80506c00
> Call Trace:<ffffffff80142248>{notifier_call_chain+40}
> <ffffffff8014de57>{cpu_up} <ffffffff8010e163>{init+275}
> <ffffffff80111a9e>{child_rip+8}
> <ffffffff8010e050>{init+0}
<ffffffff80111a96>{child_rip+0}
>
>
> Code: 0f 0b 68 ae a2 48 80 c2 6e 00 31 c0 89 d9 48 c7 c2 c2 a2 48
> RIP <ffffffff8015500c>{cpu_callback+76} RSP <ffff88000068be98>
> <0>Kernel panic - not syncing: Attempted to kill init!
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Hi, On Fri, 2006-02-03 at 09:44 -0600, Ryan Grimm wrote:> I tried the patch you pointed out but still get the same result. I > turned off ''detect_soft_lockups'', and get the message below.I''m also completely unable to get x86_64 to work at all from any tree with CONFIG_SMP set. I get a cpu_up notifier chain panic if CONFIG_HOTPLUG_CPU is not set; with CONFIG_HOTPLUG_CPU=y, it doesn''t even compile. Replacing one missing function in setup64-xen.c fixes that for dom0 but domU is more complex. If nobody''s working on this, I''ll take a stab at trying to sort things out so that at least one config set works for x86_64 SMP. It''s really a rather important use case! --Stephen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stephen C. Tweedie wrote:> Hi, > > On Fri, 2006-02-03 at 09:44 -0600, Ryan Grimm wrote: > >> I tried the patch you pointed out but still get the same result. I >> turned off ''detect_soft_lockups'', and get the message below. > > I''m also completely unable to get x86_64 to work at all from any tree > with CONFIG_SMP set. I get a cpu_up notifier chain panic if > CONFIG_HOTPLUG_CPU is not set; with CONFIG_HOTPLUG_CPU=y, it doesn''t > even compile. Replacing one missing function in setup64-xen.c fixes > that for dom0 but domU is more complex. > > If nobody''s working on this, I''ll take a stab at trying to sort things > out so that at least one config set works for x86_64 SMP. It''s > really a rather important use case! > > --Stephen > >Restoring x86_64 SMP is high priority, and you should be very welcome. Please take a look at Christian''s latest fix 8775; it may have fixed that. I''m working on other things now. Jun --- Intel Open Source Technology Center _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Stephen C. Tweedie wrote: > > Hi, > > > > On Fri, 2006-02-03 at 09:44 -0600, Ryan Grimm wrote: > > > >> I tried the patch you pointed out but still get the same result. I > >> turned off ''detect_soft_lockups'', and get the message below. > > > > I''m also completely unable to get x86_64 to work at all from any tree > > with CONFIG_SMP set. I get a cpu_up notifier chain panic if > > CONFIG_HOTPLUG_CPU is not set; with CONFIG_HOTPLUG_CPU=y, it doesn''t > > even compile. Replacing one missing function in setup64-xen.c fixes > > that for dom0 but domU is more complex. > > > > If nobody''s working on this, I''ll take a stab at trying to sort things > > out so that at least one config set works for x86_64 SMP. It''s > > really a rather important use case! > > > > --Stephen > > > > > > Restoring x86_64 SMP is high priority, and you should be very welcome. > Please take a look at Christian''s latest fix 8775; it may have fixed > that. I''m working on other things now.I can confirm that 8775 fixes SMP dom0 on 64 for me (the main problem was the cpu_possible_map which is why CONFIG_HOTPLUG_CPU worked - it prefills the possible_map to NR_CPUs) cheers, S. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, On Tue, 2006-02-07 at 17:45 +0000, Steven Hand wrote:> I can confirm that 8775 fixes SMP dom0 on 64 for me (the main problem > was the cpu_possible_map which is why CONFIG_HOTPLUG_CPU worked - it > prefills the possible_map to NR_CPUs)With that and Chris Wright''s ia32_sysenter patch in the tree, things boot just fine for me now on x86_64 SMP dom0 --- great. That''s without CONFIG_HOTPLUG_CPU, though --- I''ll have to check CPU hotplug out separately, as that''s still a useful feature to keep for Xen, and it does appear to work fine on ia32 right now. ia32 emulation may still have a problem, though --- I''m digging into another such oops (appended) right now. --Stephen Kernel BUG at arch/x86_64/mm/../../i386/mm/hypervisor.c:196 invalid opcode: 0000 [1] SMP CPU 3 Modules linked in: nfsd exportfs video thermal processor fan button battery ac Pid: 3859, comm: java Not tainted 2.6.16-rc2-xen0 #1 RIP: e030:[<ffffffff8011d375>] <ffffffff8011d375>{xen_pgd_pin+85} RSP: e02b:ffff8800241a1bf8 EFLAGS: 00010082 RAX: 00000000ffffffea RBX: ffff88001a857900 RCX: 000000000001f684 RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8800241a1bf8 RBP: ffff8800241a1c18 R08: ffff8800203ccff8 R09: 0000000000000200 R10: 0000000000007ff0 R11: ffff800000000000 R12: ffff88001a857900 R13: ffff880038bc2dc0 R14: ffff880001ed8360 R15: 0000003b7bcd351b FS: 00002b2fa6dc81e0(0000) GS:ffffffff80663180(0000) knlGS:0000000000000000 CS: e033 DS: 0000 ES: 0000 Process java (pid: 3859, threadinfo ffff8800241a0000, task ffff880000922200) Stack: ffff880000000003 00000000000323d3 ffff880001ed8360 ffff800000000000 ffff8800241a1c38 ffffffff8011c9f3 ffff880000922200 0000000000000003 ffff8800241a1d18 ffffffff8048e259 Call Trace: <ffffffff8011c9f3>{mm_pin+371} <ffffffff8048e259>{__sched_text_start+1625} <ffffffff8013933c>{lock_timer_base+44} <ffffffff8013933c>{lock_timer_base+44} <ffffffff801394ca>{__mod_timer+186} <ffffffff8048f16c>{schedule_timeout+156} <ffffffff80138e10>{process_timeout+0} <ffffffff80148fcf>{do_futex+671} <ffffffff80304a5c>{end_pirq+76} <ffffffff80128190>{default_wake_function+0} <ffffffff8030511e>{evtchn_do_upcall+142} <ffffffff80150104>{compat_sys_futex+196} <ffffffff8011e571>{ia32_sysret+0} <ffffffff8011e506>{ia32_syscall+30} Code: 0f 0b 68 38 bb 4b 80 c2 c4 00 90 c9 c3 66 66 66 90 66 66 66 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7 Feb 2006, at 20:17, Stephen C. Tweedie wrote:> With that and Chris Wright''s ia32_sysenter patch in the tree, things > boot just fine for me now on x86_64 SMP dom0 --- great. > > That''s without CONFIG_HOTPLUG_CPU, though --- I''ll have to check CPU > hotplug out separately, as that''s still a useful feature to keep for > Xen, and it does appear to work fine on ia32 right now.Yes, we need it for suspend/resume of SMP guests.> ia32 emulation may still have a problem, though --- I''m digging into > another such oops (appended) right now.Changeset 8778 might fix this. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, On Tue, 2006-02-07 at 23:20 +0000, Keir Fraser wrote:> > ia32 emulation may still have a problem, though --- I''m digging into > > another such oops (appended) right now. > > Changeset 8778 might fix this.Right, as soon as I saw it appear I started a build to test that out. Will report back. --Stephen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, On Tue, 2006-02-07 at 20:17 -0500, Stephen Tweedie wrote:> > Changeset 8778 might fix this. > > Right, as soon as I saw it appear I started a build to test that out. > Will report back.I certainly haven''t seen this resurface with that cset applied, thanks. --Stephen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel