Chris Wright
2005-Jul-28 19:36 UTC
[Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
During boot of dom0, I get the following. Box is remote, so I''m
now waiting for proxy power button pusher ;-) tg3 init fails doing
dma_alloc_coherent (I assume this is from tg3_init_one -> tg3_test_dma ->
pci_alloc_consistent -> dma_alloc_coherent). Looks like an issue with
the amount of memory in dom0. Is this known?
thanks,
-chris
root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
kernel /xen.gz com1=115200,8n1 console=com1,vga
[Multiboot-elf, <0x100000:0x7c3e0:0x29c20>, shtab=0x1a6078,
entry=0x100000]
module /vmlinuz-2.6.12-xen0 root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0
[Multiboot-module @ 0x1a7000, 0x6b43f8 bytes]
module /initrd-generic-2.6.img
[Multiboot-module @ 0x85c000, 0x2ab400 bytes]
__ __ _____ ___ _ _
\ \/ /___ _ __ |___ / / _ \ __| | _____ _____| |
\ // _ \ ''_ \ |_ \| | | |__ / _` |/ _ \ \ / / _ \ |
/ \ __/ | | | ___) | |_| |__| (_| | __/\ V / __/ |
/_/\_\___|_| |_| |____(_)___/ \__,_|\___| \_/ \___|_|
http://www.cl.cam.ac.uk/netos/xen
University of Cambridge Computer Laboratory
Xen version 3.0-devel (chrisw@pdx.osdl.net) (gcc version 3.4.3 20041125 (Red
Hat 3.4.3-7)) Thu Jul 28 10:54:44 PDT 2005
Latest ChangeSet: Thu Jul 28 05:34:45 2005
501a70f3ae968e46e27b9003febf05253f4cf949
(XEN) Physical RAM map:
(XEN) 0000000000000000 - 000000000009dc00 (usable)
(XEN) 000000000009dc00 - 00000000000a0000 (reserved)
(XEN) 00000000000d0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 000000007ff60000 (usable)
(XEN) 000000007ff60000 - 000000007ff72000 (ACPI data)
(XEN) 000000007ff72000 - 000000007ff80000 (ACPI NVS)
(XEN) 000000007ff80000 - 0000000080000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec00400 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fff80000 - 0000000100000000 (reserved)
(XEN) System RAM: 2046MB (2096116kB)
(XEN) Xen heap: 14MB (14632kB)
(XEN) found SMP MP-table at 000f7270
(XEN) DMI present.
(XEN) Using APIC driver default
(XEN) ACPI: RSDP (v002 PTLTD ) @
0x00000000000f7210
(XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP 0x00000000) @
0x000000007ff6d5a7
(XEN) ACPI: FADT (v003 AMD HAMMER 0x06040000 PTEC 0x000f4240) @
0x000000007ff71d0f
(XEN) ACPI: SRAT (v001 AMD HAMMER 0x06040000 AMD 0x00000001) @
0x000000007ff71e03
(XEN) ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @
0x000000007ff71ecb
(XEN) ACPI: ASF! (v016 MBI CETP 0x06040000 PTL 0x00000001) @
0x000000007ff71f59
(XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT 0x0100000e) @
0x0000000000000000
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 15:5 APIC version 16
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 15:5 APIC version 16
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
(XEN) ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
(XEN) IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
(XEN) ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
(XEN) IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
(XEN) ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
(XEN) IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 5 I/O APICs
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Initializing CPU#0
(XEN) Detected 1994.377 MHz processor.
(XEN) Using scheduler: Simple EDF Scheduler (sedf)
(XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
(XEN) CPU: L2 Cache: 1024K (64 bytes/line)
(XEN) CPU0: AMD Opteron(tm) Processor 246 stepping 08
(XEN) Booting processor 1/1 eip 90000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
(XEN) CPU: L2 Cache: 1024K (64 bytes/line)
(XEN) CPU1: AMD Opteron(tm) Processor 246 stepping 08
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 1.193MHz PIT
(XEN) Brought up 2 CPUs
(XEN) mtrr: v2.0 (20020519)
(XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
(XEN) mtrr: probably your BIOS does not setup all CPUs.
(XEN) mtrr: corrected configuration.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen-ELF header found:
''GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE=0xffffffff80000000,LOADER=generic''
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 0000000020000000->0000000040000000 (382799 pages to be
allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff80100000->ffffffff805ab086
(XEN) Init. ramdisk: ffffffff805ac000->ffffffff80857400
(XEN) Phys-Mach map: ffffffff80858000->ffffffff80a4dd3c
(XEN) Page tables: ffffffff80a4e000->ffffffff80a57000
(XEN) Start info: ffffffff80a57000->ffffffff80a58000
(XEN) Boot stack: ffffffff80a58000->ffffffff80a59000
(XEN) TOTAL: ffffffff80000000->ffffffff80c00000
(XEN) ENTRY ADDRESS: ffffffff80100000
(XEN) Initrd len 0x2ab400, start at 0xffffffff805ac000
(XEN) Scrubbing Free RAM: .....................done.
(XEN) Xen trace buffers: initialised
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch input to Xen).
Linux version 2.6.12.2-xen0 (chrisw@fuzzy.pdx.osdl.net) (gcc version 3.4.3
20041125 (Red Hat 3.4.3-7)) #6 Thu Jul 28 10:53:46 PDT 2005
kernel direct mapping tables upto ffff88007d74f000 @ c00000-fef000
Registering memory for bootmem: from 1000000, size = 7c74f000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
Setting APIC routing to flat
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 7d800000 (gap: 7d74f000:828b1000)
Built 1 zonelists
Kernel command line: root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 131072 bytes)
Xen reported: 1994.377 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 2007552k/2055484k available (2820k kernel code, 31364k reserved, 1119k
data, 272k init)
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU: AMD Opteron(tm) Processor 246 stepping 08
checking if image is initramfs... it is
NET: Registered protocol family 16
PCI: Using configuration type 1
ACPI: Subsystem revision 20050309
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Probing PCI hardware (bus 00)
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 *5 10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 10 *11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 *10 11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 5 10 *11)
ACPI: PCI Root Bridge [PCI1] (0000:08)
PCI: Probing PCI hardware (bus 08)
xen_mem: Initialising balloon driver.
SCSI subsystem initialized
PCI: Using ACPI for IRQ routing
PCI: If a device doesn''t work, try "pci=routeirq". If it
helps, post a report
Grant table initialized
IA-32 Microcode Update Driver: v1.14 <tigran@veritas.com>
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
Initializing Cryptographic API
PCI: MSI quirk detected. pci_msi_quirk set.
PCI: MSI quirk detected. pci_msi_quirk set.
PCI: MSI quirk detected. pci_msi_quirk set.
PCI: MSI quirk detected. pci_msi_quirk set.
Real Time Clock Driver v1.12
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered
floppy0: no floppy controllers found
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
loop: loaded (max 8 devices)
HP CISS Driver (v 2.6.6)
Intel(R) PRO/1000 Network Driver - version 6.0.54-k2
Copyright (c) 1999-2004 Intel Corporation.
pcnet32.c:v1.30j 29.04.2005 tsbogend@alpha.franken.de
e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
e100: Copyright(c) 1999-2005 Intel Corporation
tg3.c:v3.31 (June 8, 2005)
ACPI: PCI Interrupt 0000:03:02.0[A] -> GSI 28 (level, low) -> IRQ 28
(XEN) (file=dom_mem_ops.c, line=61) Could not allocate a frame
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at "arch/xen/x86_64/mm/hypervisor.c":299
invalid operand: 0000 [1]
CPU 0
Modules linked in:
Pid: 1, comm: swapper Not tainted 2.6.12.2-xen0
RIP: e030:[<ffffffff8011a32a>]
<ffffffff8011a32a>{xen_contig_memory+586}
RSP: e02b:ffff880002ee1bf8 EFLAGS: 00010296
RAX: 0000000000000000 RBX: 000000000000000c RCX: ffffffff8011a324
RDX: 0000000000000001 RSI: ffff880002ee1c28 RDI: 00000000ffffffff
RBP: 0000000000000001 R08: 0000000000007ff0 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000002 R14: 0000000000000001 R15: ffff800000000000
FS: 0000000000000000(0000) GS:ffffffff80533a80(0000) knlGS:0000000000000000
CS: e033 DS: 0000 ES: 0000
Process swapper (pid: 1, threadinfo ffff880002ee0000, task ffff880002edf410)
Stack: 000000001002029a ffff880078cc6000 0000000202edf410 0000000052aa4d54
0000000000000000 ffff880078cc4000 0000000000044a89 0000000000000001
ffff880078cc4000 0000000000000000
Call Trace:<ffffffff80114920>{dma_alloc_coherent+224}
<ffffffff802a8096>{tg3_init_one+7302}
<ffffffff803bdafa>{wait_for_completion+170}
<ffffffff80126dd0>{default_wake_function+0}
<ffffffff802212d6>{pci_device_probe+134}
<ffffffff8025ac0d>{driver_probe_device+77}
<ffffffff8025ad2b>{driver_attach+75}
<ffffffff8025b118>{bus_add_driver+152}
<ffffffff80220fe5>{pci_register_driver+117}
<ffffffff801090f2>{init+178}
<ffffffff8010e567>{child_rip+8} <ffffffff80109040>{init+0}
<ffffffff8010e55f>{child_rip+0}
Code: 0f 0b 08 91 3d 80 ff ff ff ff 2b 01 b8 01 00 00 00 44 89 f1
RIP <ffffffff8011a32a>{xen_contig_memory+586} RSP <ffff880002ee1bf8>
<0>Kernel panic - not syncing: Attempted to kill init!
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Jerone Young
2005-Jul-28 19:44 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
What build are you using? I have a dual opteron with the same card & driver (tg3) ... and works fine. I''ll try today''s build. But the build I''m using is only a few days old and has no problems with x86-64. I''ll try todays and see what happens. On Thu, 2005-07-28 at 12:36 -0700, Chris Wright wrote:> During boot of dom0, I get the following. Box is remote, so I''m > now waiting for proxy power button pusher ;-) tg3 init fails doing > dma_alloc_coherent (I assume this is from tg3_init_one -> tg3_test_dma -> > pci_alloc_consistent -> dma_alloc_coherent). Looks like an issue with > the amount of memory in dom0. Is this known? > > thanks, > -chris > > root (hd0,0) > Filesystem type is ext2fs, partition type 0x83 > kernel /xen.gz com1=115200,8n1 console=com1,vga > [Multiboot-elf, <0x100000:0x7c3e0:0x29c20>, shtab=0x1a6078, entry=0x100000] > module /vmlinuz-2.6.12-xen0 root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0 > [Multiboot-module @ 0x1a7000, 0x6b43f8 bytes] > module /initrd-generic-2.6.img > [Multiboot-module @ 0x85c000, 0x2ab400 bytes] > > __ __ _____ ___ _ _ > \ \/ /___ _ __ |___ / / _ \ __| | _____ _____| | > \ // _ \ ''_ \ |_ \| | | |__ / _` |/ _ \ \ / / _ \ | > / \ __/ | | | ___) | |_| |__| (_| | __/\ V / __/ | > /_/\_\___|_| |_| |____(_)___/ \__,_|\___| \_/ \___|_| > > http://www.cl.cam.ac.uk/netos/xen > University of Cambridge Computer Laboratory > > Xen version 3.0-devel (chrisw@pdx.osdl.net) (gcc version 3.4.3 20041125 (Red Hat 3.4.3-7)) Thu Jul 28 10:54:44 PDT 2005 > Latest ChangeSet: Thu Jul 28 05:34:45 2005 501a70f3ae968e46e27b9003febf05253f4cf949 > > (XEN) Physical RAM map: > (XEN) 0000000000000000 - 000000000009dc00 (usable) > (XEN) 000000000009dc00 - 00000000000a0000 (reserved) > (XEN) 00000000000d0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 000000007ff60000 (usable) > (XEN) 000000007ff60000 - 000000007ff72000 (ACPI data) > (XEN) 000000007ff72000 - 000000007ff80000 (ACPI NVS) > (XEN) 000000007ff80000 - 0000000080000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec00400 (reserved) > (XEN) 00000000fee00000 - 00000000fee01000 (reserved) > (XEN) 00000000fff80000 - 0000000100000000 (reserved) > (XEN) System RAM: 2046MB (2096116kB) > (XEN) Xen heap: 14MB (14632kB) > (XEN) found SMP MP-table at 000f7270 > (XEN) DMI present. > (XEN) Using APIC driver default > (XEN) ACPI: RSDP (v002 PTLTD ) @ 0x00000000000f7210 > (XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP 0x00000000) @ 0x000000007ff6d5a7 > (XEN) ACPI: FADT (v003 AMD HAMMER 0x06040000 PTEC 0x000f4240) @ 0x000000007ff71d0f > (XEN) ACPI: SRAT (v001 AMD HAMMER 0x06040000 AMD 0x00000001) @ 0x000000007ff71e03 > (XEN) ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP 0x00000000) @ 0x000000007ff71ecb > (XEN) ACPI: ASF! (v016 MBI CETP 0x06040000 PTL 0x00000001) @ 0x000000007ff71f59 > (XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT 0x0100000e) @ 0x0000000000000000 > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 15:5 APIC version 16 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 15:5 APIC version 16 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23 > (XEN) ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24]) > (XEN) IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27 > (XEN) ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28]) > (XEN) IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31 > (XEN) ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32]) > (XEN) IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35 > (XEN) ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36]) > (XEN) IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) Enabling APIC mode: Flat. Using 5 I/O APICs > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Initializing CPU#0 > (XEN) Detected 1994.377 MHz processor. > (XEN) Using scheduler: Simple EDF Scheduler (sedf) > (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) > (XEN) CPU: L2 Cache: 1024K (64 bytes/line) > (XEN) CPU0: AMD Opteron(tm) Processor 246 stepping 08 > (XEN) Booting processor 1/1 eip 90000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) > (XEN) CPU: L2 Cache: 1024K (64 bytes/line) > (XEN) CPU1: AMD Opteron(tm) Processor 246 stepping 08 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs > (XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 1.193MHz PIT > (XEN) Brought up 2 CPUs > (XEN) mtrr: v2.0 (20020519) > (XEN) mtrr: your CPUs had inconsistent fixed MTRR settings > (XEN) mtrr: probably your BIOS does not setup all CPUs. > (XEN) mtrr: corrected configuration. > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen-ELF header found: ''GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE=0xffffffff80000000,LOADER=generic'' > (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000020000000->0000000040000000 (382799 pages to be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff80100000->ffffffff805ab086 > (XEN) Init. ramdisk: ffffffff805ac000->ffffffff80857400 > (XEN) Phys-Mach map: ffffffff80858000->ffffffff80a4dd3c > (XEN) Page tables: ffffffff80a4e000->ffffffff80a57000 > (XEN) Start info: ffffffff80a57000->ffffffff80a58000 > (XEN) Boot stack: ffffffff80a58000->ffffffff80a59000 > (XEN) TOTAL: ffffffff80000000->ffffffff80c00000 > (XEN) ENTRY ADDRESS: ffffffff80100000 > (XEN) Initrd len 0x2ab400, start at 0xffffffff805ac000 > (XEN) Scrubbing Free RAM: .....................done. > (XEN) Xen trace buffers: initialised > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to switch input to Xen). > Linux version 2.6.12.2-xen0 (chrisw@fuzzy.pdx.osdl.net) (gcc version 3.4.3 20041125 (Red Hat 3.4.3-7)) #6 Thu Jul 28 10:53:46 PDT 2005 > kernel direct mapping tables upto ffff88007d74f000 @ c00000-fef000 > Registering memory for bootmem: from 1000000, size = 7c74f000 > ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23 > ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24]) > IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27 > ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28]) > IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31 > ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32]) > IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35 > ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36]) > IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39 > ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) > Setting APIC routing to flat > Using ACPI (MADT) for SMP configuration information > Allocating PCI resources starting at 7d800000 (gap: 7d74f000:828b1000) > Built 1 zonelists > Kernel command line: root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0 > Initializing CPU#0 > PID hash table entries: 4096 (order: 12, 131072 bytes) > Xen reported: 1994.377 MHz processor. > Console: colour VGA+ 80x25 > Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes) > Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes) > Memory: 2007552k/2055484k available (2820k kernel code, 31364k reserved, 1119k data, 272k init) > Mount-cache hash table entries: 256 > CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) > CPU: L2 Cache: 1024K (64 bytes/line) > CPU: AMD Opteron(tm) Processor 246 stepping 08 > checking if image is initramfs... it is > NET: Registered protocol family 16 > PCI: Using configuration type 1 > ACPI: Subsystem revision 20050309 > ACPI: Interpreter enabled > ACPI: Using IOAPIC for interrupt routing > ACPI: PCI Root Bridge [PCI0] (0000:00) > PCI: Probing PCI hardware (bus 00) > ACPI: PCI Interrupt Link [LNKA] (IRQs 3 *5 10 11) > ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 10 *11) > ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 *10 11) > ACPI: PCI Interrupt Link [LNKD] (IRQs 3 5 10 *11) > ACPI: PCI Root Bridge [PCI1] (0000:08) > PCI: Probing PCI hardware (bus 08) > xen_mem: Initialising balloon driver. > SCSI subsystem initialized > PCI: Using ACPI for IRQ routing > PCI: If a device doesn''t work, try "pci=routeirq". If it helps, post a report > Grant table initialized > IA-32 Microcode Update Driver: v1.14 <tigran@veritas.com> > IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $ > Initializing Cryptographic API > PCI: MSI quirk detected. pci_msi_quirk set. > PCI: MSI quirk detected. pci_msi_quirk set. > PCI: MSI quirk detected. pci_msi_quirk set. > PCI: MSI quirk detected. pci_msi_quirk set. > Real Time Clock Driver v1.12 > serio: i8042 AUX port at 0x60,0x64 irq 12 > serio: i8042 KBD port at 0x60,0x64 irq 1 > io scheduler noop registered > io scheduler anticipatory registered > io scheduler deadline registered > io scheduler cfq registered > floppy0: no floppy controllers found > RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize > loop: loaded (max 8 devices) > HP CISS Driver (v 2.6.6) > Intel(R) PRO/1000 Network Driver - version 6.0.54-k2 > Copyright (c) 1999-2004 Intel Corporation. > pcnet32.c:v1.30j 29.04.2005 tsbogend@alpha.franken.de > e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI > e100: Copyright(c) 1999-2005 Intel Corporation > tg3.c:v3.31 (June 8, 2005) > ACPI: PCI Interrupt 0000:03:02.0[A] -> GSI 28 (level, low) -> IRQ 28 > (XEN) (file=dom_mem_ops.c, line=61) Could not allocate a frame > ----------- [cut here ] --------- [please bite here ] --------- > Kernel BUG at "arch/xen/x86_64/mm/hypervisor.c":299 > invalid operand: 0000 [1] > CPU 0 > Modules linked in: > Pid: 1, comm: swapper Not tainted 2.6.12.2-xen0 > RIP: e030:[<ffffffff8011a32a>] <ffffffff8011a32a>{xen_contig_memory+586} > RSP: e02b:ffff880002ee1bf8 EFLAGS: 00010296 > RAX: 0000000000000000 RBX: 000000000000000c RCX: ffffffff8011a324 > RDX: 0000000000000001 RSI: ffff880002ee1c28 RDI: 00000000ffffffff > RBP: 0000000000000001 R08: 0000000000007ff0 R09: 0000000000000001 > R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000 > R13: 0000000000000002 R14: 0000000000000001 R15: ffff800000000000 > FS: 0000000000000000(0000) GS:ffffffff80533a80(0000) knlGS:0000000000000000 > CS: e033 DS: 0000 ES: 0000 > Process swapper (pid: 1, threadinfo ffff880002ee0000, task ffff880002edf410) > Stack: 000000001002029a ffff880078cc6000 0000000202edf410 0000000052aa4d54 > 0000000000000000 ffff880078cc4000 0000000000044a89 0000000000000001 > ffff880078cc4000 0000000000000000 > Call Trace:<ffffffff80114920>{dma_alloc_coherent+224} <ffffffff802a8096>{tg3_init_one+7302} > <ffffffff803bdafa>{wait_for_completion+170} <ffffffff80126dd0>{default_wake_function+0} > <ffffffff802212d6>{pci_device_probe+134} <ffffffff8025ac0d>{driver_probe_device+77} > <ffffffff8025ad2b>{driver_attach+75} <ffffffff8025b118>{bus_add_driver+152} > <ffffffff80220fe5>{pci_register_driver+117} <ffffffff801090f2>{init+178} > <ffffffff8010e567>{child_rip+8} <ffffffff80109040>{init+0} > <ffffffff8010e55f>{child_rip+0} > > Code: 0f 0b 08 91 3d 80 ff ff ff ff 2b 01 b8 01 00 00 00 44 89 f1 > RIP <ffffffff8011a32a>{xen_contig_memory+586} RSP <ffff880002ee1bf8> > <0>Kernel panic - not syncing: Attempted to kill init! > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- Jerone Young IBM Linux Technology Center jyoung5@us.ibm.com 512-838-1157 (T/L: 678-1157) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Chris Wright
2005-Jul-28 19:52 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
* Jerone Young (jyoung5@us.ibm.com) wrote:> What build are you using? I have a dual opteron with the same card & > driver (tg3) ... and works fine. I''ll try today''s build. But the build > I''m using is only a few days old and has no problems with x86-64. I''ll > try todays and see what happens.Both current hg, as well as a pull from yesterday. The box is remote, so a reboot is a very slow operation. I just confirmed that reducing the amount of dom0 memory fixes the issue. Machine has 2G, and I was giving all of it to dom0. thanks, -chris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Scott Parish
2005-Jul-28 20:03 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
On Thu, Jul 28, 2005 at 09:59:22PM +0100, Ian Pratt wrote:> Do you know how large a contiguous chunk the tg3 needs?A quick glance at the code, it looks like it requests at least 4 pages, and possibly multiple times.> I guess we could hold onto a few large chunks before making the dom0 > memory allocation.I''m surprised that the mfns are already fragmented that early in boot. Or maybe this case is just an example of bad luck? sRp -- Scott Parish Signed-off-by: srparish@us.ibm.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jerone Young
2005-Jul-28 20:59 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
So when I remove "dom0_mem" from the xen boot parmater & let dom0 allocate all memory I hit the same problem. But also with todays HG build I can nolonger fully boot Dom0. Stops just after printing "io scheduler cfq registered". I''ll talk a look at both to see what is going on. On Thu, 2005-07-28 at 12:52 -0700, Chris Wright wrote:> * Jerone Young (jyoung5@us.ibm.com) wrote: > > What build are you using? I have a dual opteron with the same card & > > driver (tg3) ... and works fine. I''ll try today''s build. But the build > > I''m using is only a few days old and has no problems with x86-64. I''ll > > try todays and see what happens. > > Both current hg, as well as a pull from yesterday. The box is remote, > so a reboot is a very slow operation. I just confirmed that reducing > the amount of dom0 memory fixes the issue. Machine has 2G, and I was > giving all of it to dom0. > > thanks, > -chris > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- Jerone Young IBM Linux Technology Center jyoung5@us.ibm.com 512-838-1157 (T/L: 678-1157) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt
2005-Jul-28 20:59 UTC
RE: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
> During boot of dom0, I get the following. Box is remote, so > I''m now waiting for proxy power button pusher ;-) tg3 init > fails doing dma_alloc_coherent (I assume this is from > tg3_init_one -> tg3_test_dma -> pci_alloc_consistent -> > dma_alloc_coherent). Looks like an issue with the amount of > memory in dom0. Is this known?Is this with a debug=y build of Xen? I can see how that might provoke the situation more than other builds. Do you know how large a contiguous chunk the tg3 needs? I guess we could hold onto a few large chunks before making the dom0 memory allocation. Alternatively, if we get a failure, we should round up to the next power of two and then balloon out that much memory. Repeat until success... Ian> thanks, > -chris > > root (hd0,0) > Filesystem type is ext2fs, partition type 0x83 kernel > /xen.gz com1=115200,8n1 console=com1,vga > [Multiboot-elf, <0x100000:0x7c3e0:0x29c20>, > shtab=0x1a6078, entry=0x100000] module /vmlinuz-2.6.12-xen0 > root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0 > [Multiboot-module @ 0x1a7000, 0x6b43f8 bytes] module > /initrd-generic-2.6.img > [Multiboot-module @ 0x85c000, 0x2ab400 bytes] > > __ __ _____ ___ _ _ > \ \/ /___ _ __ |___ / / _ \ __| | _____ _____| | > \ // _ \ ''_ \ |_ \| | | |__ / _` |/ _ \ \ / / _ \ | > / \ __/ | | | ___) | |_| |__| (_| | __/\ V / __/ | > /_/\_\___|_| |_| |____(_)___/ \__,_|\___| \_/ \___|_| > > http://www.cl.cam.ac.uk/netos/xen > University of Cambridge Computer Laboratory > > Xen version 3.0-devel (chrisw@pdx.osdl.net) (gcc version > 3.4.3 20041125 (Red Hat 3.4.3-7)) Thu Jul 28 10:54:44 PDT > 2005 Latest ChangeSet: Thu Jul 28 05:34:45 2005 > 501a70f3ae968e46e27b9003febf05253f4cf949 > > (XEN) Physical RAM map: > (XEN) 0000000000000000 - 000000000009dc00 (usable) > (XEN) 000000000009dc00 - 00000000000a0000 (reserved) > (XEN) 00000000000d0000 - 0000000000100000 (reserved) > (XEN) 0000000000100000 - 000000007ff60000 (usable) > (XEN) 000000007ff60000 - 000000007ff72000 (ACPI data) > (XEN) 000000007ff72000 - 000000007ff80000 (ACPI NVS) > (XEN) 000000007ff80000 - 0000000080000000 (reserved) > (XEN) 00000000fec00000 - 00000000fec00400 (reserved) > (XEN) 00000000fee00000 - 00000000fee01000 (reserved) > (XEN) 00000000fff80000 - 0000000100000000 (reserved) > (XEN) System RAM: 2046MB (2096116kB) > (XEN) Xen heap: 14MB (14632kB) > (XEN) found SMP MP-table at 000f7270 > (XEN) DMI present. > (XEN) Using APIC driver default > (XEN) ACPI: RSDP (v002 PTLTD > ) @ 0x00000000000f7210 > (XEN) ACPI: XSDT (v001 PTLTD XSDT 0x06040000 LTP > 0x00000000) @ 0x000000007ff6d5a7 > (XEN) ACPI: FADT (v003 AMD HAMMER 0x06040000 PTEC > 0x000f4240) @ 0x000000007ff71d0f > (XEN) ACPI: SRAT (v001 AMD HAMMER 0x06040000 AMD > 0x00000001) @ 0x000000007ff71e03 > (XEN) ACPI: MADT (v001 PTLTD APIC 0x06040000 LTP > 0x00000000) @ 0x000000007ff71ecb > (XEN) ACPI: ASF! (v016 MBI CETP 0x06040000 PTL > 0x00000001) @ 0x000000007ff71f59 > (XEN) ACPI: DSDT (v001 AMD-K8 AMDACPI 0x06040000 MSFT > 0x0100000e) @ 0x0000000000000000 > (XEN) ACPI: Local APIC address 0xfee00000 > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > (XEN) Processor #0 15:5 APIC version 16 > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > (XEN) Processor #1 15:5 APIC version 16 > (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23 > (XEN) ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24]) > (XEN) IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27 > (XEN) ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28]) > (XEN) IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31 > (XEN) ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32]) > (XEN) IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35 > (XEN) ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36]) > (XEN) IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39 > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) > (XEN) ACPI: IRQ0 used by override. > (XEN) ACPI: IRQ2 used by override. > (XEN) Enabling APIC mode: Flat. Using 5 I/O APICs > (XEN) Using ACPI (MADT) for SMP configuration information > (XEN) Initializing CPU#0 > (XEN) Detected 1994.377 MHz processor. > (XEN) Using scheduler: Simple EDF Scheduler (sedf) > (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 > bytes/line) > (XEN) CPU: L2 Cache: 1024K (64 bytes/line) > (XEN) CPU0: AMD Opteron(tm) Processor 246 stepping 08 > (XEN) Booting processor 1/1 eip 90000 > (XEN) Initializing CPU#1 > (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 > bytes/line) > (XEN) CPU: L2 Cache: 1024K (64 bytes/line) > (XEN) CPU1: AMD Opteron(tm) Processor 246 stepping 08 > (XEN) Total of 2 processors activated. > (XEN) ENABLING IO-APIC IRQs > (XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1 > (XEN) checking TSC synchronization across 2 CPUs: passed. > (XEN) Platform timer is 1.193MHz PIT > (XEN) Brought up 2 CPUs > (XEN) mtrr: v2.0 (20020519) > (XEN) mtrr: your CPUs had inconsistent fixed MTRR settings > (XEN) mtrr: probably your BIOS does not setup all CPUs. > (XEN) mtrr: corrected configuration. > (XEN) *** LOADING DOMAIN 0 *** > (XEN) Xen-ELF header found: > ''GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE=0xffffffff80000000,LOADER=generic''> (XEN) PHYSICAL MEMORY ARRANGEMENT: > (XEN) Dom0 alloc.: 0000000020000000->0000000040000000 > (382799 pages to be allocated) > (XEN) VIRTUAL MEMORY ARRANGEMENT: > (XEN) Loaded kernel: ffffffff80100000->ffffffff805ab086 > (XEN) Init. ramdisk: ffffffff805ac000->ffffffff80857400 > (XEN) Phys-Mach map: ffffffff80858000->ffffffff80a4dd3c > (XEN) Page tables: ffffffff80a4e000->ffffffff80a57000 > (XEN) Start info: ffffffff80a57000->ffffffff80a58000 > (XEN) Boot stack: ffffffff80a58000->ffffffff80a59000 > (XEN) TOTAL: ffffffff80000000->ffffffff80c00000 > (XEN) ENTRY ADDRESS: ffffffff80100000 > (XEN) Initrd len 0x2ab400, start at 0xffffffff805ac000 > (XEN) Scrubbing Free RAM: .....................done. > (XEN) Xen trace buffers: initialised > (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times to > switch input to Xen). > Linux version 2.6.12.2-xen0 (chrisw@fuzzy.pdx.osdl.net) (gcc > version 3.4.3 20041125 (Red Hat 3.4.3-7)) #6 Thu Jul 28 > 10:53:46 PDT 2005 kernel direct mapping tables upto > ffff88007d74f000 @ c00000-fef000 Registering memory for > bootmem: from 1000000, size = 7c74f000 > ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) > ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) > ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) > ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) > ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0]) > IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23 > ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24]) > IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27 > ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28]) > IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31 > ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32]) > IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35 > ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36]) > IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39 > ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge) > Setting APIC routing to flat Using ACPI (MADT) for SMP > configuration information Allocating PCI resources starting > at 7d800000 (gap: 7d74f000:828b1000) Built 1 zonelists Kernel > command line: root=/dev/VolGroup00/LogVol00 ro > console=tty0,ttyS0 Initializing CPU#0 PID hash table entries: > 4096 (order: 12, 131072 bytes) Xen reported: 1994.377 MHz processor. > Console: colour VGA+ 80x25 > Dentry cache hash table entries: 262144 (order: 9, 2097152 > bytes) Inode-cache hash table entries: 131072 (order: 8, > 1048576 bytes) > Memory: 2007552k/2055484k available (2820k kernel code, > 31364k reserved, 1119k data, 272k init) Mount-cache hash > table entries: 256 > CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line) > CPU: L2 Cache: 1024K (64 bytes/line) > CPU: AMD Opteron(tm) Processor 246 stepping 08 checking if > image is initramfs... it is > NET: Registered protocol family 16 > PCI: Using configuration type 1 > ACPI: Subsystem revision 20050309 > ACPI: Interpreter enabled > ACPI: Using IOAPIC for interrupt routing > ACPI: PCI Root Bridge [PCI0] (0000:00) > PCI: Probing PCI hardware (bus 00) > ACPI: PCI Interrupt Link [LNKA] (IRQs 3 *5 10 11) > ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 10 *11) > ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 *10 11) > ACPI: PCI Interrupt Link [LNKD] (IRQs 3 5 10 *11) > ACPI: PCI Root Bridge [PCI1] (0000:08) > PCI: Probing PCI hardware (bus 08) > xen_mem: Initialising balloon driver. > SCSI subsystem initialized > PCI: Using ACPI for IRQ routing > PCI: If a device doesn''t work, try "pci=routeirq". If it > helps, post a report Grant table initialized > IA-32 Microcode Update Driver: v1.14 <tigran@veritas.com> > IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak > Exp $ Initializing Cryptographic API > PCI: MSI quirk detected. pci_msi_quirk set. > PCI: MSI quirk detected. pci_msi_quirk set. > PCI: MSI quirk detected. pci_msi_quirk set. > PCI: MSI quirk detected. pci_msi_quirk set. > Real Time Clock Driver v1.12 > serio: i8042 AUX port at 0x60,0x64 irq 12 > serio: i8042 KBD port at 0x60,0x64 irq 1 io scheduler noop > registered io scheduler anticipatory registered io scheduler > deadline registered io scheduler cfq registered > floppy0: no floppy controllers found > RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize > loop: loaded (max 8 devices) > HP CISS Driver (v 2.6.6) > Intel(R) PRO/1000 Network Driver - version 6.0.54-k2 > Copyright (c) 1999-2004 Intel Corporation. > pcnet32.c:v1.30j 29.04.2005 tsbogend@alpha.franken.de > e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI > e100: Copyright(c) 1999-2005 Intel Corporation > tg3.c:v3.31 (June 8, 2005) > ACPI: PCI Interrupt 0000:03:02.0[A] -> GSI 28 (level, low) -> IRQ 28 > (XEN) (file=dom_mem_ops.c, line=61) Could not allocate a frame > ----------- [cut here ] --------- [please bite here ] > --------- Kernel BUG at "arch/xen/x86_64/mm/hypervisor.c":299 > invalid operand: 0000 [1] > CPU 0 > Modules linked in: > Pid: 1, comm: swapper Not tainted 2.6.12.2-xen0 > RIP: e030:[<ffffffff8011a32a>] > <ffffffff8011a32a>{xen_contig_memory+586} > RSP: e02b:ffff880002ee1bf8 EFLAGS: 00010296 > RAX: 0000000000000000 RBX: 000000000000000c RCX: ffffffff8011a324 > RDX: 0000000000000001 RSI: ffff880002ee1c28 RDI: 00000000ffffffff > RBP: 0000000000000001 R08: 0000000000007ff0 R09: 0000000000000001 > R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000 > R13: 0000000000000002 R14: 0000000000000001 R15: ffff800000000000 > FS: 0000000000000000(0000) GS:ffffffff80533a80(0000) > knlGS:0000000000000000 > CS: e033 DS: 0000 ES: 0000 > Process swapper (pid: 1, threadinfo ffff880002ee0000, task > ffff880002edf410) > Stack: 000000001002029a ffff880078cc6000 0000000202edf410 > 0000000052aa4d54 > 0000000000000000 ffff880078cc4000 0000000000044a89 > 0000000000000001 > ffff880078cc4000 0000000000000000 Call > Trace:<ffffffff80114920>{dma_alloc_coherent+224} > <ffffffff802a8096>{tg3_init_one+7302} > <ffffffff803bdafa>{wait_for_completion+170} > <ffffffff80126dd0>{default_wake_function+0} > <ffffffff802212d6>{pci_device_probe+134} > <ffffffff8025ac0d>{driver_probe_device+77} > <ffffffff8025ad2b>{driver_attach+75} > <ffffffff8025b118>{bus_add_driver+152} > <ffffffff80220fe5>{pci_register_driver+117} > <ffffffff801090f2>{init+178} > <ffffffff8010e567>{child_rip+8} <ffffffff80109040>{init+0} > <ffffffff8010e55f>{child_rip+0} > > Code: 0f 0b 08 91 3d 80 ff ff ff ff 2b 01 b8 01 00 00 00 44 > 89 f1 RIP <ffffffff8011a32a>{xen_contig_memory+586} RSP > <ffff880002ee1bf8> <0>Kernel panic - not syncing: Attempted > to kill init! > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Chris Wright
2005-Jul-28 21:07 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
* Ian Pratt (m+Ian.Pratt@cl.cam.ac.uk) wrote:> > > During boot of dom0, I get the following. Box is remote, so > > I''m now waiting for proxy power button pusher ;-) tg3 init > > fails doing dma_alloc_coherent (I assume this is from > > tg3_init_one -> tg3_test_dma -> pci_alloc_consistent -> > > dma_alloc_coherent). Looks like an issue with the amount of > > memory in dom0. Is this known? > > Is this with a debug=y build of Xen? I can see how that might provoke > the situation more than other builds.Yes, it is.> Do you know how large a contiguous chunk the tg3 needs?I believe that one was just 2 pages.> I guess we could hold onto a few large chunks before making the dom0 > memory allocation. > > Alternatively, if we get a failure, we should round up to the next power > of two and then balloon out that much memory. Repeat until success...I guess that''d be quick that early in boot. thanks, -chris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt
2005-Jul-28 21:25 UTC
RE: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
> > I guess we could hold onto a few large chunks before making > the dom0 > > memory allocation. > > I''m surprised that the mfns are already fragmented that early in boot. > Or maybe this case is just an example of bad luck?debug builds go out of their way to fragment things -- its proved useful for catching drivers that are taking liberties. Maybe not too smart when used in conjunction with dom0 grabbing all of memory... It would be good to confirm that things are OK with non debug builds, but I still don''t think it''s a good idea to rely on this. Let''s hold a couple of MB back from dom0 to avoid problems. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Gerd Knorr
2005-Jul-29 08:33 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
"Scott Parish" <srparish@us.ibm.com> writes:> I''m surprised that the mfns are already fragmented that early in boot. > Or maybe this case is just an example of bad luck?debug=y assigns the pages in reversed order (grep for REVERSE_START in arch/x86/domain_build.c). Gerd -- panic("it works"); /* avoid being flooded with debug messages */ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2005-Jul-29 10:03 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
On 28 Jul 2005, at 20:36, Chris Wright wrote:> During boot of dom0, I get the following. Box is remote, so I''m > now waiting for proxy power button pusher ;-) tg3 init fails doing > dma_alloc_coherent (I assume this is from tg3_init_one -> tg3_test_dma > -> > pci_alloc_consistent -> dma_alloc_coherent). Looks like an issue with > the amount of memory in dom0. Is this known?Allocate less memory to dom0 -- just a few mb less should make the allocation succeed. The failure path in alloc_coherent should invoke the balloon driver rather than barfing immediately. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2005-Jul-29 10:07 UTC
Re: [Xen-devel] dma_alloc_coherent issue with tg3 in x86_64 build
On 28 Jul 2005, at 22:25, Ian Pratt wrote:>> I''m surprised that the mfns are already fragmented that early in boot. >> Or maybe this case is just an example of bad luck? > > debug builds go out of their way to fragment things -- its proved > useful > for catching drivers that are taking liberties. Maybe not too smart > when > used in conjunction with dom0 grabbing all of memory... > > It would be good to confirm that things are OK with non debug builds, > but I still don''t think it''s a good idea to rely on this. Let''s hold a > couple of MB back from dom0 to avoid problems.It only reverses the page order in domain0 -- so debug build should have no worse behaviour (Xen ought to merge the reversed pages together when domain0 frees the previously-mapped pages). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel