Alexander Farber
2011-Apr-28 19:07 UTC
[CentOS] Server offline :-( please help to repair software RAID
Hello,
since weeks I was ignoring this warning at my CentOS 5.6/64 bit machine -
/etc/cron.weekly/99-raid-check:
WARNING: mismatch_cnt is not 0 on /dev/md0
in the hope that the software RAID will slowly repair itself.
I also had executed "echo 100000 >
/proc/sys/dev/raid/speed_limit_max"
on the advice from the mailing list.
But now my web server is offline - I had to boot it remotely with rescue system.
Does anybody please have an advice what commands to run
and do you think it is a RAID problem at all?
# dmesg
Linux version 2.6.34 (root at imagemaster30) (gcc version 4.3.2 (Debian
4.3.2-1.1) ) #20 SMP Mon Jul 19 18:35:15 CEST 2010
Command line: ramdisk_size=81920 initrd=rescue-image-2.6-64
root=/dev/ram BOOT_IMAGE=rescue-kernel-2.6-64
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000ddfb0000 (usable)
BIOS-e820: 00000000ddfb0000 - 00000000ddfbe000 (ACPI data)
BIOS-e820: 00000000ddfbe000 - 00000000ddfe0000 (ACPI NVS)
BIOS-e820: 00000000ddfe0000 - 00000000ddfee000 (reserved)
BIOS-e820: 00000000ddff0000 - 00000000de000000 (reserved)
BIOS-e820: 00000000ff700000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
NX (Execute Disable) protection: active
DMI present.
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==>
(reserved)
e820 update range: 0000000000000000 - 0000000000001000 (usable) ==>
(reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
No AGP bridge found
last_pfn = 0x120000 max_arch_pfn = 0x400000000
MTRR default type: uncachable
MTRR fixed ranges enabled:
00000-9FFFF write-back
A0000-EFFFF uncachable
F0000-FFFFF write-protect
MTRR variable ranges enabled:
0 base 000000000000 mask FFFF80000000 write-back
1 base 000080000000 mask FFFFC0000000 write-back
2 base 0000C0000000 mask FFFFE0000000 write-back
3 disabled
4 disabled
5 disabled
6 disabled
7 disabled
TOM2: 0000000120000000 aka 4608M
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
e820 update range: 00000000e0000000 - 0000000100000000 (usable) ==>
(reserved)
last_pfn = 0xddfb0 max_arch_pfn = 0x400000000
initial memory mapped : 0 - 20000000
found SMP MP-table at [ffff8800000ff780] ff780
Using GB pages for direct mapping
init_memory_mapping: 0000000000000000-00000000ddfb0000
0000000000 - 00c0000000 page 1G
00c0000000 - 00dde00000 page 2M
00dde00000 - 00ddfb0000 page 4k
kernel direct mapping tables up to ddfb0000 @ 12000-15000
init_memory_mapping: 0000000100000000-0000000120000000
0100000000 - 0120000000 page 2M
kernel direct mapping tables up to 120000000 @ 14000-16000
RAMDISK: 7d792000 - 80000000
ACPI: RSDP 00000000000faf80 00014 (v00 ACPIAM)
ACPI: RSDT 00000000ddfb0000 0003C (v01 032510 RSDT1503 20100325 MSFT 00000097)
ACPI: FACP 00000000ddfb0200 00084 (v02 032510 FACP1503 20100325 MSFT 00000097)
ACPI: DSDT 00000000ddfb0440 0447E (v01 A96B3 A96B3210 00000210 INTL 20051117)
ACPI: FACS 00000000ddfbe000 00040
ACPI: APIC 00000000ddfb0390 0006C (v01 032510 APIC1503 20100325 MSFT 00000097)
ACPI: MCFG 00000000ddfb0400 0003C (v01 032510 OEMMCFG 20100325 MSFT 00000097)
ACPI: OEMB 00000000ddfbe040 00071 (v01 032510 OEMB1503 20100325 MSFT 00000097)
ACPI: HPET 00000000ddfb48c0 00038 (v01 032510 OEMHPET 20100325 MSFT 00000097)
ACPI: SSDT 00000000ddfb4900 0088C (v01 A M I POWERNOW 00000001 AMD 00000001)
ACPI: Local APIC address 0xfee00000
Scanning NUMA topology in Northbridge 24
No NUMA configuration found
Faking a node at 0000000000000000-0000000120000000
Initmem setup node 0 0000000000000000-0000000120000000
NODE_DATA [0000000100000000 - 0000000100004fff]
[ffffea0000000000-ffffea0003ffffff] PMD ->
[ffff880100200000-ffff880103bfffff] on node 0
Zone PFN ranges:
DMA 0x00000010 -> 0x00001000
DMA32 0x00001000 -> 0x00100000
Normal 0x00100000 -> 0x00120000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
0: 0x00000010 -> 0x0000009f
0: 0x00000100 -> 0x000ddfb0
0: 0x00100000 -> 0x00120000
On node 0 totalpages: 1040191
DMA zone: 56 pages used for memmap
DMA zone: 0 pages reserved
DMA zone: 3927 pages, LIFO batch:0
DMA32 zone: 14280 pages used for memmap
DMA32 zone: 890856 pages, LIFO batch:31
Normal zone: 1792 pages used for memmap
Normal zone: 129280 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8300 base: 0xfed00000
SMP: Allowing 4 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 24
early_res array is doubled to 64 at [15000 - 157ff]
Allocating PCI resources starting at de000000 (gap: de000000:21700000)
setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:4 nr_node_ids:1
PERCPU: Embedded 26 pages/cpu @ffff880001800000 s75304 r8192 d23000 u524288
pcpu-alloc: s75304 r8192 d23000 u524288 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3
Built 1 zonelists in Node order, mobility grouping on. Total pages: 1024063
Policy zone: Normal
Kernel command line: ramdisk_size=81920 initrd=rescue-image-2.6-64
root=/dev/ram BOOT_IMAGE=rescue-kernel-2.6-64
PID hash table entries: 4096 (order: 3, 32768 bytes)
Checking aperture...
No AGP bridge found
Node 0: aperture @ 449c000000 size 32 MB
Aperture beyond 4GB. Ignoring.
Your BIOS doesn't leave a aperture memory hole
Please enable the IOMMU option in the BIOS setup
This costs you 64 MB of RAM
Mapping aperture over 65536 KB of RAM @ 20000000
Subtract (54 early reservations)
#1 [0001000000 - 00017a7914] TEXT DATA BSS
#2 [007d792000 - 0080000000] RAMDISK
#3 [00017a8000 - 00017a8175] BRK
#4 [00000ff790 - 0000100000] BIOS reserved
#5 [00000ff780 - 00000ff790] MP-table mpf
#6 [000009ec00 - 00000e3e10] BIOS reserved
#7 [00000e3f7c - 00000ff780] BIOS reserved
#8 [00000e3e10 - 00000e3f7c] MP-table mpc
#9 [0000010000 - 0000012000] TRAMPOLINE
#10 [0000012000 - 0000014000] PGTABLE
#11 [0000014000 - 0000015000] PGTABLE
#12 [0100000000 - 0100005000] NODE_DATA
#13 [00017a8180 - 00017a9180] BOOTMEM
#14 [00017a7940 - 00017a7c40] BOOTMEM
#15 [0100005000 - 0100006000] BOOTMEM
#16 [0100006000 - 0100007000] BOOTMEM
#17 [0100200000 - 0103c00000] MEMMAP 0
#18 [00017a7c40 - 00017a7dc0] BOOTMEM
#19 [00017a9180 - 00017c1180] BOOTMEM
#20 [00017c1180 - 00017c4180] BOOTMEM
#21 [00017c5000 - 00017c6000] BOOTMEM
#22 [00017a7dc0 - 00017a7e01] BOOTMEM
#23 [00017a7e40 - 00017a7e83] BOOTMEM
#24 [00017c4180 - 00017c43e8] BOOTMEM
#25 [00017a7ec0 - 00017a7f28] BOOTMEM
#26 [00017a7f40 - 00017a7fa8] BOOTMEM
#27 [00017c4400 - 00017c4468] BOOTMEM
#28 [00017c4480 - 00017c44e8] BOOTMEM
#29 [00017c4500 - 00017c4568] BOOTMEM
#30 [00017c4580 - 00017c45e8] BOOTMEM
#31 [00017c4600 - 00017c4668] BOOTMEM
#32 [00017c4680 - 00017c46e8] BOOTMEM
#33 [00017c4700 - 00017c4768] BOOTMEM
#34 [00017c4780 - 00017c47e8] BOOTMEM
#35 [00017c4800 - 00017c485d] BOOTMEM
#36 [00017c4880 - 00017c48dd] BOOTMEM
#37 [0001800000 - 000181a000] BOOTMEM
#38 [0001880000 - 000189a000] BOOTMEM
#39 [0001900000 - 000191a000] BOOTMEM
#40 [0001980000 - 000199a000] BOOTMEM
#41 [00017a7fc0 - 00017a7fc8] BOOTMEM
#42 [00017c4900 - 00017c4908] BOOTMEM
#43 [00017c4940 - 00017c4950] BOOTMEM
#44 [00017c4980 - 00017c49a0] BOOTMEM
#45 [00017c49c0 - 00017c4af0] BOOTMEM
#46 [00017c4b00 - 00017c4b50] BOOTMEM
#47 [00017c4b80 - 00017c4bd0] BOOTMEM
#48 [00017c6000 - 00017ce000] BOOTMEM
#49 [0020000000 - 0024000000] BOOTMEM
#50 [000199a000 - 000599a000] BOOTMEM
#51 [00017ce000 - 00017ee000] BOOTMEM
#52 [000181a000 - 000185a000] BOOTMEM
#53 [0000015800 - 000001d800] BOOTMEM
Memory: 3920020k/4718592k available (4511k kernel code, 557828k
absent, 240744k reserved, 2194k data, 636k init)
Hierarchical RCU implementation.
NR_IRQS:1280
Console: colour VGA+ 80x25
console [tty0] enabled
hpet clockevent registered
Fast TSC calibration failed
TSC: Unable to calibrate against PIT
TSC: using HPET reference calibration
Detected 2500.084 MHz processor.
Calibrating delay loop (skipped), value calculated using timer
frequency.. 5000.16 BogoMIPS (lpj=10000324)
Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Mount-cache hash table entries: 256
tseg: 0000000000
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 6 MCE banks
Performance Events: AMD PMU driver.
... version: 0
... bit width: 48
... generic registers: 4
... value mask: 0000ffffffffffff
... max period: 00007fffffffffff
... fixed-purpose events: 0
... event mask: 000000000000000f
Freeing SMP alternatives: 52k freed
ACPI: Core revision 20100121
Setting APIC routing to flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: Quad-Core AMD Opteron(tm) Processor 1381 stepping 02
Booting Node 0, Processors #1 #2 #3 Ok.
Brought up 4 CPUs
Total of 4 processors activated (20000.75 BogoMIPS).
NET: Registered protocol family 16
node 0 link 0: io port [1000, ffffff]
TOM: 00000000e0000000 aka 3584M
Fam 10h mmconf [e0000000, efffffff]
node 0 link 0: mmio [fc000000, fdfeffff]
node 0 link 0: mmio [e0000000, efffffff] ==> none
node 0 link 0: mmio [a0000, bffff]
node 0 link 0: mmio [fdff0000, ffffffff]
TOM2: 0000000120000000 aka 4608M
bus: [00, 07] on node 0 link 0
bus: 00 index 0 [io 0x0000-0xffff]
bus: 00 index 1 [mem 0xf0000000-0xffffffff]
bus: 00 index 2 [mem 0x000a0000-0x000bffff]
bus: 00 index 3 [mem 0x120000000-0xfcffffffff]
ACPI: bus type pci registered
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem
0xe0000000-0xefffffff] (base 0xe0000000)
PCI: not using MMCONFIG
PCI: Using configuration type 1 for base access
PCI: Using configuration type 1 for extended access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Executed 2 blocks of module-level executable AML code
ACPI: Interpreter enabled
ACPI: (supports S0 S5)
ACPI: Using IOAPIC for interrupt routing
PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem
0xe0000000-0xefffffff] (base 0xe0000000)
PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI
motherboard resources
ACPI Warning: Incorrect checksum in table [OEMB] - 00, should be FD
(20100121/tbutils-314)
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use
"pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (0000:00)
pci_root PNP0A03:00: host bridge window [io 0x0000-0x0cf7]
pci_root PNP0A03:00: host bridge window [io 0x0d00-0xffff]
pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A03:00: host bridge window [mem 0x000d0000-0x000dffff]
pci_root PNP0A03:00: host bridge window [mem 0xde000000-0xdfffffff]
pci_root PNP0A03:00: host bridge window [mem 0xf0000000-0xfebfffff]
pci 0000:00:04.0: PME# supported from D0 D3hot D3cold
pci 0000:00:04.0: PME# disabled
pci 0000:00:05.0: PME# supported from D0 D3hot D3cold
pci 0000:00:05.0: PME# disabled
pci 0000:00:12.0: reg 10: [io 0xb000-0xb007]
pci 0000:00:12.0: reg 14: [io 0xa000-0xa003]
pci 0000:00:12.0: reg 18: [io 0x9000-0x9007]
pci 0000:00:12.0: reg 1c: [io 0x8000-0x8003]
pci 0000:00:12.0: reg 20: [io 0x7000-0x700f]
pci 0000:00:12.0: reg 24: [mem 0xfe7ff800-0xfe7ffbff]
pci 0000:00:12.0: set SATA to AHCI mode
pci 0000:00:13.0: reg 10: [mem 0xfe7fe000-0xfe7fefff]
pci 0000:00:13.1: reg 10: [mem 0xfe7fd000-0xfe7fdfff]
pci 0000:00:13.2: reg 10: [mem 0xfe7fc000-0xfe7fcfff]
pci 0000:00:13.3: reg 10: [mem 0xfe7fb000-0xfe7fbfff]
pci 0000:00:13.4: reg 10: [mem 0xfe7fa000-0xfe7fafff]
pci 0000:00:13.5: reg 10: [mem 0xfe7ff000-0xfe7ff0ff]
pci 0000:00:13.5: supports D1 D2
pci 0000:00:13.5: PME# supported from D0 D1 D2 D3hot
pci 0000:00:13.5: PME# disabled
pci 0000:00:14.0: reg 10: [io 0x0b00-0x0b0f]
pci 0000:00:14.0: reg 14: [mem 0x8000a010-0x8000a01f 64bit]
pci 0000:00:14.1: reg 10: [io 0x0000-0x0007]
pci 0000:00:14.1: reg 14: [io 0x0000-0x0003]
pci 0000:00:14.1: reg 18: [io 0x0000-0x0007]
pci 0000:00:14.1: reg 1c: [io 0x0000-0x0003]
pci 0000:00:14.1: reg 20: [io 0xff00-0xff0f]
pci 0000:01:05.0: reg 10: [mem 0xfc000000-0xfdffffff 64bit pref]
pci 0000:01:05.0: reg 18: [mem 0xfe9f0000-0xfe9fffff 64bit]
pci 0000:01:05.0: reg 20: [io 0xc000-0xc0ff]
pci 0000:01:05.0: reg 24: [mem 0xfe800000-0xfe8fffff]
pci 0000:01:05.0: supports D1 D2
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0: bridge window [io 0xc000-0xcfff]
pci 0000:00:01.0: bridge window [mem 0xfe800000-0xfe9fffff]
pci 0000:00:01.0: bridge window [mem 0xfc000000-0xfdffffff 64bit pref]
pci 0000:02:00.0: reg 10: [mem 0xfeae0000-0xfeafffff]
pci 0000:02:00.0: reg 18: [io 0xd800-0xd81f]
pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
pci 0000:02:00.0: PME# disabled
pci 0000:00:04.0: PCI bridge to [bus 02-02]
pci 0000:00:04.0: bridge window [io 0xd000-0xdfff]
pci 0000:00:04.0: bridge window [mem 0xfea00000-0xfeafffff]
pci 0000:00:04.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:03:00.0: reg 10: [mem 0xfebe0000-0xfebfffff]
pci 0000:03:00.0: reg 18: [io 0xe800-0xe81f]
pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
pci 0000:03:00.0: PME# disabled
pci 0000:00:05.0: PCI bridge to [bus 03-03]
pci 0000:00:05.0: bridge window [io 0xe000-0xefff]
pci 0000:00:05.0: bridge window [mem 0xfeb00000-0xfebfffff]
pci 0000:00:05.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:00:14.4: PCI bridge to [bus 04-04] (subtractive decode)
pci 0000:00:14.4: bridge window [io 0xf000-0x0000] (disabled)
pci 0000:00:14.4: bridge window [mem 0xfff00000-0x000fffff] (disabled)
pci 0000:00:14.4: bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7] (subtractive decode)
pci 0000:00:14.4: bridge window [io 0x0d00-0xffff] (subtractive decode)
pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000bffff]
(subtractive decode)
pci 0000:00:14.4: bridge window [mem 0x000d0000-0x000dffff]
(subtractive decode)
pci 0000:00:14.4: bridge window [mem 0xde000000-0xdfffffff]
(subtractive decode)
pci 0000:00:14.4: bridge window [mem 0xf0000000-0xfebfffff]
(subtractive decode)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCE4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCE5._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0PC._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 9) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 7 10 11 12 14 15) *0, disabled.
vgaarb: device added: PCI:0000:01:05.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
SCSI subsystem initialized
libata version 3.00 loaded.
PCI: Using ACPI for IRQ routing
PCI: pci_cache_line_size set to 64 bytes
pci 0000:00:14.0: no compatible bridge window for [mem
0x8000a010-0x8000a01f 64bit]
reserve RAM buffer: 000000000009f000 - 000000000009ffff
reserve RAM buffer: 00000000ddfb0000 - 00000000dfffffff
Switching to clocksource tsc
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 14 devices
ACPI: ACPI bus type pnp unregistered
system 00:08: [mem 0xfec00000-0xfec00fff] could not be reserved
system 00:08: [mem 0xfee00000-0xfee00fff] has been reserved
system 00:09: [io 0x04d0-0x04d1] has been reserved
system 00:09: [io 0x040b] has been reserved
system 00:09: [io 0x04d6] has been reserved
system 00:09: [io 0x0c00-0x0c01] has been reserved
system 00:09: [io 0x0c14] has been reserved
system 00:09: [io 0x0c50-0x0c51] has been reserved
system 00:09: [io 0x0c52] has been reserved
system 00:09: [io 0x0c6c] has been reserved
system 00:09: [io 0x0c6f] has been reserved
system 00:09: [io 0x0cd0-0x0cd1] has been reserved
system 00:09: [io 0x0cd2-0x0cd3] has been reserved
system 00:09: [io 0x0cd4-0x0cd5] has been reserved
system 00:09: [io 0x0cd6-0x0cd7] has been reserved
system 00:09: [io 0x0cd8-0x0cdf] has been reserved
system 00:09: [io 0x0800-0x089f] has been reserved
system 00:09: [io 0x0b10-0x0b1f] has been reserved
system 00:09: [io 0x0900-0x090f] has been reserved
system 00:09: [io 0x0910-0x091f] has been reserved
system 00:09: [io 0xfe00-0xfefe] has been reserved
system 00:09: [mem 0xffb80000-0xffbfffff] has been reserved
system 00:0b: [io 0x0a00-0x0a0f] has been reserved
system 00:0b: [io 0x0a10-0x0a1f] has been reserved
system 00:0c: [mem 0xe0000000-0xefffffff] has been reserved
system 00:0d: [mem 0x00000000-0x0009ffff] could not be reserved
system 00:0d: [mem 0x000c0000-0x000cffff] has been reserved
system 00:0d: [mem 0x000e0000-0x000fffff] could not be reserved
system 00:0d: [mem 0x00100000-0xddffffff] could not be reserved
system 00:0d: [mem 0xfec00000-0xffffffff] could not be reserved
pci 0000:00:14.0: BAR 1: assigned [mem 0xde000000-0xde00000f 64bit]
pci 0000:00:14.0: BAR 1: error updating (0xde000004 != 0x8000a014)
pci 0000:00:14.0: BAR 1: set to [mem 0xde000000-0xde00000f 64bit] (PCI
address [0xde000000-0xde00000f]
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0: bridge window [io 0xc000-0xcfff]
pci 0000:00:01.0: bridge window [mem 0xfe800000-0xfe9fffff]
pci 0000:00:01.0: bridge window [mem 0xfc000000-0xfdffffff 64bit pref]
pci 0000:00:04.0: PCI bridge to [bus 02-02]
pci 0000:00:04.0: bridge window [io 0xd000-0xdfff]
pci 0000:00:04.0: bridge window [mem 0xfea00000-0xfeafffff]
pci 0000:00:04.0: bridge window [mem pref disabled]
pci 0000:00:05.0: PCI bridge to [bus 03-03]
pci 0000:00:05.0: bridge window [io 0xe000-0xefff]
pci 0000:00:05.0: bridge window [mem 0xfeb00000-0xfebfffff]
pci 0000:00:05.0: bridge window [mem pref disabled]
pci 0000:00:14.4: PCI bridge to [bus 04-04]
pci 0000:00:14.4: bridge window [io disabled]
pci 0000:00:14.4: bridge window [mem disabled]
pci 0000:00:14.4: bridge window [mem pref disabled]
pci 0000:00:04.0: setting latency timer to 64
pci 0000:00:05.0: setting latency timer to 64
pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7]
pci_bus 0000:00: resource 5 [io 0x0d00-0xffff]
pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: resource 7 [mem 0x000d0000-0x000dffff]
pci_bus 0000:00: resource 8 [mem 0xde000000-0xdfffffff]
pci_bus 0000:00: resource 9 [mem 0xf0000000-0xfebfffff]
pci_bus 0000:01: resource 0 [io 0xc000-0xcfff]
pci_bus 0000:01: resource 1 [mem 0xfe800000-0xfe9fffff]
pci_bus 0000:01: resource 2 [mem 0xfc000000-0xfdffffff 64bit pref]
pci_bus 0000:02: resource 0 [io 0xd000-0xdfff]
pci_bus 0000:02: resource 1 [mem 0xfea00000-0xfeafffff]
pci_bus 0000:03: resource 0 [io 0xe000-0xefff]
pci_bus 0000:03: resource 1 [mem 0xfeb00000-0xfebfffff]
pci_bus 0000:04: resource 4 [io 0x0000-0x0cf7]
pci_bus 0000:04: resource 5 [io 0x0d00-0xffff]
pci_bus 0000:04: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:04: resource 7 [mem 0x000d0000-0x000dffff]
pci_bus 0000:04: resource 8 [mem 0xde000000-0xdfffffff]
pci_bus 0000:04: resource 9 [mem 0xf0000000-0xfebfffff]
NET: Registered protocol family 2
IP route cache hash table entries: 131072 (order: 8, 1048576 bytes)
TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
UDP hash table entries: 2048 (order: 4, 65536 bytes)
UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
NET: Registered protocol family 1
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
pci 0000:01:05.0: Boot video device
PCI: CLS 64 bytes, default 64
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 41400k freed
PCI-DMA: Disabling AGP.
PCI-DMA: aperture base @ 20000000 size 65536 KB
PCI-DMA: using GART IOMMU.
PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
HugeTLB registered 2 MB page size, pre-allocated 0 pages
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Installing knfsd (copyright (C) 1996 okir at monad.swb.de).
NTFS driver 2.1.29 [Flags: R/W].
fuse init (API version 7.13)
JFS: nTxBlock = 8192, nTxLock = 65536
SGI XFS with security attributes, large block/inode numbers, no debug enabled
aufs 2-standalone.tree-34-20100621
msgmni has been set to 7866
alg: No test for stdrng (krng)
io scheduler noop registered
io scheduler cfq registered (default)
pcieport 0000:00:04.0: setting latency timer to 64
pcieport 0000:00:04.0: irq 24 for MSI/MSI-X
pcieport 0000:00:05.0: setting latency timer to 64
pcieport 0000:00:05.0: irq 25 for MSI/MSI-X
input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
ACPI: duty_cycle spans bit 4
Real Time Clock Driver v1.12b
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
brd: module loaded
loop: module loaded
drbd: initialized. Version: 8.3.7 (api:88/proto:86-92)
drbd: built-in
drbd: registered as block device major 147
drbd: minor_table @ 0xffff88011bd1a380
3ware Storage Controller device driver for Linux v1.26.02.002.
ahci 0000:00:12.0: version 3.0
ahci 0000:00:12.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
ahci 0000:00:12.0: controller can't do 64bit DMA, forcing 32bit
ahci 0000:00:12.0: AHCI 0001.0100 32 slots 4 ports 3 Gbps 0xf impl SATA mode
ahci 0000:00:12.0: flags: ncq sntf ilck pm led clo pmp pio slum part ccc
scsi0 : ahci
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
ata1: SATA max UDMA/133 irq_stat 0x00400040, connection status changed
ata2: SATA max UDMA/133 irq_stat 0x00400040, connection status changed
ata3: SATA max UDMA/133 abar m1024 at 0xfe7ff800 port 0xfe7ffa00 irq 22
ata4: SATA max UDMA/133 abar m1024 at 0xfe7ff800 port 0xfe7ffa80 irq 22
Intel(R) PRO/1000 Network Driver - version 7.3.21-k5-NAPI
Copyright (c) 1999-2006 Intel Corporation.
e1000e: Intel(R) PRO/1000 Network Driver - 1.0.2-k2
e1000e: Copyright (c) 1999 - 2009 Intel Corporation.
e1000e 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
e1000e 0000:02:00.0: setting latency timer to 64
e1000e 0000:02:00.0: irq 26 for MSI/MSI-X
e1000e 0000:02:00.0: Disabling ASPM L0s
0000:02:00.0: eth0: (PCI Express:2.5GB/s:Width x1) 40:61:86:ee:20:46
0000:02:00.0: eth0: Intel(R) PRO/1000 Network Connection
0000:02:00.0: eth0: MAC: 2, PHY: 2, PBA No: ffffff-0ff
e1000e 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
e1000e 0000:03:00.0: setting latency timer to 64
e1000e 0000:03:00.0: irq 27 for MSI/MSI-X
e1000e 0000:03:00.0: Disabling ASPM L0s
0000:03:00.0: eth1: (PCI Express:2.5GB/s:Width x1) 40:61:86:ee:20:47
0000:03:00.0: eth1: Intel(R) PRO/1000 Network Connection
0000:03:00.0: eth1: MAC: 2, PHY: 2, PBA No: ffffff-0ff
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk at qualcomm.com>
PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
PNP: PS/2 appears to have AUX port disabled, if this is incorrect
please boot with i8042.nopnp
serio: i8042 KBD port at 0x60,0x64 irq 1
mice: PS/2 mouse device common for all mice
input: PC Speaker as /devices/platform/pcspkr/input/input2
i2c /dev entries driver
piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
ata3: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
k10temp 0000:00:18.3: unreliable CPU thermal sensor; monitoring disabled
w83627ehf: Found W83627DHG chip at 0xa10
Software Watchdog Timer: 0.07 initialized. soft_noboot=0
soft_margin=60 sec (nowayout= 0)
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
device-mapper: ioctl: 4.17.0-ioctl (2010-03-05) initialised: dm-devel at
redhat.com
cpuidle: using governor ladder
TCP cubic registered
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
sit0: Disabled Privacy Extensions
ip6tnl0: Disabled Privacy Extensions
NET: Registered protocol family 17
powernow-k8: Found 1 Quad-Core AMD Opteron(tm) Processor 1381
processors (4 cpu cores) (version 2.20.00)
powernow-k8: 0 : pstate 0 (2500 MHz)
powernow-k8: 1 : pstate 1 (1800 MHz)
powernow-k8: 2 : pstate 2 (1300 MHz)
powernow-k8: 3 : pstate 3 (800 MHz)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ATA-8: Hitachi HDS721050CLA362, JP2OA39C, max UDMA/133
ata1.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata1.00: SB600 AHCI: limiting to 255 sectors per cmd
ata1.00: SB600 AHCI: limiting to 255 sectors per cmd
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access ATA Hitachi HDS72105 JP2O PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't
support DPO or FUA
sda: sda1 sda2 sda3 sda4 <
ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata2.00: ATA-8: Hitachi HDS721050CLA362, JP2OA39C, max UDMA/133
ata2.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata2.00: SB600 AHCI: limiting to 255 sectors per cmd
sda5
ata2.00: SB600 AHCI: limiting to 255 sectors per cmd
ata2.00: configured for UDMA/133
scsi 1:0:0:0: Direct-Access ATA Hitachi HDS72105 JP2O PQ: 0 ANSI: 5
sd 1:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 1:0:0:0: [sdb] Write Protect is off
sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't
support DPO or FUA
sdb: sdb1 sdb2 sdb3 sdb4 < sda6 >
sd 0:0:0:0: [sda] Attached SCSI disk
sdb5 sdb6 >
sd 1:0:0:0: [sdb] Attached SCSI disk
Freeing unused kernel memory: 636k freed
aufs test_add:252:busybox[1175]: uid/gid/perm /squash 0/0/0755, 0/0/01777
md: bind<sdb6>
md: bind<sda6>
raid1: md3 is not clean -- starting background reconstruction
raid1: raid set md3 active with 2 out of 2 mirrors
md3: detected capacity change from 0 to 189594992640
md: resync of RAID array md3
md3:
md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
unknown partition table
md: using maximum available idle IO bandwidth (but not more than
200000 KB/sec) for resync.
md: using 128k window, over a total of 185151360 blocks.
md: bind<sdb5>
md: bind<sda5>
raid1: raid set md2 active with 2 out of 2 mirrors
md2: detected capacity change from 0 to 284393668608
md2: unknown partition table
md: bind<sdb3>
md: bind<sda3>
raid1: md1 is not clean -- starting background reconstruction
raid1: raid set md1 active with 2 out of 2 mirrors
md1: detected capacity change from 0 to 20971454464
md1: unknown partition table
md: delaying resync of md1 until md3 has finished (they share one or
more physical units)
md: bind<sdb1>
md: bind<sda1>
raid1: raid set md0 active with 2 out of 2 mirrors
md0: detected capacity change from 0 to 1048510464
md0: unknown partition table
e1000e 0000:02:00.0: irq 26 for MSI/MSI-X
e1000e 0000:02:00.0: irq 26 for MSI/MSI-X
ADDRCONF(NETDEV_UP): eth0: link is not ready
e1000e: eth0 NIC Link is Up 100 Mbps Full Duplex, Flow Control: None
0000:02:00.0: eth0: 10/100 speed: disabling TSO
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Loading iSCSI transport class v2.0-870.
iscsi: registered transport (tcp)
# mdadm -D /dev/md0
/dev/md0:
Version : 00.90
Creation Time : Sat Mar 19 22:52:12 2011
Raid Level : raid1
Array Size : 1023936 (1000.11 MiB 1048.51 MB)
Used Dev Size : 1023936 (1000.11 MiB 1048.51 MB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Wed Apr 27 11:35:21 2011
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 87db17c2:d806a38c:3d186b3c:53958f34
Events : 0.14
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
Thank you
Alex
Alexander Farber
2011-Apr-28 19:10 UTC
[CentOS] Server offline :-( please help to repair software RAID
Additional info (how many RAID arrays do I have??):
# mdadm -D /dev/md3
/dev/md3:
Version : 00.90
Creation Time : Sat Mar 19 22:53:25 2011
Raid Level : raid1
Array Size : 185151360 (176.57 GiB 189.59 GB)
Used Dev Size : 185151360 (176.57 GiB 189.59 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 3
Persistence : Superblock is persistent
Update Time : Thu Apr 28 21:09:12 2011
State : clean, resyncing
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Rebuild Status : 38% complete
UUID : 1b3668a3:4b6c5593:3d186b3c:53958f34
Events : 0.15
Number Major Minor RaidDevice State
0 8 6 0 active sync /dev/sda6
1 8 22 1 active sync /dev/sdb6
Les Mikesell
2011-Apr-28 19:28 UTC
[CentOS] Server offline :-( please help to repair software RAID
On 4/28/2011 2:07 PM, Alexander Farber wrote:> Hello, > > since weeks I was ignoring this warning at my CentOS 5.6/64 bit machine - > > /etc/cron.weekly/99-raid-check: > WARNING: mismatch_cnt is not 0 on /dev/md0 > > in the hope that the software RAID will slowly repair itself. > > I also had executed "echo 100000> /proc/sys/dev/raid/speed_limit_max" > on the advice from the mailing list. > > But now my web server is offline - I had to boot it remotely with rescue system. > > Does anybody please have an advice what commands to run > and do you think it is a RAID problem at all? >A 'cat /proc/mdstat' should show the state of the raid mirroring. I don't see anything that would explain not booting, though. Raid1 works normally even when only one member is available and should continue to work while rebuilding. Maybe the problem that caused the mismatch has corrupted the drive the system normally boots. -- Les Mikesell lesmikesell at gmail.com