Hi Everyone,
I have a server which has a raid array on /dev/sdb. After a crash I
tried to mount the array and it failed with:
mount: wrong fs type, bad option, bad superblock on /dev/sdb,
missing codepage or other error
dmesg reports:
EXT3-fs: invalid journal inode.
I tried to run e2fsck and it failed (The output can be seen below).
If anyone has any suggestions on how I can restore the filesystem I
would greatly appreciate the help.
Thanks in advance,
Sabin
*** BEGIN e2fsck output *****
root at 0[sabin]# e2fsck -b 32768 /dev/sdb
e2fsck 1.38 (30-Jun-2005)
Superblock has an invalid ext3 journal (inode 8).
Clear<y>? yes
*** ext3 journal has been deleted - filesystem is now ext2 only ***
Superblock doesn't have has_journal flag, but has ext3 journal inode.
Clear<y>? yes
/dev/sdb was not cleanly unmounted, check forced.
e2fsck: Illegal doubly indirect block found while reading bad blocks
inode
This doesn't bode well, but we'll try to go on...
Pass 1: Checking inodes, blocks, and sizes
Bad block inode has illegal block(s). Clear<y>? yes
Illegal block #9 (2933653514) in bad block inode. CLEARED.
Group 32's block bitmap (1048576) is bad. Relocate<y>? yes
Block 8 in the primary group descriptors is on the bad block list
If the block is really bad, the filesystem can not be fixed.
You can remove this block from the bad block list and hope
that the block is really OK. But there are no guarantees.
Clear<y>? yes
Bad block inode has an indirect block (1048577) that conflicts with
filesystem metadata. CLEARED.
Bad block inode has an indirect block (1048576) that conflicts with
filesystem metadata. CLEARED.
The bad block inode has probably been corrupted. You probably
should stop now and run e2fsck -c to scan for bad blocks
in the filesystem.
Continue<y>? no
e2fsck: aborted
root at 0[sabin]# e2fsck -c /dev/sdb
e2fsck 1.38 (30-Jun-2005)
Superblock has an invalid ext3 journal (inode 8).
Clear<y>? yes
*** ext3 journal has been deleted - filesystem is now ext2 only ***
Superblock doesn't have has_journal flag, but has ext3 journal inode.
Clear<y>? yes
Inode count in superblock is 151584768, should be 152633344.
Fix<y>? yes
ext2fs_block_iterate: Ext2 file too big while sanity checking the bad
blocks inode
On Tuesday 26 September 2006 01:09 pm, Sabin Dang wrote:> Hi Everyone, > > I have a server which has a raid array on /dev/sdb. After a crash I > tried to mount the array and it failed with: > > mount: wrong fs type, bad option, bad superblock on /dev/sdb, > missing codepage or other errorUmm.. there's no superblock (or filesystem) on /dev/sdb. Filesystems are on partitions.. such as /dev/sdb1 or sdb2. Not the raw device.> dmesg reports: > EXT3-fs: invalid journal inode. > > I tried to run e2fsck and it failed (The output can be seen below). > If anyone has any suggestions on how I can restore the filesystem I > would greatly appreciate the help.Ouch.. hope you didn't just hose your own filesystem. Just to be safe... boot from CD into rescue mode.. and try mounting it the PARTITIONS as ext2. If problems persist, nuke the remaining journal (if any) my mounting it as ext2 and/or using fsck.ext2. Once you get it clean at the ext2 level and can cleanly mount it.. recreate the journal with tune2fs -j: # tune2fs -j /dev/sdb2 (for example) Then try rembooting ans see how things go. Tweeks
On Tue, 26 Sep 2006, Sabin Dang wrote:> I have a server which has a raid array on /dev/sdb.So, is it a hardware raid or something? are there any device related messages in the syslog?> root at 0[sabin]# e2fsck -b 32768 /dev/sdb > e2fsck 1.38 (30-Jun-2005)A couple of questions here: - Do you really meant to check sdb, not sdb1 or sth.? - Why did you have to specify an alternative superblock here? Did it not run without -b? - Any chance you you could use a more current version of e2fsprogs (like 1.39 or so)? - Did you backup sdb *before* trying to fsck? Christian. -- BOFH excuse #110: The rolling stones concert down the road caused a brown out
(please post on-list, so that all ppl can read/reply to your mail) On Tue, 26 Sep 2006, Sabin Dang wrote:> It is a hardware raid (3ware, RAID 5 + hostspare ) and the raid reports all > drives are fine...and no errors in syslog then, I suppose. How big is the array?> I have a backup, but unfortunately the data on the raid is needed very > quickly to meet a deadline. Restoring from backups is possible but is time > consuming (a process I've started on another system already, just hoping to > get things up and running quickly).Sure. What I meant was: if you can (i.e. if you have another at least equal-sized disk(array)): dd your "bad" sdb to this other device as a backup, before attempting to play with fsck and "fsck -n" already tells you that the fs is severely damaged. That way you could play back the (nevertheless corrupt) original sdb if fsck is "fixing" more than needed. Christian. -- BOFH excuse #43: boss forgot system password