Ext3 users - Nov 2006 - ext3 corrupted

Hi there,

 For years I've been using the ext3 file system without to think that it can
ever gets broken so bad.
This was until last week when a box that I have running Linux from a SanDisk CF
went down.
Since then I am struggling with this CF trying to understand what is happening.

 The CF is SanDisk ultra II 1GB.
  On this I have 4 partitions all of them with ext3:
   boot
   /
   swap
   data

 On "data" partition I am doing a kind of logging. On power failure
the box went down and never came
back.
  The problem is of course the data partition. 

running e2fsck /dev/sdc8 returns:

e2fsck 1.38 (30-Jun-2005)
/dev/sdc8: Attempt to read block from filesystem resulted in short read while
reading block 275
/dev/sdc8: Attempt to read block from filesystem resulted in short read reading
journal superblock
e2fsck: Attempt to read block from file-system resulted in short read while
checking ext3 journal for /dev/sdc8


which looks like being the result of the write access on power failure.

Then I did:

mke2fs -n /dev/sdc8:

mke2fs 1.38 (30-Jun-2005)
Filesystem labelOS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
108544 inodes, 433940 blocks
21697 blocks (5.00%) reserved for the super user
First data block=1
53 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
        8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409


I performed than the fsck.ext3 using the backup superblocks (e2fsck -c -b
<superblock_bckp> /dev/sdc8) but I got constantly the result:

e2fsck 1.38 (30-Jun-2005)
/dev/sdc8: Attempt to read block from filesystem resulted in short read while
reading block 275
/dev/sdc8: Attempt to read block from filesystem resulted in short read reading
journal superblock
e2fsck: Attempt to read block from filesystem resulted in short read while
checking ext3 journal for /dev/sdc8


The disappointing thing is that a windows application (Nucleus Kernel Linux)
shows the file system
and recover completely the partition in very short time.

 There is any suggestion about what should I do?

 Best regards,
 Bogdan.





__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/ext3-users/attachments/20061115/e50b3ba4/attachment.htm>

On Wed, 15 Nov 2006, Bogdan Scintee wrote:
> e2fsck 1.38 (30-Jun-2005)
If you can, please upgrade to a current version of e2fsprogs:
http://sourceforge.net/project/showfiles.php?group_id=2406
> /dev/sdc8: Attempt to read block from filesystem resulted in short read
while reading block 275
> /dev/sdc8: Attempt to read block from filesystem resulted in short read
reading journal superblock
> e2fsck: Attempt to read block from file-system resulted in short read while
checking ext3 journal for /dev/sdc8
Please check your system logfiles/dmesg for disk/IO errors.

If there are any, your best bet is to save the raw data to a save place 
(with "dd" or better "dd_rescue") and try e2fsck again on
the saved
image, eg.:

# dd_rescue if=/dev/your_CF_disk of=~/not-on-your-CF-disk/CF.img
# e2fsck ~/not-on-your-CF-disk/CF.img

...and see what it gives.

Christian.
-- 
BOFH excuse #413:

Cow-tippers tipped a cow onto the server.

[ please reply on-list, so that others can help too ]


On Thu, 16 Nov 2006, Bogdan Scintee wrote:
> Buffer I/O error on device sdc8, logical block 512
> sd 4:0:0:0: SCSI error: return code = 0x8000002
> sdc: Current: sense key: Medium Error
>    Additional sense: Unrecovered read error
> end_request: I/O error, dev sdc, sector 1134514
So, it really is the device generating the errors, the filesystem can't 
do much here :(
> I got finally a img file. Running the e2fsck on this image I got
>
> e2fsprogs-1.39/e2fsck/e2fsck -v -d sdc8.img
Very good, but I forgot one thing: if you have enough space, make a 
backup of sdc8.img, then try e2fsck. If e2fsck screws up, you still have 
the backup, even if the device (your CF card) fails completly.
> e2fsck 1.39 (29-May-2006)
> Superblock has an invalid ext3 journal (inode 8).
> Clear<y>? yes
> *** ext3 journal has been deleted - filesystem is now ext2 only ***
> Superblock doesn't have has_journal flag, but has ext3 journal inode.
> Clear<y>? yes
> sdc8.img was not cleanly unmounted, check forced.
> Pass 1: Checking inodes, blocks, and sizes
> Journal inode is not in use, but contains data.  Clear<y>? yes
> Pass 2: Checking directory structure
> Directory inode 2, block 0, offset 0: directory corrupted
> Salvage<y>? yes
> Missing '.' in directory inode 2.
> Fix<y>? yes
> Setting filetype for entry '.' in ??? (2) to 2.
> Missing '..' in directory inode 2.
> Fix<y>? yes
> Setting filetype for entry '..' in ??? (2) to 2.
> Pass 3: Checking directory connectivity
> '..' in / (2) is <The NULL inode> (0), should be / (2).
> Fix<y>? yes
> Unconnected directory inode 4097 (/???)
> Connect to /lost+found<y>? yes
> /lost+found not found.  Create<y>? yes
> Unconnected directory inode 61441 (/???)
> Connect to /lost+found<y>? yes
> Unconnected directory inode 8193 (/???)
> Connect to /lost+found<y>? yes
> Unconnected directory inode 28673 (/???)
> Connect to /lost+found<y>? yes
> Unconnected directory inode 30721 (/???)
> Connect to /lost+found<y>? yes
> Unconnected directory inode 57345 (/???)
> Connect to /lost+found<y>? yes
> Pass 4: Checking reference counts
> Inode 4097 ref count is 3, should be 2.  Fix<y>? yes
> Inode 8193 ref count is 5, should be 4.  Fix<y>? yes
> Inode 28673 ref count is 3, should be 2.  Fix<y>? yes
> Inode 30721 ref count is 7, should be 6.  Fix<y>? yes
> Inode 57345 ref count is 3, should be 2.  Fix<y>? yes
> Inode 61441 ref count is 9, should be 8.  Fix<y>? yes
> Pass 5: Checking group summary information
> Block bitmap differences:  -(276--8192) -(8454--8761)
> Fix<y>? yes
> Free blocks count wrong for group #0 (65535, counted=7917).
> Fix<y>? yes
> Free blocks count wrong for group #1 (4763, counted=5071).
> Fix<y>? yes
> Free blocks count wrong (285521, counted=293747).
> Fix<y>? yes
>
> sdc8.img: ***** FILE SYSTEM WAS MODIFIED *****
>    1051 inodes used (0%)
>     125 non-contiguous inodes (11.9%)
>         # of inodes with ind/dind/tind blocks: 405/68/0
>  140196 blocks used (32%)
>       0 bad blocks
>       0 large files
>     967 regular files
>      74 directories
>       0 character device files
>       0 block device files
>       0 fifos
>      -6 links
>       0 symbolic links (0 fast symbolic links)
>       0 sockets
> --------
>    1035 files
>
>  I tried different options during the recovery of the image but
>  unfortunately the result wasn't
> as expected. At the end part of the files where recovered but the
> parent folders names where replaced by
> inode number (I guess) and attached to lost+found directory.
>
>  I tried different options during the recovery of the image but
unfortunately the result wasn't
> as expected. At the end part of the files where recovered but the parent
folders names where replaced by
> inode number (I guess) and attached to lost+found directory.
Yes, when you had a lot of small files, even 1 GB of data in lost+found 
is a pain to reconstruct :(
> As I said in my previous e-mail the windoze application (Nucleus Kernel
Linux) is very fast and seems to recover
> more information from CF.
I too came across some win32 tools to recover b0rked filesystems: one is 
called "R-Linux", the other one "Stellar Phoenix Linux",
demos
available.
> I have a question now: The journal is kept only on the primary superblock
or it has also copies on every alternative
> superblock?
I don't know...
> My feeling is that the CF got a badblock exactly on the journal and the
e2fsck can't correct the information, therefore
> can't complete the job.
>
>  Do you have any knowledge about a application which is able to handle such
situation?
Dunno, maybe somebody else will have a look on this...

Christian.
-- 
BOFH excuse #453:

Spider infestation in warm case parts