Ext3 users - Mar 2007 - How To Recover From Creating >2TB ext3 Filesystem on MSDOS Partition Table?

(I've already sent this message to Ted Ts'o directly. I should
have sent it to this list first but I didn't know about it
until today. My apologies to Ted.)

Last Friday a system that I just inherited refused to mount
a file system that had been working fine for about 6 months.
This is on a Scientific Linux 4.3 system using a 2.6.9
kernel. This is another Linux distribution based on RHEL 4.
I don't think the actual hardware is relevant
here so I won't mention it. If there's more information you'd
like to see I'd be happy to provide it.

It turns out that this 4.2TB file system was created in an
msdos partition table, as shown below:

----
GNU Parted 1.6.19
Using /dev/sdb
(parted) p
Disk geometry for /dev/sdb: 0.000-4291443.000 megabytes
Disk label type: msdos
Minor    Start       End     Type      Filesystem  Flags
1          0.031  97137.567  primary   ext3
----

Running fsck fails as shown below:

----
e2fsck 1.35 (28-Feb-2004)
The filesystem size (according to the superblock) is 1098609033 blocks
The physical size of the device is 24867209 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>? yes

Error reading block 24870914 (Invalid argument) while doing inode scan.
----

I have 2 questions:

1) How did this system run just file for ~6 months using this
file system as a /home? I'm suspecting that the problem
actually occurred long ago when the file system allocated
meta or user data in blocks that are somehow unreachable
by fsck but exactly how this could have happened isn't
clear. Although it's too late now, I'd really like
to know what happened.

2) Given that this happened, how can I recover as many
files as possible from this file system? The professor
who owns this system had put his faith in hardware
RAID so he had never backed it up. He's very nervous
right now.

Any information or help you can provide would be
very much appreciated.

Cordially,
Jon Forrest
Unix Computing Support
College of Chemistry
Univ. of Cal. Berkeley
173 Tan Hall
Berkeley, CA
94720-1460
510-643-1032
jlforrest at berkeley.edu

Can u recreate your sdb1 using parted, but specifying a different end 
size, or just use "-1" ? And maybe try changing the label to
"gpt" ?
Then run e2fsck -n and see what it does. I wonder how you were able to 
create a 4TB ext3 filesystem with the msdos label under SL4.3. Never 
worked for me without the labelling it gpt.

Jon Forrest wrote:
> (I've already sent this message to Ted Ts'o directly. I should
> have sent it to this list first but I didn't know about it
> until today. My apologies to Ted.)
>
> Last Friday a system that I just inherited refused to mount
> a file system that had been working fine for about 6 months.
> This is on a Scientific Linux 4.3 system using a 2.6.9
> kernel. This is another Linux distribution based on RHEL 4.
> I don't think the actual hardware is relevant
> here so I won't mention it. If there's more information you'd
> like to see I'd be happy to provide it.
>
> It turns out that this 4.2TB file system was created in an
> msdos partition table, as shown below:
>
> ----
> GNU Parted 1.6.19
> Using /dev/sdb
> (parted) p
> Disk geometry for /dev/sdb: 0.000-4291443.000 megabytes
> Disk label type: msdos
> Minor    Start       End     Type      Filesystem  Flags
> 1          0.031  97137.567  primary   ext3
> ----
>
> Running fsck fails as shown below:
>
> ----
> e2fsck 1.35 (28-Feb-2004)
> The filesystem size (according to the superblock) is 1098609033 blocks
> The physical size of the device is 24867209 blocks
> Either the superblock or the partition table is likely to be corrupt!
> Abort<y>? yes
>
> Error reading block 24870914 (Invalid argument) while doing inode scan.
> ----
>
> I have 2 questions:
>
> 1) How did this system run just file for ~6 months using this
> file system as a /home? I'm suspecting that the problem
> actually occurred long ago when the file system allocated
> meta or user data in blocks that are somehow unreachable
> by fsck but exactly how this could have happened isn't
> clear. Although it's too late now, I'd really like
> to know what happened.
>
> 2) Given that this happened, how can I recover as many
> files as possible from this file system? The professor
> who owns this system had put his faith in hardware
> RAID so he had never backed it up. He's very nervous
> right now.
>
> Any information or help you can provide would be
> very much appreciated.
>
> Cordially,
> Jon Forrest
> Unix Computing Support
> College of Chemistry
> Univ. of Cal. Berkeley
> 173 Tan Hall
> Berkeley, CA
> 94720-1460
> 510-643-1032
> jlforrest at berkeley.edu
>
> _______________________________________________
> Ext3-users mailing list
> Ext3-users at redhat.com
> https://www.redhat.com/mailman/listinfo/ext3-users

On Mar 12, 2007  08:29 -0700, Jon Forrest wrote:
> Last Friday a system that I just inherited refused to mount
> a file system that had been working fine for about 6 months.
> This is on a Scientific Linux 4.3 system using a 2.6.9
> kernel. This is another Linux distribution based on RHEL 4.
> I don't think the actual hardware is relevant
> here so I won't mention it. If there's more information you'd
> like to see I'd be happy to provide it.
> 
> ----
> e2fsck 1.35 (28-Feb-2004)
> The filesystem size (according to the superblock) is 1098609033 blocks
> The physical size of the device is 24867209 blocks
> Either the superblock or the partition table is likely to be corrupt!
> Abort<y>? yes
> 
> Error reading block 24870914 (Invalid argument) while doing inode scan.
Did you recently update your kernel?  Is your kernel using CONFIG_LBD?
If CONFIG_LBD is not set, then any use of > 2TB is completely unsafe.
It will silently and fatally corrupt your filesystem.  I'd pointed this
out previously, but the patch I submitted wasn't accepted.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.