This fixed the problem:
On FC3's rescue disk...
1) Do startup network interfaces
2) Don't try to automatically mount the filesystems - not even readonly
3) lvm vgchange --ignorelockingfailure -P -a y
4) fdisk -l, and guess which partition is which based on size: the small one was
/boot, and the large one was /
5) mkdir /mnt/boot
6) mount /dev/hda1 /mnt/boot
7) Look up the device node for the root filesystem in /mnt/boot/grub/grub.conf
8) A first tentative step, to see if things are working: fsck -n
/dev/VolGroup00/LogVol00
9) Dive in: fsck -f -y /dev/VolGroup00/LogVol00
10) Wait a while... Be patient. Don't interrupt it
11) Reboot
On Wed, 15 Dec 2004 15:41:04 -0800, Dan Stromberg wrote:
>
> I have a Fedora Core 3 system at home, that was running fine, but now
> won't boot.
>
> Someone shut the power off on it without doing an orderly shutdown, and
> also I sometimes apply patches with "yum -y update" without doing
a reboot
> immediately afterward - I suppose either of these could be related to my
> system not booting.
>
> I have a lot of information about the early stages of the problem at:
>
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=142737
>
> In short, FC3 won't boot, and the FC3 rescue CD's automatic
filesystem
> mounting crashes. Mounting the filesystem manually errors with
"invalid
> argument". I suspect this is either ext3 corruption, or lvm2
corruption,
> or both.
>
> I've made a copy of the partition on another system, and have been
testing
> various things on that copy, like various fsck commands, e2salvage,
> e2extract, but fsck complains about bad superblocks, e2salvage apparently
> ran out of memory, and e2extract just listed millions of 0 length files.
>
> I wrote a small python script that hunts for ext3 magic numbers, and it
> found some in both a known-good ext3, as well as my corrupt partition
> image. The first offsets were the same, but others were different. All
> ended in hexadecimal 38. Does anyone know how to convert such numbers,
> relative to the beginning of the partition in bytes, to an appropriate
> fsck -b argument? What units does fsck -b take?
>
> The disk itself appears to be fine, as I can mount its /boot, and I got no
> errors when I dd'd off the partition image.
>
> When I left for work this morning, I had for loop with 1 million fsck's
> with different -b's (and -vn) running against a copy of the partition,
to
> see if it would eventually hit upon a usable superblock (assuming mkfs -n
> isn't doing what it should, and also, I just don't want to type
every
> last number...). But it doesn't seem likely to bear fruit, really.
>
> I also ran memtest86 on the system that had the trouble for a little over
> an hour, but found no errors.
>
> The machine was a $299 deal from bilsystem.com, which arrived unassembled.
> However, it's been stable until now, other than a time I had to
> replace its RAM.
>
> Does anyone have any suggestions for me? I'd really like to get this
data
> back!
>
> PS: I wrote something very much like e2extract for the atari 800 when I
> was in high school... If anyone has any thoughts about the general
> structure of such a program for ext3... I might dive into writing one. A
> small tree diagram of the on-disk data structures involved with 1-n and
> n-1 and n-n relationships might be enough to get a good start on it. But
> I'd rather not reinvent the wheel if it's already out there.
>
> Thanks!