I am having major problems on one of my file-servers. I have the root filesystem on an Ext3 software RAID 1 between four SCSI drives. It had been running for about two months without problems, but today I found it was having some trouble with unlinked inodes and I/O errors on certain files. I took it down to single-user mode, and ran fsck on it. That operation mixed up files like /etc/hosts with /etc/rc.sysinit, and vice versa. I straightened those out, and ran fsck again. That happened to trash such files as /etc/raidtab and /etc/passwd and caused I/O errors so that I could not straighten it out. Running fsck again made the problem worse by making /etc/rc.sysinit unaccessible, and preventing me from booting up at all. I don't have helpful snippets from the error log since I was on a serial terminal and was incapable of paging up. I'm glad I finally got the terminal up to 38400 b/s though.... Right now, I am probably going to have to re-install, but I am wondering if there is anything I can do in the future to prevent errors like this. Thanks for the help! -- -- Skylar Thompson (skylar@attglobal.net)
On Sun, Jan 20, 2002 at 12:13:44PM -0600, Skylar Thompson wrote:> I am having major problems on one of my file-servers. I have the root > filesystem on an Ext3 software RAID 1 between four SCSI drives. It > had been running for about two months without problems, but today I > found it was having some trouble with unlinked inodes and I/O errors > on certain files. I took it down to single-user mode, and ran fsck on > it. That operation mixed up files like /etc/hosts with > /etc/rc.sysinit, and vice versa. I straightened those out, and ran > fsck again. That happened to trash such files as /etc/raidtab and > /etc/passwd and caused I/O errors so that I could not straighten it > out. Running fsck again made the problem worse by making > /etc/rc.sysinit unaccessible, and preventing me from booting up at > all. I don't have helpful snippets from the error log since I was on > a serial terminal and was incapable of paging up. I'm glad I finally > got the terminal up to 38400 b/s though.... > > Right now, I am probably going to have to re-install, but I am wondering if > there is anything I can do in the future to prevent errors like this.This really sounds like a br0ken harddisk. -- Ralf Hildebrandt (Im Auftrag des Referat V A) Ralf.Hildebrandt@charite.de Charite Campus Virchow-Klinikum Tel. +49 (0)30-450 570-155 Referat V A - Kommunikationsnetze - Fax. +49 (0)30-450 570-916 To rephrase, spam is not the answer. Spam is the question. Death is not the answer, but pretty close to it.
Hi, On Sun, Jan 20, 2002 at 12:13:44PM -0600, Skylar Thompson wrote:> I am having major problems on one of my file-servers. I have the root > filesystem on an Ext3 software RAID 1 between four SCSI drives. It had been > running for about two months without problems, but today I found it was > having some trouble with unlinked inodes and I/O errors on certain files. I > took it down to single-user mode, and ran fsck on it. That operation mixed > up files like /etc/hosts with /etc/rc.sysinit, and vice versa. I > straightened those out, and ran fsck again. That happened to trash such > files as /etc/raidtab and /etc/passwd and caused I/O errors so that I could > not straighten it out. Running fsck again made the problem worse by making > /etc/rc.sysinit unaccessible, and preventing me from booting up at all. I don't have helpful snippets from the error log since I was on a serial > terminal and was incapable of paging up. I'm glad I finally got the > terminal up to 38400 b/s though....There's not enough information here for us to have any hope of diagnosing it accurately. What sort of IO errors were you getting? -EIO errors returned to the application, or IO errors from device drivers in the kernel console? Cheers, Stephen