thr3ads.net - Ext3 users - Going readonly too frequently [Jan 2010]

If this information is useful, please help other people find it:
Share via:

Muhammed Sameer

2010-Jan-26 13:15 UTC

Going readonly too frequently

Hey,

* Our server is going readonly atleast 10 - 15 times a day with the below error

<snip>
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1):
ext3_free_blocks_sb: bit already cleared for block 53771686
Jan 26 12:57:36 mailbox kernel: Aborting journal on device sdb1.
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_reserve_inode_write: Journal has aborted
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in ext3_truncate:
Journal has aborted
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_reserve_inode_write: Journal has aborted
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in ext3_orphan_del:
Journal has aborted
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_reserve_inode_write: Journal has aborted
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_delete_inode: Journal has aborted
Jan 26 12:57:36 mailbox kernel: __journal_remove_journal_head: freeing
b_committed_data
Jan 26 12:57:36 mailbox last message repeated 3 times
Jan 26 12:57:36 mailbox kernel: ext3_abort called.
Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1):
ext3_journal_start_sb: Detected aborted journal
Jan 26 12:57:36 mailbox kernel: Remounting filesystem read-only
Jan 26 12:58:04 mailbox kernel: __journal_remove_journal_head: freeing
b_committed_data
Jan 26 12:58:05 mailbox kernel: __journal_remove_journal_head: freeing
b_committed_data
</snip>

* Our kernel is 
2.6.18-182.el5

* Our OS is
Red Hat Enterprise Linux Server release 5.2 (Tikanga)

* We even tried fsck  but the problem persists


Regards,
Muhammed Sameer

Eric Sandeen

2010-Jan-26 15:48 UTC

head link

Going readonly too frequently

Muhammed Sameer wrote:> Hey,
> 
> * Our server is going readonly atleast 10 - 15 times a day with the below
error
ouch - and even just once is "too frequently"
:)> 
> <snip>
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1):
ext3_free_blocks_sb: bit already cleared for block 53771686
> Jan 26 12:57:36 mailbox kernel: Aborting journal on device sdb1.
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_truncate: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_orphan_del: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
ext3_delete_inode: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: __journal_remove_journal_head: freeing
b_committed_data
> Jan 26 12:57:36 mailbox last message repeated 3 times
> Jan 26 12:57:36 mailbox kernel: ext3_abort called.
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1):
ext3_journal_start_sb: Detected aborted journal
> Jan 26 12:57:36 mailbox kernel: Remounting filesystem read-only
> Jan 26 12:58:04 mailbox kernel: __journal_remove_journal_head: freeing
b_committed_data
> Jan 26 12:58:05 mailbox kernel: __journal_remove_journal_head: freeing
b_committed_data
> </snip>
> 
> * Our kernel is 
> 2.6.18-182.el5
> 
> * Our OS is
> Red Hat Enterprise Linux Server release 5.2 (Tikanga)
> 
> * We even tried fsck  but the problem persists
It would probably be best to open a support ticket with Red Hat for this one,
since it's a Red Hat kernel.

If you have any way to reproduce it that'd be very useful information - 
perhaps even a test using an e2image of the filesystem in question.

Thanks,
-Eric
> Regards,
> Muhammed Sameer
> 
> 
>       
> 
> _______________________________________________
> Ext3-users mailing list
> Ext3-users at redhat.com
> https://www.redhat.com/mailman/listinfo/ext3-users

Stephen Samuel (gmail)

2010-Jan-27 07:25 UTC

head link

Going readonly too frequently

Clearly you have a corrupt filesystem. Of all these times that the
filesystem has gone read-only, how many times have you done FSCKs?  Have you
done two FSCKs in a row?  It's rare, but  I've seen a second FSCK find
stuff
that was missed on the first run.

A smart report will at least tell you if the drive has been suffering errors
recently.

Do you have a SMART report for the drive?

On Tue, Jan 26, 2010 at 5:15 AM, Muhammed Sameer <samix_119 at
yahoo.com>wrote:
> Hey,
>
> * Our server is going readonly atleast 10 - 15 times a day with the below
> error
>
> <snip>
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1):
> ext3_free_blocks_sb: bit already cleared for block 53771686
> Jan 26 12:57:36 mailbox kernel: Aborting journal on device sdb1.
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
> ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
> ext3_truncate: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
> ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
> ext3_orphan_del: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
> ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) in
> ext3_delete_inode: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: __journal_remove_journal_head: freeing
> b_committed_data
> Jan 26 12:57:36 mailbox last message repeated 3 times
> Jan 26 12:57:36 mailbox kernel: ext3_abort called.
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1):
> ext3_journal_start_sb: Detected aborted journal
> Jan 26 12:57:36 mailbox kernel: Remounting filesystem read-only
> Jan 26 12:58:04 mailbox kernel: __journal_remove_journal_head: freeing
> b_committed_data
> Jan 26 12:58:05 mailbox kernel: __journal_remove_journal_head: freeing
> b_committed_data
> </snip>
>
> * Our kernel is
> 2.6.18-182.el5
>
> * Our OS is
> Red Hat Enterprise Linux Server release 5.2 (Tikanga)
>
> * We even tried fsck  but the problem persists
>
>
> Regards,
> Muhammed Sameer
>
>
>
>
> _______________________________________________
> Ext3-users mailing list
> Ext3-users at redhat.com
> https://www.redhat.com/mailman/listinfo/ext3-users
>


-- 
Stephen Samuel http://www.bcgreen.com  Software, like love,
778-861-7641                              grows when you give it away
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/ext3-users/attachments/20100126/4c7485f0/attachment.htm>

Muhammed Sameer

2010-Feb-05 21:22 UTC

head link

Going readonly too frequently

Hey,

* We upgraded the firmware on our RAID device and the FS seems to be stable for
now.

* Thank you guys for all your inputs, I really appreciate it!

Regards,
Muhammed Sameer

--- On Tue, 1/26/10, Muhammed Sameer <samix_119 at yahoo.com> wrote:
> From: Muhammed Sameer <samix_119 at yahoo.com>
> Subject: Going readonly too frequently
> To: ext3-users at redhat.com
> Date: Tuesday, January 26, 2010, 8:15 AM
> Hey,
> 
> * Our server is going readonly atleast 10 - 15 times a day
> with the below error
> 
> <snip>
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device
> sdb1): ext3_free_blocks_sb: bit already cleared for block
> 53771686
> Jan 26 12:57:36 mailbox kernel: Aborting journal on device
> sdb1.
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1)
> in ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1)
> in ext3_truncate: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1)
> in ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1)
> in ext3_orphan_del: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1)
> in ext3_reserve_inode_write: Journal has aborted
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1)
> in ext3_delete_inode: Journal has aborted
> Jan 26 12:57:36 mailbox kernel:
> __journal_remove_journal_head: freeing b_committed_data
> Jan 26 12:57:36 mailbox last message repeated 3 times
> Jan 26 12:57:36 mailbox kernel: ext3_abort called.
> Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device
> sdb1): ext3_journal_start_sb: Detected aborted journal
> Jan 26 12:57:36 mailbox kernel: Remounting filesystem
> read-only
> Jan 26 12:58:04 mailbox kernel:
> __journal_remove_journal_head: freeing b_committed_data
> Jan 26 12:58:05 mailbox kernel:
> __journal_remove_journal_head: freeing b_committed_data
> </snip>
> 
> * Our kernel is 
> 2.6.18-182.el5
> 
> * Our OS is
> Red Hat Enterprise Linux Server release 5.2 (Tikanga)
> 
> * We even tried fsck? but the problem persists
> 
> 
> Regards,
> Muhammed Sameer
> 
> 
> ? ? ? 
> 
> _______________________________________________
> Ext3-users mailing list
> Ext3-users at redhat.com
> https://www.redhat.com/mailman/listinfo/ext3-users
>

Ext3 users - Jan 2010 - Going readonly too frequently

Going readonly too frequently

Going readonly too frequently

Going readonly too frequently

Going readonly too frequently