Ext3 users - Jun 2013 - Re: removing external journal

On 2013-06-26, at 9:38 AM, Eric Sandeen wrote:
> On 6/25/13 3:13 AM, Folkert van Heusden wrote:
>> 
>> I have a system with an ext4 filesystem with its journal on an other
>> device (an SSD).
>> Now this SSD dropped of the sata bus so the filesystem went r/o.
>> I would like to remove the journal but it says it can't because
>> needs_check is set.
> 
> What does it actually say?  there is no needs_check flag AFAIK.
I think he means "needs_recovery" - EXT3_FEATURE_INCOMPAT_RECOVER
set when journal recovery is needed after an unclean shutdown.

What you want is to run (per the tune2fs(8) man page, maybe something
should go into the e2fsck(8) man page as well?):

   tune2fs -f -O ^has_journal /dev/XXX
   e2fsck -fp /dev/XXX
   tune2fs -J device=/dev/YYY /dev/XXX

This will remove the journal from the filesystem, run e2fsck
to fix whatever problems it finds, then re-add the journal
device (if you have one, otherwise just use "tune2fs -j" to
add an internal journal).

Note that "tune2fs" reports the following for "-f":

       -f  Force the tune2fs operation to complete even in the
           face of errors. This option is useful when removing
           the has_journal filesystem feature from a filesystem
           which has an external journal (or is corrupted such
           that it appears to have an external journal), but that
           external journal is not available.

           WARNING: Removing an external journal from a filesystem
           which was  not  cleanly unmounted without first
           replaying the external journal can result in severe
           data loss and filesystem corruption.

If you can get the journal device back, it would be better, but
if not you don't really have any choice.  Any corruption found
should be not much worse than running without any journal, and
be limited to files/dirs that were in the middle of being modified.


What is interesting is that the filesystem _should_ be able to
handle the loss of the journal device by first dropping the
RECOVER feature and instead marking itself dirty as it would for
ext2.  It needs a full e2fsck after an unclean shutdown in either
case, though a clean shutdown would be totally safe, and no further
action would be needed if the journal device returned at some later
time.  The journal device shouldn't make the filesystem _less_
reliable.

Cheers, Andreas
>> I can't run fsck because the journal is not
>> reachable. Is there any way to solve this? I understand I lost any
>> pending changes in the journal.
> 
> Can you use debugfs to change the state of the fs so that it allows
> you to do this?
> 
> Or maybe specifying a zero-filled file as the external journal would
> allow fsck to proceed.
> 
> -Eric
> 
>> 
>> regards
>> 
>> --
>> www.vanheusden.com
>> 
>> _______________________________________________
>> Ext3-users mailing list
>> Ext3-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/ext3-users
>> 
> 
> _______________________________________________
> Ext3-users mailing list
> Ext3-users@redhat.com
> https://www.redhat.com/mailman/listinfo/ext3-users

Cheers, Andreas

Eric, Andreas,
>>> I have a system with an ext4 filesystem with its journal on an
other
>>> device (an SSD).
>>> Now this SSD dropped of the sata bus so the filesystem went r/o.
>>> I would like to remove the journal but it says it can't because
>>> needs_check is set.
>>
>> What does it actually say?  there is no needs_check flag AFAIK.
>
> I think he means "needs_recovery" - EXT3_FEATURE_INCOMPAT_RECOVER
> set when journal recovery is needed after an unclean shutdown.
Yes, that's the one.
> What you want is to run (per the tune2fs(8) man page, maybe something
> should go into the e2fsck(8) man page as well?):
>    tune2fs -f -O ^has_journal /dev/XXX
That one was denied because of the needs_recovery.

Fortunately I was able to bring the SSD back to life and got it all to recover!

The problem was with an intel motherboard with a couple of sata ports.
4 of them are from a marvell chipset and those give lots of problems.
I moved all hdds and ssds to the other ports and hopefully that solves
the instability.
> What is interesting is that the filesystem _should_ be able to
> handle the loss of the journal device by first dropping the
> RECOVER feature and instead marking itself dirty as it would for
> ext2.  It needs a full e2fsck after an unclean shutdown in either
> case, though a clean shutdown would be totally safe, and no further
> action would be needed if the journal device returned at some later
> time.  The journal device shouldn't make the filesystem _less_
> reliable.
Oh funny thing: after powering up the system again, the system said
that the filesystem said it was clean while there were changes pending
in the journal! After applying those changes, one filesystem was
hosed, the other was fine. Hooray for backups.


www.vanheusden.com

On 6/27/13 3:57 AM, Folkert van Heusden wrote:
> Eric, Andreas,
> 
>>>> I have a system with an ext4 filesystem with its journal on an
other
>>>> device (an SSD).
>>>> Now this SSD dropped of the sata bus so the filesystem went
r/o.
>>>> I would like to remove the journal but it says it can't
because
>>>> needs_check is set.
>>>
>>> What does it actually say?  there is no needs_check flag AFAIK.
>>
>> I think he means "needs_recovery" -
EXT3_FEATURE_INCOMPAT_RECOVER
>> set when journal recovery is needed after an unclean shutdown.
> 
> Yes, that's the one.
> 
>> What you want is to run (per the tune2fs(8) man page, maybe something
>> should go into the e2fsck(8) man page as well?):
>>    tune2fs -f -O ^has_journal /dev/XXX
> 
> That one was denied because of the needs_recovery.
Even with the "-f" ?

Hum, that sounds like it might be a bug then.  What version?

-Eric