Ext3 users - Aug 2001 - BUG: Assertion failure with ext3-0.95 for 2.4.7

Hello ext3-users,


I tested ext3 on a Linux for S/390 with several stress and benchmark test
tests and faced a kernel bug message.
The console showed the following output:

Message from syslogd@boeaet34 at Fri Aug  3 11:34:16 2001 ...
boeaet34 kernel: Assertion failure in journal_forget() at
transaction.c:1184: "!
jh->b_committed_data"

I tried the Patch from http://www.zip.com.au/~akpm/ext3-2.4-0.9.5-247.gz
with the kernel 2.4.7 with a new LVM- patch(0.9.1)  and some S/390 specific
patches. I use mke2fs version 1.22.
S/390 is a 32bit big endian machine. After compiling and running the kernel
I created an ext3-file system on an 70GB LVM. When running the postmark
test I get (reproduceable) the message from above. dmesg shows:

kernel BUG at transaction.c:1184!
illegal operation: 0001
CPU:    1
Process bench (pid: 2453, stackpage=08CEF000)

Kernel PSW:    07080000 8007f458         =journal_forget
task: 08cee000 ksp: 08cefaa8 pt_regs: 08cefa10
Kernel GPRS:
00000000  8001c118  00000022  00000001
8007f456  00c27000  00194f9a  00000001
030d2c80  074ed294  00001899  092ca350
0001f94c  8007f2c8  8007f456  08cefaa8
Kernel ACRS:
00000000  00000000  00000000  00000000
00000001  00000000  00000000  00000000
00000000  00000000  00000000  00000000
00000000  00000000  00000000  00000000
Kernel BackChain  CallChain
       08cefaa8   [<0007f456>]                =journal_forget
       08cefb10   [<000744c6>]                =ext3_forget
       08cefb70   [<000767b4>]                =ext3_clear_blocks
       08cefbd8   [<000768d4>]                =ext3_free_data
       08cefc50   [<00076c38>]                =ext3_truncate
       08cefd08   [<00074732>]                =ext3_delete_inode
       08cefd68   [<0006659a>]                =iput
       08cefdc8   [<00063dfc>]                =d_delete

I resolved the functions using the system.map file.

Has anyone saw this message before? Any ideas, clues, hints?

Please CC me , because I am not on the list.


--
Mit freundlichen Grüßen / Best Regards

Christian Bornträger
IBM Deutschland Entwicklung GmbH
eServer SW  System Evaluation + Test
email: CBORNTRA@de.ibm.com
Tel +49 7031-16-3507

On Wed, Aug 08, 2001 at 10:46:41AM +0200, Christian Borntraeger wrote:
> I tested ext3 on a Linux for S/390 with several stress and benchmark test
> tests and faced a kernel bug message.
> The console showed the following output:
> 
> Message from syslogd@boeaet34 at Fri Aug  3 11:34:16 2001 ...
> boeaet34 kernel: Assertion failure in journal_forget() at
> transaction.c:1184: "!
> jh->b_committed_data"
Hmm.  I managed to get that oops on my PPC box too.  Can you turn on the
buffer trace code, reproduce it and post the log of that?  I bet it looks
a lot like the one I got...

-- 
Tom Rini (TR1265)
http://gate.crashing.org/~trini/

>Well ext3 has more debugging checks than ext3 at the moment, and also
requires the
>underlying blocklayers (LVM/RAID etc) to not lie. So the test _IS_
relevant,
>not to caste blame, but to find the interaction.....

OK, I tried it now  with md  and the bug was there again. Now during a rm
-rf * .
As the bug ouccured with LVM __and__  with MD it looks like a ext3-problem.
Tom Rini reported the same problem on a PPC-Box. Possibly it is a big
endian problem.

The message was again:
kernel: Assertion failure in journal_forget() at transaction.c:1184:
"!jh->b_committed_data"


Here a have a new backtrace, I resolved the functions manually from
system.map -hopefully without a mistake.

kernel BUG at transaction.c:1184!
illegal operation: 0001
CPU:    1
Process rm (pid: 1917, stackpage=06125000)

Kernel PSW:    070c0000 8008023c   =journal_forget
task: 06124000 ksp: 06125938 pt_regs: 061258a0
Kernel GPRS:
00000000  8001c118  00000022  00000001
8008023a  00c2a000  00197198  00000001
12fc1880  08d06494  00001899  1575c9b0
0001f94c  800800ac  8008023a  06125938
Kernel ACRS:
00000000  00000000  00000000  00000000
00000001  00000000  00000000  00000000
00000000  00000000  00000000  00000000
00000000  00000000  00000000  00000000
Kernel BackChain  CallChain
       06125938   [<0008023a>]           =journal_forget
       061259a0   [<000747e6>]           =ext3_forget
       06125a08   [<00076c00>]           =ext3_clear_blocks
       06125a70   [<00076d20>]           =ext3_free_data
       06125ae8   [<00076ee4>]           =ext3_free_branches
       06125b60   [<00076e44>]           =ext3_free_branches
       06125bd8   [<00076e44>]           =ext3_free_branches
       06125c50   [<0007723c>]           =ext3_truncate


greetings

Christian Borntraeger wrote:
> 
> Hello ext3-users,
> 
> I tested ext3 on a Linux for S/390 with several stress and benchmark test
> tests and faced a kernel bug message.
> The console showed the following output:
> 
> Message from syslogd@boeaet34 at Fri Aug  3 11:34:16 2001 ...
> boeaet34 kernel: Assertion failure in journal_forget() at
> transaction.c:1184: "!
> jh->b_committed_data"
> 
Simple bug, subtle symptoms.  Could you please retest 0.9.5
with this patch?  Thanks.

--- ext3-0_9_5/fs/ext3/inode.c	Mon Jul 30 05:46:12 2001
+++ ext3/fs/ext3/inode.c	Thu Aug  9 00:03:34 2001
@@ -1522,7 +1523,7 @@
 	 * AKPM: turn on bforget in journal_forget()!!!
 	 */
 	for (p = first; p < last; p++) {
-		u32 nr = *p;
+		u32 nr = le32_to_cpu(*p);
 		if (nr) {
 			struct buffer_head *bh;
 

Now, if all on-disk structures were defined in terms of something
like

	struct disk32 {
		u32 x;
	}

then these things wold never happen - the compiler would catch
it.

-

> If it's possible, could you please also test journalled data mode?
It will take a while, but it is already planned.
> It'd be interesting to sanity test recovery as well, but doing
> thorough testing of recovery is hard.  That's why the ext3 patch
> places interesting debug/devel code way down inside the IDE device
> driver...
S/390 has no IDE. That might be the reason why I faced depmod-problems if I
include ext3 and jbd-debug as module.
Nevertheless debugging seems to work if I compile ext3 in the kernel.

greetings

> If it's possible, could you please also test journalled data mode?
I finished some tests with journalled data mode and with ordered mode. Both
modes seems to run correct, in the meaning of stability and correct log
replay, after a forced restart on S390 architecture, but further testing is
planned.

Today I started testing ext3 on an IBM zSeries  (s390x in the kernel tree).
It might be interesting for you, that s390x is a 64bit big endian machine.
I will post the results if I face any problems.

--
Mit freundlichen Grüßen / Best Regards

Christian Bornträger
IBM Deutschland Entwicklung GmbH
eServer SW  System Evaluation + Test
email: CBORNTRA@de.ibm.com
Tel +49 7031-16-3507