thr3ads.net - Btrfs devel - 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479! [Apr 2011]

If this information is useful, please help other people find it:
Share via:

Sergei Trofimovich

2011-Apr-02 09:19 UTC

2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

The partition is a physical ~5GB --mixed lzo compressed partition.

The kernel 2.6.39-rc1 + reverted commit
c59021f846881a957ac5afe456d0f59d6a517b61.
(see http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg09083.html)

This time I''ve really filled whole partition and kernel OOpsed.
Note, the blow came right after INFO about hung write task (see full dmesg),
so it could be bad guy.

Apr  1 11:59:54 st kernel: [168399.199337] ------------[ cut here ]------------
Apr  1 11:59:54 st kernel: [168399.199368] kernel BUG at
fs/btrfs/extent-tree.c:5479!
Apr  1 11:59:54 st kernel: [168399.199380] invalid opcode: 0000 [#1] PREEMPT SMP
Apr  1 11:59:54 st kernel: [168399.199397] last sysfs file:
/sys/devices/virtual/vtconsole/vtcon1/uevent
Apr  1 11:59:54 st kernel: [168399.199409] CPU 0 
Apr  1 11:59:54 st kernel: [168399.199416] Modules linked in: btrfs bridge stp
llc zlib_deflate lzo_decompress lzo_compress crc3c libcrc32c tun kvm_amd kvm
fuse nouveau ttm drm_kms_helper drm i2c_algo_bit forcedeth i2c_core 8139cp
8139too cfbcopyarea cfbimblt cfbfillrect [last unloaded: btrfs]
Apr  1 11:59:54 st kernel: [168399.199508] 
Apr  1 11:59:54 st kernel: [168399.199516] Pid: 18178, comm: btrfs-endio-wri Not
tainted 2.6.39-rc1+ #2 To Be Filled By O.E.M. T Be Filled By
O.E.M./ALiveNF6G-VSTA
Apr  1 11:59:54 st kernel: [168399.199540] RIP: 0010:[<ffffffffa04c1123>] 
[<ffffffffa04c1123>] alloc_reserved_file_extent.clone66+0x213/0x220
[btrfs]
Apr  1 11:59:54 st kernel: [168399.199589] RSP: 0018:ffff88000de7ba80  EFLAGS:
00010286
Apr  1 11:59:54 st kernel: [168399.199600] RAX: 00000000ffffffe4 RBX:
ffff880077f17bd0 RCX: 0000000000000000
Apr  1 11:59:54 st kernel: [168399.199612] RDX: 0000000000000000 RSI:
ffff880077f17bd0 RDI: 0000000000000000
Apr  1 11:59:54 st kernel: [168399.199623] RBP: ffff88000de7bb00 R08:
0000000000000000 R09: 0000000000000000
Apr  1 11:59:54 st kernel: [168399.199635] R10: ffffffffffffffe4 R11:
0000000000000002 R12: 00000000000000b2
Apr  1 11:59:54 st kernel: [168399.199647] R13: 0000000000000000 R14:
ffff88000de7bbb0 R15: ffff88001e16ee00
Apr  1 11:59:54 st kernel: [168399.199659] FS:  00007fcc3e5f2700(0000)
GS:ffff88007bc00000(0000) knlGS:00000000f68deb20
Apr  1 11:59:54 st kernel: [168399.199673] CS:  0010 DS: 0000 ES: 0000 CR0:
000000008005003b
Apr  1 11:59:54 st kernel: [168399.199683] CR2: 00007f87634fab00 CR3:
0000000033310000 CR4: 00000000000006f0
Apr  1 11:59:54 st kernel: [168399.199694] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Apr  1 11:59:54 st kernel: [168399.199705] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Apr  1 11:59:54 st kernel: [168399.199718] Process btrfs-endio-wri (pid: 18178,
threadinfo ffff88000de7a000, task ffff88007083d80)
Apr  1 11:59:54 st kernel: [168399.199731] Stack:
Apr  1 11:59:54 st kernel: [168399.199736]  ffff880000000035 00000000002c9a4e
0000000000000005 0000000000000000
Apr  1 11:59:54 st kernel: [168399.199755]  ffff880036370928 ffffffffa04f984e
ffff88000de7bad0 ffff8800166dc000
Apr  1 11:59:54 st kernel: [168399.199773]  ffff88000de7bb10 0000003500000001
ffff88000de7baf0 ffff88000616aa20
Apr  1 11:59:54 st kernel: [168399.199790] Call Trace:
Apr  1 11:59:54 st kernel: [168399.199814]  [<ffffffffa04f984e>] ?
unmap_extent_buffer+0xe/0x40 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199831]  [<ffffffffa04c414c>]
run_clustered_refs+0x2ec/0x860 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199850]  [<ffffffffa04f9755>] ?
map_private_extent_buffer+0xd5/0x1c0 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199871]  [<ffffffffa0514b01>] ?
btrfs_find_ref_cluster+0xe1/0x180 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199889]  [<ffffffffa04c4780>]
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199909]  [<ffffffffa04d5ca8>]
__btrfs_end_transaction+0x68/0x220 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199928]  [<ffffffffa04d5ea0>]
btrfs_end_transaction+0x10/0x20 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199946]  [<ffffffffa04dba0c>]
btrfs_finish_ordered_io+0x28c/0x330 [btrfs]
Apr  1 11:59:54 st kernel: [168399.199965]  [<ffffffff810a2e32>] ?
test_clear_page_writeback+0xf2/0x130
Apr  1 11:59:54 st kernel: [168399.199984]  [<ffffffffa04dbac5>]
btrfs_writepage_end_io_hook+0x15/0x20 [btrfs]
Apr  1 11:59:54 st kernel: [168399.200004]  [<ffffffffa04f4adb>]
end_bio_extent_writepage+0x13b/0x180 [btrfs]
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff8110f528>]
bio_endio+0x18/0x30
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffffa04cf1ac>]
end_workqueue_fn+0xec/0x120 [btrfs]
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffffa0501dfc>]
worker_loop+0xac/0x510 [btrfs]
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffffa0501d50>] ?
btrfs_queue_worker+0x300/0x300 [btrfs]
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff8105c0e6>]
kthread+0x96/0xa0
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff815758d4>]
kernel_thread_helper+0x4/0x10
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff8105c050>] ?
kthread_worker_fn+0x190/0x190
Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff815758d0>] ?
gs_change+0xb/0xb
Apr  1 11:59:54 st kernel: [168399.200010] Code: 48 83 c4 58 b8 f4 ff ff ff 5b
41 5c 41 5d 41 5e 41 5f c9 c3 49 8b 56 09 49 8b 3 48 c7 c7 50 19 52 a0 31 c0 e8
de fb 0a e1 0f 0b <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41
57 41 56
Apr  1 11:59:54 st kernel: [168399.200010] RIP  [<ffffffffa04c1123>]
alloc_reserved_file_extent.clone.66+0x213/0x220 [btrfs]
Apr  1 11:59:54 st kernel: [168399.200010]  RSP <ffff88000de7ba80>
Apr  1 11:59:54 st kernel: [168399.203852] ---[ end trace c4a85bad852f7be8 ]---

-- 

  Sergei

liubo

2011-Apr-02 09:37 UTC

head link

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

On 04/02/2011 05:19 PM, Sergei Trofimovich wrote:> The partition is a physical ~5GB --mixed lzo compressed partition.
> 
> The kernel 2.6.39-rc1 + reverted commit
c59021f846881a957ac5afe456d0f59d6a517b61.
> (see http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg09083.html)
> 
Hi, Sergei,

I''m digging this...

Can u show me steps to reproduce this?

thanks,
liubo
> This time I''ve really filled whole partition and kernel OOpsed.
> Note, the blow came right after INFO about hung write task (see full
dmesg),
> so it could be bad guy.
> 
> Apr  1 11:59:54 st kernel: [168399.199337] ------------[ cut here
]------------
> Apr  1 11:59:54 st kernel: [168399.199368] kernel BUG at
fs/btrfs/extent-tree.c:5479!
> Apr  1 11:59:54 st kernel: [168399.199380] invalid opcode: 0000 [#1]
PREEMPT SMP
> Apr  1 11:59:54 st kernel: [168399.199397] last sysfs file:
/sys/devices/virtual/vtconsole/vtcon1/uevent
> Apr  1 11:59:54 st kernel: [168399.199409] CPU 0 
> Apr  1 11:59:54 st kernel: [168399.199416] Modules linked in: btrfs bridge
stp llc zlib_deflate lzo_decompress lzo_compress crc3c libcrc32c tun kvm_amd kvm
fuse nouveau ttm drm_kms_helper drm i2c_algo_bit forcedeth i2c_core 8139cp
8139too cfbcopyarea cfbimblt cfbfillrect [last unloaded: btrfs]
> Apr  1 11:59:54 st kernel: [168399.199508] 
> Apr  1 11:59:54 st kernel: [168399.199516] Pid: 18178, comm:
btrfs-endio-wri Not tainted 2.6.39-rc1+ #2 To Be Filled By O.E.M. T Be Filled By
O.E.M./ALiveNF6G-VSTA
> Apr  1 11:59:54 st kernel: [168399.199540] RIP:
0010:[<ffffffffa04c1123>]  [<ffffffffa04c1123>]
alloc_reserved_file_extent.clone66+0x213/0x220 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199589] RSP: 0018:ffff88000de7ba80 
EFLAGS: 00010286
> Apr  1 11:59:54 st kernel: [168399.199600] RAX: 00000000ffffffe4 RBX:
ffff880077f17bd0 RCX: 0000000000000000
> Apr  1 11:59:54 st kernel: [168399.199612] RDX: 0000000000000000 RSI:
ffff880077f17bd0 RDI: 0000000000000000
> Apr  1 11:59:54 st kernel: [168399.199623] RBP: ffff88000de7bb00 R08:
0000000000000000 R09: 0000000000000000
> Apr  1 11:59:54 st kernel: [168399.199635] R10: ffffffffffffffe4 R11:
0000000000000002 R12: 00000000000000b2
> Apr  1 11:59:54 st kernel: [168399.199647] R13: 0000000000000000 R14:
ffff88000de7bbb0 R15: ffff88001e16ee00
> Apr  1 11:59:54 st kernel: [168399.199659] FS:  00007fcc3e5f2700(0000)
GS:ffff88007bc00000(0000) knlGS:00000000f68deb20
> Apr  1 11:59:54 st kernel: [168399.199673] CS:  0010 DS: 0000 ES: 0000 CR0:
000000008005003b
> Apr  1 11:59:54 st kernel: [168399.199683] CR2: 00007f87634fab00 CR3:
0000000033310000 CR4: 00000000000006f0
> Apr  1 11:59:54 st kernel: [168399.199694] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
> Apr  1 11:59:54 st kernel: [168399.199705] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
> Apr  1 11:59:54 st kernel: [168399.199718] Process btrfs-endio-wri (pid:
18178, threadinfo ffff88000de7a000, task ffff88007083d80)
> Apr  1 11:59:54 st kernel: [168399.199731] Stack:
> Apr  1 11:59:54 st kernel: [168399.199736]  ffff880000000035
00000000002c9a4e 0000000000000005 0000000000000000
> Apr  1 11:59:54 st kernel: [168399.199755]  ffff880036370928
ffffffffa04f984e ffff88000de7bad0 ffff8800166dc000
> Apr  1 11:59:54 st kernel: [168399.199773]  ffff88000de7bb10
0000003500000001 ffff88000de7baf0 ffff88000616aa20
> Apr  1 11:59:54 st kernel: [168399.199790] Call Trace:
> Apr  1 11:59:54 st kernel: [168399.199814]  [<ffffffffa04f984e>] ?
unmap_extent_buffer+0xe/0x40 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199831]  [<ffffffffa04c414c>]
run_clustered_refs+0x2ec/0x860 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199850]  [<ffffffffa04f9755>] ?
map_private_extent_buffer+0xd5/0x1c0 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199871]  [<ffffffffa0514b01>] ?
btrfs_find_ref_cluster+0xe1/0x180 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199889]  [<ffffffffa04c4780>]
btrfs_run_delayed_refs+0xc0/0x210 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199909]  [<ffffffffa04d5ca8>]
__btrfs_end_transaction+0x68/0x220 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199928]  [<ffffffffa04d5ea0>]
btrfs_end_transaction+0x10/0x20 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199946]  [<ffffffffa04dba0c>]
btrfs_finish_ordered_io+0x28c/0x330 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.199965]  [<ffffffff810a2e32>] ?
test_clear_page_writeback+0xf2/0x130
> Apr  1 11:59:54 st kernel: [168399.199984]  [<ffffffffa04dbac5>]
btrfs_writepage_end_io_hook+0x15/0x20 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.200004]  [<ffffffffa04f4adb>]
end_bio_extent_writepage+0x13b/0x180 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff8110f528>]
bio_endio+0x18/0x30
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffffa04cf1ac>]
end_workqueue_fn+0xec/0x120 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffffa0501dfc>]
worker_loop+0xac/0x510 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffffa0501d50>] ?
btrfs_queue_worker+0x300/0x300 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff8105c0e6>]
kthread+0x96/0xa0
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff815758d4>]
kernel_thread_helper+0x4/0x10
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff8105c050>] ?
kthread_worker_fn+0x190/0x190
> Apr  1 11:59:54 st kernel: [168399.200010]  [<ffffffff815758d0>] ?
gs_change+0xb/0xb
> Apr  1 11:59:54 st kernel: [168399.200010] Code: 48 83 c4 58 b8 f4 ff ff ff
5b 41 5c 41 5d 41 5e 41 5f c9 c3 49 8b 56 09 49 8b 3 48 c7 c7 50 19 52 a0 31 c0
e8 de fb 0a e1 0f 0b <0f> 0b 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5
41 57 41 56
> Apr  1 11:59:54 st kernel: [168399.200010] RIP  [<ffffffffa04c1123>]
alloc_reserved_file_extent.clone.66+0x213/0x220 [btrfs]
> Apr  1 11:59:54 st kernel: [168399.200010]  RSP <ffff88000de7ba80>
> Apr  1 11:59:54 st kernel: [168399.203852] ---[ end trace c4a85bad852f7be8
]---
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Sergei Trofimovich

2011-Apr-02 10:41 UTC

head link

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

On Sat, 02 Apr 2011 17:37:58 +0800
liubo <liubo2009@cn.fujitsu.com> wrote:
> On 04/02/2011 05:19 PM, Sergei Trofimovich wrote:
> > The partition is a physical ~5GB --mixed lzo compressed partition.
> > 
> > The kernel 2.6.39-rc1 + reverted commit
c59021f846881a957ac5afe456d0f59d6a517b61.
> > (see
http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg09083.html)
> > 
> 
> Hi, Sergei,
> 
> I''m digging this...
> 
> Can u show me steps to reproduce this?
I use the filesystem as a storage of large CVS tree and
as temp storage for large compilations, so I can roughly
describe what I did and when it failed.

I''ve formatter btrfs 5G partition as --mixed and mounter it with lzo
compression
on the kernel of version ''v2.6.38-4148-g054cfaa'', then checked
out there
large CVS tree (~170K files, weights 177MB), copied there linux source (not
built)
and copied my ''/var/''. I ran compiles there and started to get
-ENOSPC
OOpses when ''df -h'' reported 3.5G free.

As Linus pulled josef''s changes, so I''ve updated to
v2.6.38-6555-ga44f99c
and kernel started to OOps right after mount (added assert started to trigger
earlier).
I''ve reported it to this ML (link above). josef and sensille helped me
to debug what''s
going wrong [both CCed]. sensille pointed to the commit, which is guilty to
miscomputing
available space. As I understood they know what exactly screwed up.

The second case (this one):
I still use the same filesystem (didn''t reformat, so it might carry
some corruption
after debugging patches).
I''ve reverted your change c59021f846881a957ac5afe456d0f59d6a517b61
and made sure it stops OOpsing for me, then updated to 2.6.39-rc1
and reverted only this commit. Filesystem became usable until I''ve
decided
to run large compile on it (clang debug source).

I think at the time of OOps the following things did happen simultaneously:

1. one process was splitting debug symbols of some binary:
  - opened original binary for read
  - write to new file (stripped binary)
  - write debug symbols to separate file

2. another process logged that action to log file

3. the filesystem filled-up and OOpsed. At the time of OOps
   ''df -h'' showed 200M free.

I''m trying to reproduce this second case ATM (build takes
more, that an hour).

-- 

  Sergei

liubo

2011-Apr-02 11:30 UTC

head link

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

On 04/02/2011 06:41 PM, Sergei Trofimovich wrote:> On Sat, 02 Apr 2011 17:37:58 +0800
> liubo <liubo2009@cn.fujitsu.com> wrote:
> 
>> On 04/02/2011 05:19 PM, Sergei Trofimovich wrote:
>>> The partition is a physical ~5GB --mixed lzo compressed partition.
>>>
>>> The kernel 2.6.39-rc1 + reverted commit
c59021f846881a957ac5afe456d0f59d6a517b61.
>>> (see
http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg09083.html)
>>>
>> Hi, Sergei,
>>
>> I''m digging this...
>>
>> Can u show me steps to reproduce this?
> 
> I use the filesystem as a storage of large CVS tree and
> as temp storage for large compilations, so I can roughly
> describe what I did and when it failed.
> 
> I''ve formatter btrfs 5G partition as --mixed and mounter it with
lzo compression
> on the kernel of version ''v2.6.38-4148-g054cfaa'', then
checked out there
> large CVS tree (~170K files, weights 177MB), copied there linux source (not
built)
> and copied my ''/var/''. I ran compiles there and started
to get -ENOSPC
> OOpses when ''df -h'' reported 3.5G free.
> 
> As Linus pulled josef''s changes, so I''ve updated to
v2.6.38-6555-ga44f99c
> and kernel started to OOps right after mount (added assert started to
trigger earlier).
> I''ve reported it to this ML (link above). josef and sensille
helped me to debug what''s
> going wrong [both CCed]. sensille pointed to the commit, which is guilty to
miscomputing
> available space. As I understood they know what exactly screwed up.
> 
Great thanks for these details.

I did not consider the "mix" case when making the guilty patch, sorry.
Frankly, I''m still trying to reproduce your first bug, and on my box
"mix + lzo" does not cause bug...

Seems that you are using opensuse''s kernel.
> The second case (this one):
> I still use the same filesystem (didn''t reformat, so it might
carry some corruption
> after debugging patches).
> I''ve reverted your change c59021f846881a957ac5afe456d0f59d6a517b61
> and made sure it stops OOpsing for me, then updated to 2.6.39-rc1
> and reverted only this commit. Filesystem became usable until I''ve
decided
> to run large compile on it (clang debug source).
> 
> I think at the time of OOps the following things did happen simultaneously:
> 
> 1. one process was splitting debug symbols of some binary:
>   - opened original binary for read
>   - write to new file (stripped binary)
>   - write debug symbols to separate file
> 
> 2. another process logged that action to log file
> 
> 3. the filesystem filled-up and OOpsed. At the time of OOps
>    ''df -h'' showed 200M free.
> 
> I''m trying to reproduce this second case ATM (build takes
> more, that an hour).
> 
All right, thanks for the work.

thanks,
liubo
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Sergei Trofimovich

2011-Apr-02 12:55 UTC

head link

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

On Sat, 02 Apr 2011 19:30:40 +0800
liubo <liubo2009@cn.fujitsu.com> wrote:
> On 04/02/2011 06:41 PM, Sergei Trofimovich wrote:
> > On Sat, 02 Apr 2011 17:37:58 +0800
> > liubo <liubo2009@cn.fujitsu.com> wrote:
> > 
> >> On 04/02/2011 05:19 PM, Sergei Trofimovich wrote:
> >>> The partition is a physical ~5GB --mixed lzo compressed
partition.
> >>>
> >>> The kernel 2.6.39-rc1 + reverted commit
c59021f846881a957ac5afe456d0f59d6a517b61.
> >>> (see
http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg09083.html)
> >>>
> >> Hi, Sergei,
> >>
> >> I''m digging this...
> >>
> >> Can u show me steps to reproduce this?
> > 
> > I use the filesystem as a storage of large CVS tree and
> > as temp storage for large compilations, so I can roughly
> > describe what I did and when it failed.
> > 
> > I''ve formatter btrfs 5G partition as --mixed and mounter it
with lzo compression
> > on the kernel of version ''v2.6.38-4148-g054cfaa'',
then checked out there
> > large CVS tree (~170K files, weights 177MB), copied there linux source
(not built)
> > and copied my ''/var/''. I ran compiles there and
started to get -ENOSPC
> > OOpses when ''df -h'' reported 3.5G free.
> > 
> > As Linus pulled josef''s changes, so I''ve updated to
v2.6.38-6555-ga44f99c
> > and kernel started to OOps right after mount (added assert started to
trigger earlier).
> > I''ve reported it to this ML (link above). josef and sensille
helped me to debug what''s
> > going wrong [both CCed]. sensille pointed to the commit, which is
guilty to miscomputing
> > available space. As I understood they know what exactly screwed up.
> > 
> 
> Great thanks for these details.
> 
> I did not consider the "mix" case when making the guilty patch,
sorry.
> Frankly, I''m still trying to reproduce your first bug, and on my
box "mix + lzo" does not cause bug...
> 
> Seems that you are using opensuse''s kernel.
I use only Linus''s mainline kernels. I''ll try to cook-up some
deterministic script
to torture filesystem and reproduce that OOps.
> > The second case (this one):
> > I still use the same filesystem (didn''t reformat, so it might
carry some corruption
> > after debugging patches).
> > I''ve reverted your change
c59021f846881a957ac5afe456d0f59d6a517b61
> > and made sure it stops OOpsing for me, then updated to 2.6.39-rc1
> > and reverted only this commit. Filesystem became usable until
I''ve decided
> > to run large compile on it (clang debug source).
> > 
> > I think at the time of OOps the following things did happen
simultaneously:
> > 
> > 1. one process was splitting debug symbols of some binary:
> >   - opened original binary for read
> >   - write to new file (stripped binary)
> >   - write debug symbols to separate file
> > 
> > 2. another process logged that action to log file
> > 
> > 3. the filesystem filled-up and OOpsed. At the time of OOps
> >    ''df -h'' showed 200M free.
> > 
> > I''m trying to reproduce this second case ATM (build takes
> > more, that an hour).
Right after reboot I''ve decided to fill that partition even more
and see what happens (didn''t remove temporary clang build).

I have tried to copy to the partition (there was ~500M free reported by df)
a bunch of small object files until ''cp'' will report
out-of-space,
hen attempted to copy a bit more of small files until I get bored.
Nothing bad did happen.

Then I''ve decided to delete files I copied and temporray clang-built
files
on FS and kernel OOpsed:

[69392.216454] BUG: unable to handle kernel NULL pointer dereference at
0000000000000030
[69392.216493] IP: [<ffffffffa02352c7>] btrfs_print_leaf+0x27/0x890
[btrfs]
[69392.216537] PGD 0 
[69392.216547] Oops: 0000 [#1] PREEMPT SMP 
[69392.216562] last sysfs file: /sys/devices/virtual/vtconsole/vtcon1/uevent
[69392.216575] CPU 0 
[69392.216581] Modules linked in: bridge stp llc btrfs zlib_deflate
lzo_decompress lzo_compress crc32c libcrc32c tun kvm_amd kvm fuse nouveau ttm
drm_kms_helpe
r 8139cp drm i2c_algo_bit i2c_core 8139too cfbcopyarea cfbimgblt cfbfillrect
forcedeth
[69392.216668] 
[69392.216675] Pid: 22974, comm: rm Not tainted 2.6.39-rc1+ #2 To Be Filled By
O.E.M. To Be Filled By O.E.M./ALiveNF6G-VSTA
[69392.216699] RIP: 0010:[<ffffffffa02352c7>]  [<ffffffffa02352c7>]
btrfs_print_leaf+0x27/0x890 [btrfs]
[69392.216724] RSP: 0018:ffff880060111aa8  EFLAGS: 00010283
[69392.216734] RAX: 00000000ffffffe4 RBX: 0000000000000000 RCX: 00000000ffffffe4
[69392.216746] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000001
[69392.216757] RBP: ffff880060111b28 R08: 0000000000000000 R09: 0000000000000000
[69392.216769] R10: ffffffffffffffe4 R11: 0000000000000002 R12: 0000000000000005
[69392.216781] R13: ffff880077397800 R14: 0000000000000000 R15: 0000000000a64000
[69392.216794] FS:  00007f4b0cb06700(0000) GS:ffff88007bc00000(0000)
knlGS:00000000f68c1b20
[69392.216807] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[69392.216817] CR2: 0000000000000030 CR3: 000000000f2e6000 CR4: 00000000000006f0
[69392.216829] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[69392.216840] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[69392.216853] Process rm (pid: 22974, threadinfo ffff880060110000, task
ffff88002994d840)
[69392.216865] Stack:
[69392.216871]  00000000d3750000 ffffffffa0261114 ffff880077397800
00000000d3693000
[69392.216890]  ffff880060111b28 ffff880072158ab0 00000000d3750000
00000000003000a8
[69392.216910]  ffff880060111b00 ffffffff810d2937 ffff88001a5c9ea0
ffff880072158ab0
[69392.216929] Call Trace:
[69392.216953]  [<ffffffffa0261114>] ? set_extent_dirty+0x24/0x30 [btrfs]
[69392.216968]  [<ffffffff810d2937>] ? kmem_cache_alloc+0x97/0xb0
[69392.216985]  [<ffffffffa022d777>] __btrfs_free_extent+0x647/0x700
[btrfs]
[69392.216999]  [<ffffffff8103b351>] ? get_parent_ip+0x11/0x50
[69392.217006]  [<ffffffff8103b351>] ? get_parent_ip+0x11/0x50
[69392.217006]  [<ffffffffa023007b>] run_clustered_refs+0x21b/0x860
[btrfs]
[69392.217006]  [<ffffffffa022f73e>] ? btrfs_free_extent+0x4e/0xd0 [btrfs]
[69392.217006]  [<ffffffffa0280b00>] ? btrfs_find_ref_cluster+0xe0/0x180
[btrfs]
[69392.217006]  [<ffffffffa0230780>] btrfs_run_delayed_refs+0xc0/0x210
[btrfs]
[69392.217006]  [<ffffffffa0241ca8>] __btrfs_end_transaction+0x68/0x220
[btrfs]
[69392.217006]  [<ffffffffa0241ea0>] btrfs_end_transaction+0x10/0x20
[btrfs]
[69392.217006]  [<ffffffffa024b75f>] btrfs_evict_inode+0x19f/0x1f0 [btrfs]
[69392.217006]  [<ffffffff810fa2d0>] evict+0x80/0x160
[69392.217006]  [<ffffffff810fa4e2>] iput+0xe2/0x1b0
[69392.217006]  [<ffffffff810ef68b>] do_unlinkat+0x10b/0x1b0
[69392.217006]  [<ffffffff8111727b>] ? fsnotify_find_inode_mark+0x3b/0x50
[69392.217006]  [<ffffffff810dff91>] ? filp_close+0x61/0x90
[69392.217006]  [<ffffffff810f0c0d>] sys_unlinkat+0x1d/0x40
[69392.217006]  [<ffffffff81574c3b>] system_call_fastpath+0x16/0x1b
[69392.217006] Code: 00 00 00 00 55 48 89 e5 48 83 c4 80 4c 89 6d e8 49 89 fd bf
01 00 00 00 48 89 5d d8 4c 89 65 e0 48 89 f3 4c 89 75 f0 4c 89 7d f8 <4c>
8b 6
6 30 e8 90 61 e0 e0 48 b8 00 00 00 00 00 16 00 00 49 01 
[69392.217006] RIP  [<ffffffffa02352c7>] btrfs_print_leaf+0x27/0x890
[btrfs]
[69392.217006]  RSP <ffff880060111aa8>
[69392.217006] CR2: 0000000000000030
[69392.220992] ---[ end trace 3cfc7af696ff077b ]---

Full dmesg is in attach. Same kernel, same partition.

-- 

  Sergei

liubo

2011-Apr-08 08:44 UTC

head link

[PATCH] Btrfs: fix easily get into ENOSPC in mixed case

When a btrfs disk is created by mixed data & metadata option, it will have
no
pure data or pure metadata space info.

In btrfs''s for-linus branch, commit
78b1ea13838039cd88afdd62519b40b344d6c920
(Btrfs: fix OOPS of empty filesystem after balance) initializes space infos at
the very beginning.  The problem is this initialization does not take the mixed
case into account, which will cause btrfs will easily get into ENOSPC in mixed
case.

Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
---
 fs/btrfs/extent-tree.c |   37 ++++++++++++++++++++++++++-----------
 1 files changed, 26 insertions(+), 11 deletions(-)

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index f619c3c..1b47ae4 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -8781,23 +8781,38 @@ out:
 int btrfs_init_space_info(struct btrfs_fs_info *fs_info)
 {
 	struct btrfs_space_info *space_info;
+	struct btrfs_super_block *disk_super;
+	u64 features;
+	u64 flags;
+	int mixed = 0;
 	int ret;
 
-	ret = update_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM, 0, 0,
-								 &space_info);
-	if (ret)
-		return ret;
+	disk_super = &fs_info->super_copy;
+	if (!btrfs_super_root(disk_super))
+		return 1;
 
-	ret = update_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA, 0, 0,
-								 &space_info);
-	if (ret)
-		return ret;
+	features = btrfs_super_incompat_flags(disk_super);
+	if (features & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS)
+		mixed = 1;
 
-	ret = update_space_info(fs_info, BTRFS_BLOCK_GROUP_DATA, 0, 0,
-								 &space_info);
+	flags = BTRFS_BLOCK_GROUP_SYSTEM;
+	ret = update_space_info(fs_info, flags, 0, 0, &space_info);
 	if (ret)
-		return ret;
+		goto out;
 
+	if (mixed) {
+		flags = BTRFS_BLOCK_GROUP_METADATA | BTRFS_BLOCK_GROUP_DATA;
+		ret = update_space_info(fs_info, flags, 0, 0, &space_info);
+	} else {
+		flags = BTRFS_BLOCK_GROUP_METADATA;
+		ret = update_space_info(fs_info, flags, 0, 0, &space_info);
+		if (ret)
+			goto out;
+
+		flags = BTRFS_BLOCK_GROUP_DATA;
+		ret = update_space_info(fs_info, flags, 0, 0, &space_info);
+	}
+out:
 	return ret;
 }
 
-- 
1.6.5.2
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Sergei Trofimovich

2011-Apr-08 21:09 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

On Fri, 08 Apr 2011 16:44:37 +0800
liubo <liubo2009@cn.fujitsu.com> wrote:
> 
> When a btrfs disk is created by mixed data & metadata option, it will
have no
> pure data or pure metadata space info.
> 
> In btrfs''s for-linus branch, commit
78b1ea13838039cd88afdd62519b40b344d6c920
> (Btrfs: fix OOPS of empty filesystem after balance) initializes space infos
at
> the very beginning.  The problem is this initialization does not take the
mixed
> case into account, which will cause btrfs will easily get into ENOSPC in
mixed
> case.
> 
> Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
> ---
Tried to apply this patch on top of 2.6.39-rc2 and booted on the same partition.
It has 3.2 GBs free. With patch it started to report -ENOSPC right out.
Attempt to delete some files caused OOps:

[  220.784098] ------------[ cut here ]------------
[  220.784416] kernel BUG at /mnt/archive/src/linux-2.6/fs/btrfs/inode.c:2962!
[  220.784700] invalid opcode: 0000 [#1] PREEMPT SMP 
[  220.784969] last sysfs file: /sys/devices/virtual/bdi/0:26/uevent
[  220.785011] CPU 0 
[  220.785011] Modules linked in: bridge stp llc btrfs zlib_deflate
lzo_decompress lzo_compress crc32c libcrc32c tun kvm_amd kvm fuse nouveau ttm
drm_kms_helper drm i2c_algo_bit i2c_core cfbcopyarea cfbimgblt 8139cp
cfbfillrect 8139too forcedeth
[  220.785011] 
[  220.785011] Pid: 2881, comm: mc Not tainted 2.6.39-rc2+ #2 To Be Filled By
O.E.M. To Be Filled By O.E.M./ALiveNF6G-VSTA
[  220.785011] RIP: 0010:[<ffffffffa024a141>]  [<ffffffffa024a141>]
btrfs_unlink+0xd1/0xe0 [btrfs]
[  220.785011] RSP: 0018:ffff88006898fe28  EFLAGS: 00010286
[  220.785011] RAX: 00000000ffffffe4 RBX: 00000000ffffffe4 RCX: 0000000000006d32
[  220.785011] RDX: 0000000000006d30 RSI: 00000000000196d0 RDI: ffffea0001a3d2b8
[  220.785011] RBP: ffff88006898fe58 R08: ffffffffa021e02a R09: ffff88006898fb48
[  220.785011] R10: ffffffffffffffe4 R11: 0000000000000001 R12: ffff88005b5e2540
[  220.785011] R13: ffff88005b668040 R14: ffff880077344800 R15: ffff88005b674578
[  220.785011] FS:  00007f8587265700(0000) GS:ffff88007bc00000(0000)
knlGS:0000000000000000
[  220.785011] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  220.785011] CR2: 00007f8587277000 CR3: 0000000068a29000 CR4: 00000000000006f0
[  220.785011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  220.785011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  220.785011] Process mc (pid: 2881, threadinfo ffff88006898e000, task
ffff880070a64230)
[  220.785011] Stack:
[  220.785011]  ffff88006898fe48 00000000fffffff0 ffff88005b5e2540
ffff88005b674198
[  220.785011]  0000000000000000 00007fff6ee8d6b0 ffff88006898fe88
ffffffff810ed420
[  220.785011]  ffff88006898fe88 ffff88006898fe98 ffff88005b5e2540
ffff88005b674578
[  220.785011] Call Trace:
[  220.785011]  [<ffffffff810ed420>] vfs_unlink+0x80/0xf0
[  220.785011]  [<ffffffff810ef773>] do_unlinkat+0x173/0x1b0
[  220.785011]  [<ffffffff81051a88>] ? sys_rt_sigaction+0x68/0xc0
[  220.785011]  [<ffffffff810f0cc1>] sys_unlink+0x11/0x20
[  220.785011]  [<ffffffff81574bbb>] system_call_fastpath+0x16/0x1b
[  220.785011] Code: 4c 8b 65 e0 48 8b 5d d8 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d f8
c9 c3 0f 1f 40 00 4c 89 fe 4c 89 ef e8 25 d0 ff ff 85 c0 74 bb 0f 0b <0f>
0b 89 c3 eb cd 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57
[  220.785011] RIP  [<ffffffffa024a141>] btrfs_unlink+0xd1/0xe0 [btrfs]
[  220.785011]  RSP <ffff88006898fe28>
[  220.809847] ---[ end trace ca8cb79df9da2e2a ]---


-- 

  Sergei

Sergei Trofimovich

2011-Apr-08 21:19 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

> > When a btrfs disk is created by mixed data & metadata option, it
will have no
> > pure data or pure metadata space info.
> > 
> > In btrfs''s for-linus branch, commit
78b1ea13838039cd88afdd62519b40b344d6c920
> > (Btrfs: fix OOPS of empty filesystem after balance) initializes space
infos at
> > the very beginning.  The problem is this initialization does not take
the mixed
> > case into account, which will cause btrfs will easily get into ENOSPC
in mixed
> > case.
> > 
> > Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
> > ---
> 
> Tried to apply this patch on top of 2.6.39-rc2 and booted on the same
partition.
> It has 3.2 GBs free. With patch it started to report -ENOSPC right out.
> Attempt to delete some files caused OOps:
Sorry, I was too fast. The same OOps is reproducible on vanilla 2.6.39-rc2. Will
retest on 2.6.39-rc1.

-- 

  Sergei

Sergei Trofimovich

2011-Apr-08 21:55 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

> > > In btrfs''s for-linus branch, commit
78b1ea13838039cd88afdd62519b40b344d6c920
> > > (Btrfs: fix OOPS of empty filesystem after balance) initializes
space infos at
> > > the very beginning.  The problem is this initialization does not
take the mixed
> > > case into account, which will cause btrfs will easily get into
ENOSPC in mixed
> > > case.
> > > 
> > > Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
> > > ---
> Sorry, I was too fast. The same OOps is reproducible on vanilla 2.6.39-rc2.
Will retest on 2.6.39-rc1.
2.6.39-rc1 (no patch):

[  155.038094] ------------[ cut here ]------------
[  155.038388] kernel BUG at /mnt/archive/src/linux-2.6/fs/btrfs/inode.c:2967!
[  155.038643] invalid opcode: 0000 [#1] PREEMPT SMP 
[  155.038907] last sysfs file: /sys/devices/virtual/bdi/0:26/uevent
[  155.039011] CPU 0 
[  155.039011] Modules linked in: bridge stp llc btrfs zlib_deflate
lzo_decompress lzo_compress crc32c libcrc32c tun kvm_amd kvm fuse nouveau ttm
8139cp drm_kms_helper drm i2c_algo_bit i2c_core cfbcopyarea cfbimgblt
cfbfillrect 8139too forcedeth
[  155.039011] 
[  155.039011] Pid: 2620, comm: rm Not tainted 2.6.39-rc1 #3 To Be Filled By
O.E.M. To Be Filled By O.E.M./ALiveNF6G-VSTA
[  155.039011] RIP: 0010:[<ffffffffa0249fd1>]  [<ffffffffa0249fd1>]
btrfs_unlink+0xd1/0xe0 [btrfs]
[  155.039011] RSP: 0018:ffff880073687e28  EFLAGS: 00010286
[  155.039011] RAX: 00000000ffffffe4 RBX: 00000000ffffffe4 RCX: 00000000000000e0
[  155.039011] RDX: 00000000000000de RSI: 00000000000196d0 RDI: ffffea0001a3d980
[  155.039011] RBP: ffff880073687e58 R08: ffffffffa021df4a R09: ffff880073687b48
[  155.039011] R10: ffffffffffffffe4 R11: 0000000000000001 R12: ffff8800756eab40
[  155.039011] R13: ffff8800756eb000 R14: ffff880075e8e800 R15: ffff880077f7e858
[  155.039011] FS:  00007f99bdd9c700(0000) GS:ffff88007bc00000(0000)
knlGS:0000000000000000
[  155.039011] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  155.039011] CR2: 00007f99be7480a8 CR3: 0000000073610000 CR4: 00000000000006f0
[  155.039011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  155.039011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  155.039011] Process rm (pid: 2620, threadinfo ffff880073686000, task
ffff8800730cc230)
[  155.039011] Stack:
[  155.039011]  ffff880073687e48 00000000fffffff0 ffff8800756eab40
ffff880077f7e478
[  155.039011]  0000000000000000 0000000000000000 ffff880073687e88
ffffffff810ed3a0
[  155.039011]  ffff880073687e88 ffff880073687e98 ffff8800756eab40
ffff880077f7e858
[  155.039011] Call Trace:
[  155.039011]  [<ffffffff810ed3a0>] vfs_unlink+0x80/0xf0
[  155.039011]  [<ffffffff810ef6f3>] do_unlinkat+0x173/0x1b0
[  155.039011]  [<ffffffff8111727b>] ? fsnotify_find_inode_mark+0x3b/0x50
[  155.039011]  [<ffffffff810dff91>] ? filp_close+0x61/0x90
[  155.039011]  [<ffffffff810f0c0d>] sys_unlinkat+0x1d/0x40
[  155.039011]  [<ffffffff81574c3b>] system_call_fastpath+0x16/0x1b
[  155.039011] Code: 4c 8b 65 e0 48 8b 5d d8 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d f8
c9 c3 0f 1f 40 00 4c 89 fe 4c 89 ef e8 05 d0 ff ff 85 c0 74 bb 0f 0b <0f>
0b 89 c3 eb cd 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57
[  155.039011] RIP  [<ffffffffa0249fd1>] btrfs_unlink+0xd1/0xe0 [btrfs]
[  155.039011]  RSP <ffff880073687e28>
[  155.064611] ---[ end trace 47dc7dae76c74a2a ]---

2.6.39-rc1 (+patch):
[  100.499105] ------------[ cut here ]------------
[  100.499405] kernel BUG at /mnt/archive/src/linux-2.6/fs/btrfs/inode.c:2967!
[  100.499686] invalid opcode: 0000 [#1] PREEMPT SMP 
[  100.499948] last sysfs file: /sys/devices/virtual/bdi/0:26/uevent
[  100.500011] CPU 1 
[  100.500011] Modules linked in: bridge stp llc btrfs zlib_deflate
lzo_decompress lzo_compress crc32c libcrc32c tun kvm_amd kvm fuse nouveau ttm
drm_kms_helper drm i2c_algo_bit i2c_core cfbcopyarea cfbimgblt cfbfillrect
forcedeth 8139cp 8139too
[  100.500011] 
[  100.500011] Pid: 2447, comm: rm Not tainted 2.6.39-rc1+ #4 To Be Filled By
O.E.M. To Be Filled By O.E.M./ALiveNF6G-VSTA
[  100.500011] RIP: 0010:[<ffffffffa024a011>]  [<ffffffffa024a011>]
btrfs_unlink+0xd1/0xe0 [btrfs]
[  100.500011] RSP: 0018:ffff880070b55e28  EFLAGS: 00010286
[  100.500011] RAX: 00000000ffffffe4 RBX: 00000000ffffffe4 RCX: 0000000000000013
[  100.500011] RDX: 0000000000000011 RSI: 00000000000196d0 RDI: ffffea00019aef80
[  100.500011] RBP: ffff880070b55e58 R08: ffffffffa021df4a R09: ffff880070b55b48
[  100.500011] R10: ffffffffffffffe4 R11: 0000000000000001 R12: ffff8800756a0c00
[  100.500011] R13: ffff880075692000 R14: ffff880075e3c800 R15: ffff880077f5e478
[  100.500011] FS:  00007f01e95c5700(0000) GS:ffff88007bd00000(0000)
knlGS:0000000000000000
[  100.500011] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  100.500011] CR2: 00007fd0687a8ec4 CR3: 0000000070b29000 CR4: 00000000000006e0
[  100.500011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  100.500011] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  100.500011] Process rm (pid: 2447, threadinfo ffff880070b54000, task
ffff8800709cc230)
[  100.500011] Stack:
[  100.500011]  ffff880070b55e48 00000000fffffff0 ffff8800756a0c00
ffff880077f5e098
[  100.500011]  0000000000000000 0000000000000000 ffff880070b55e88
ffffffff810ed3a0
[  100.500011]  ffff880070b55e88 ffff880070b55e98 ffff8800756a0c00
ffff880077f5e478
[  100.500011] Call Trace:
[  100.500011]  [<ffffffff810ed3a0>] vfs_unlink+0x80/0xf0
[  100.500011]  [<ffffffff810ef6f3>] do_unlinkat+0x173/0x1b0
[  100.500011]  [<ffffffff8111727b>] ? fsnotify_find_inode_mark+0x3b/0x50
[  100.500011]  [<ffffffff810dff91>] ? filp_close+0x61/0x90
[  100.500011]  [<ffffffff810f0c0d>] sys_unlinkat+0x1d/0x40
[  100.500011]  [<ffffffff81574c3b>] system_call_fastpath+0x16/0x1b
[  100.500011] Code: 4c 8b 65 e0 48 8b 5d d8 4c 8b 6d e8 4c 8b 75 f0 4c 8b 7d f8
c9 c3 0f 1f 40 00 4c 89 fe 4c 89 ef e8 05 d0 ff ff 85 c0 74 bb 0f 0b <0f>
0b 89 c3 eb cd 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57
[  100.500011] RIP  [<ffffffffa024a011>] btrfs_unlink+0xd1/0xe0 [btrfs]
[  100.500011]  RSP <ffff880070b55e28>
[  100.525672] ---[ end trace 7e63b9144b7307fe ]---

Looks like I won''t be able to test your patch until this thing will go
away first.

-- 

  Sergei

liubo

2011-Apr-11 06:29 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

On 04/09/2011 05:55 AM, Sergei Trofimovich wrote:> [  100.500011] Call Trace:
> [  100.500011]  [<ffffffff810ed3a0>] vfs_unlink+0x80/0xf0
> [  100.500011]  [<ffffffff810ef6f3>] do_unlinkat+0x173/0x1b0
> [  100.500011]  [<ffffffff8111727b>] ?
fsnotify_find_inode_mark+0x3b/0x50
> [  100.500011]  [<ffffffff810dff91>] ? filp_close+0x61/0x90
> [  100.500011]  [<ffffffff810f0c0d>] sys_unlinkat+0x1d/0x40
> [  100.500011]  [<ffffffff81574c3b>] system_call_fastpath+0x16/0x1b
> [  100.500011] Code: 4c 8b 65 e0 48 8b 5d d8 4c 8b 6d e8 4c 8b 75 f0 4c 8b
7d f8 c9 c3 0f 1f 40 00 4c 89 fe 4c 89 ef e8 05 d0 ff ff 85 c0 74 bb 0f 0b
<0f> 0b 89 c3 eb cd 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57
> [  100.500011] RIP  [<ffffffffa024a011>] btrfs_unlink+0xd1/0xe0
[btrfs]
> [  100.500011]  RSP <ffff880070b55e28>
> [  100.525672] ---[ end trace 7e63b9144b7307fe ]---
> 
> Looks like I won''t be able to test your patch until this thing
will go away first.
Thanks a lot for testing, though.

I guess something messed up your btrfs metadata, cause when btrfs_unlink()
wanted to remove A,
it found that A was just missing...

thanks,
liubo
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Sergei Trofimovich

2011-Apr-11 20:27 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

On Mon, 11 Apr 2011 14:29:21 +0800
liubo <liubo2009@cn.fujitsu.com> wrote:
> On 04/09/2011 05:55 AM, Sergei Trofimovich wrote:
> > [  100.500011] Call Trace:
> > [  100.500011]  [<ffffffff810ed3a0>] vfs_unlink+0x80/0xf0
> > [  100.500011]  [<ffffffff810ef6f3>] do_unlinkat+0x173/0x1b0
> > [  100.500011]  [<ffffffff8111727b>] ?
fsnotify_find_inode_mark+0x3b/0x50
> > [  100.500011]  [<ffffffff810dff91>] ? filp_close+0x61/0x90
> > [  100.500011]  [<ffffffff810f0c0d>] sys_unlinkat+0x1d/0x40
> > [  100.500011]  [<ffffffff81574c3b>]
system_call_fastpath+0x16/0x1b
> > [  100.500011] Code: 4c 8b 65 e0 48 8b 5d d8 4c 8b 6d e8 4c 8b 75 f0
4c 8b 7d f8 c9 c3 0f 1f 40 00 4c 89 fe 4c 89 ef e8 05 d0 ff ff 85 c0 74 bb 0f 0b
<0f> 0b 89 c3 eb cd 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57
> > [  100.500011] RIP  [<ffffffffa024a011>] btrfs_unlink+0xd1/0xe0
[btrfs]
> > [  100.500011]  RSP <ffff880070b55e28>
> > [  100.525672] ---[ end trace 7e63b9144b7307fe ]---
> > 
> > Looks like I won''t be able to test your patch until this
thing will go away first.
> 
> Thanks a lot for testing, though.
> 
> I guess something messed up your btrfs metadata, cause when btrfs_unlink()
wanted to remove A,
> it found that A was just missing...Looks like it''s ret = -28 (a ENOSPC). Yes, you are right. Moreover, as
Arne found out,
I used wrong patch for btrfs-progs to create --mixed filesystems. I set wrong
bit in
superblock (the 8ULL << 0, not 4ULL << 0) - the COMPRESS_LZO, so my
metadata is
really screwed.

Please, disregard all my OOpses reported against --mixed FS. Sorry.

-- 

  Sergei

Sergei Trofimovich

2011-Apr-19 21:55 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

On Mon, 11 Apr 2011 14:29:21 +0800
liubo <liubo2009@cn.fujitsu.com> wrote:
> On 04/09/2011 05:55 AM, Sergei Trofimovich wrote:
> > [  100.500011] Call Trace:
> > [  100.500011]  [<ffffffff810ed3a0>] vfs_unlink+0x80/0xf0
> > [  100.500011]  [<ffffffff810ef6f3>] do_unlinkat+0x173/0x1b0
> > [  100.500011]  [<ffffffff8111727b>] ?
fsnotify_find_inode_mark+0x3b/0x50
> > [  100.500011]  [<ffffffff810dff91>] ? filp_close+0x61/0x90
> > [  100.500011]  [<ffffffff810f0c0d>] sys_unlinkat+0x1d/0x40
> > [  100.500011]  [<ffffffff81574c3b>]
system_call_fastpath+0x16/0x1b
> > [  100.500011] Code: 4c 8b 65 e0 48 8b 5d d8 4c 8b 6d e8 4c 8b 75 f0
4c 8b 7d f8 c9 c3 0f 1f 40 00 4c 89 fe 4c 89 ef e8 05 d0 ff ff 85 c0 74 bb 0f 0b
<0f> 0b 89 c3 eb cd 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57
> > [  100.500011] RIP  [<ffffffffa024a011>] btrfs_unlink+0xd1/0xe0
[btrfs]
> > [  100.500011]  RSP <ffff880070b55e28>
> > [  100.525672] ---[ end trace 7e63b9144b7307fe ]---
> > 
> > Looks like I won''t be able to test your patch until this
thing will go away first.
> 
> Thanks a lot for testing, though.
> 
> I guess something messed up your btrfs metadata, cause when btrfs_unlink()
wanted to remove A,
> it found that A was just missing...
Hello again!

I''ve decided to explore OOps myself a bit. The result is a debugging
patch I''ve attached.
I''ve applied it and you patch
    [PATCH] Btrfs: fix easily get into ENOSPC in mixed case
on top of vanilla 2.6.39-rc4

The result you can see in attached ''fault.log''. There you can
see -ENOSPC propagation.
No idea why it converted to -ENOENT in the end. The result is very similar with
vanilla 2.6.39-rc4.
I can run the debugging patch on it as well if you like.


I think interesting the parts are (they were found by Arne before):
[    0.040000] new update_space_info: 
[    0.040000] space_info has 0 free, is not full
[    0.040000] space_info total=0, used=0, pinned=0, reserved=0, may_use=0,
readonly=0
[    0.040000] new update_space_info: 
[    0.040000] space_info has 0 free, is not full
[    0.040000] space_info total=0, used=0, pinned=0, reserved=0, may_use=0,
readonly=0
[    0.040000] new update_space_info: 
[    0.040000] space_info has 0 free, is not full
[    0.040000] space_info total=0, used=0, pinned=0, reserved=0, may_use=0,
readonly=0

and:
[    0.290000] shrink_delalloc: kick writeback_inodes_sb_nr_if_idle to free 0
pages (yet 0 bytes to wb).
[    0.300000] shrink_delalloc: kick writeback_inodes_sb_nr_if_idle to free 0
pages (yet 0 bytes to wb).
[    0.310000] shrink_delalloc: kick writeback_inodes_sb_nr_if_idle to free 0
pages (yet 0 bytes to wb).
[    0.320000] reserve_metadata_bytes: shrink_delalloc(num_bytes=983040) -> 0
[    0.320000] reserve_metadata_bytes: 
[    0.320000] space_info has 18446744073708568576 free, is not full
[    0.320000] space_info total=0, used=0, pinned=0, reserved=983040, may_use=0,
readonly=0

Here I see 2 problems:
1 (major) free space is busted (as the whole space_info).
2 (minor) the original loop of shrink_delalloc busy waited 30 seconds, so I had
to trim it;
   and it seemed to know that additional IO won''t reclaim any pages
(''yet 0 bytes'' string AFAIU);
   but it''s likely a result of busted space_info: it has
reserved=983040 more, than total=0.

If I''ll revert c59021f846881a957ac5afe456d0f59d6a517b61 from vanilla
2.6.39-rc4 I''ll be able to delete
the files from filesystem.

I perform tests in usermode linux,  and run ''rm -rf'' on the
same faulty partition snapshot.
I''ve backed it up, so it''s easy to reproduce and debug. I can
provide more info if you need.
It''s a ''readable'' (read-only btrfs mount behaves
fine) 5GB image.

-- 

  Sergei

Sergei Trofimovich

2011-Apr-21 15:19 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

On Wed, 20 Apr 2011 00:55:31 +0300
Sergei Trofimovich <slyich@gmail.com> wrote:
> > Thanks a lot for testing, though.
> > 
> > I guess something messed up your btrfs metadata, cause when
btrfs_unlink() wanted to remove A,
> > it found that A was just missing...
Yeah. Now I think I completely figured out what''s going on:

My mixed partition was correct except it did''n have MIXED bit in
superblock
(I mistakenly used LZO bit for that in btrfs-progs, as I took some old patch
from maillist).

Thus I had D+M marked (aka mixed) block groups, but from superblock''s
features
there was no way to figure out there is yet such groups. That''s why
your second patch
(looking correct) didn''t help me. I''ve fallen to mixed == 0
case.
> I think interesting the parts are (they were found by Arne before):
> [    0.040000] new update_space_info: 
> [    0.040000] space_info has 0 free, is not full
> [    0.040000] space_info total=0, used=0, pinned=0, reserved=0, may_use=0,
readonly=0
> [    0.040000] new update_space_info: 
> [    0.040000] space_info has 0 free, is not full
> [    0.040000] space_info total=0, used=0, pinned=0, reserved=0, may_use=0,
readonly=0
> [    0.040000] new update_space_info: 
> [    0.040000] space_info has 0 free, is not full
> [    0.040000] space_info total=0, used=0, pinned=0, reserved=0, may_use=0,
readonly=0
Talking with Arne I was convinced my breakage is not worth fixing in kernel.
After I''ve added MIXED bit to superblocks'' features OOpses
gone.

So, once again, please disregard my OOpses.

-- 

  Sergei

Sergei Trofimovich

2011-Apr-22 19:43 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

On Fri, 08 Apr 2011 16:44:37 +0800
liubo <liubo2009@cn.fujitsu.com> wrote:
> 
> When a btrfs disk is created by mixed data & metadata option, it will
have no
> pure data or pure metadata space info.
> 
> In btrfs''s for-linus branch, commit
78b1ea13838039cd88afdd62519b40b344d6c920
> (Btrfs: fix OOPS of empty filesystem after balance) initializes space infos
at
> the very beginning.  The problem is this initialization does not take the
mixed
> case into account, which will cause btrfs will easily get into ENOSPC in
mixed
> case.
> 
> Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
> ---
This morning I''ve got an ENOSPC OOps on --mixed (properly created this
time) FS
and 2.6.39-rc4 kernel. After applying this patch OOps gone away.

Unfortunately, I haven''t saved the trace and can''t reproduce
it yet.

-- 

  Sergei

Sergei Trofimovich

2011-May-05 14:44 UTC

head link

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

On Fri, 08 Apr 2011 16:44:37 +0800
liubo <liubo2009@cn.fujitsu.com> wrote:
> 
> When a btrfs disk is created by mixed data & metadata option, it will
have no
> pure data or pure metadata space info.
> 
> In btrfs''s for-linus branch, commit
78b1ea13838039cd88afdd62519b40b344d6c920
> (Btrfs: fix OOPS of empty filesystem after balance) initializes space infos
at
> the very beginning.  The problem is this initialization does not take the
mixed
> case into account, which will cause btrfs will easily get into ENOSPC in
mixed
> case.
> 
> Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
Hi Chris!

I confirm it fixes OOpses for me.
Some clarification of what happened down the thread with me in order to convince
this patch actually makes things better.

1. The initial OOps reported by me was a result of using bad mkfs (not
interesting,
    but generated a lot of noise here)

2. Then I''ve patched mkfs properly (used tmp branch), reformatted my
partitions and
    caught -ENOSPC on ''rm -rf /var/tmp/'' (vanilla kernel),
added debugging info and have come to the
    conclusion: this patch helps in this situation, as free space was completely
miscomputed
    (it''s a regression since 2.6.38). I''ve applied it and
things _mostly_ (see 3.) solved (--mixed
    ~4G lzo filesystem) for me

3. Then I''ve caught more -ENOSPC oopses, which occured to be a bug in
mkfs for which Arne posted
    a fix today, see his email:
        [PATCH] btrfs progs: fix extra metadata chunk allocation in --mixed case

    Arne''s mkfs patch (or btrfs fi balance workaround) fixed rest of
OOpses for me.

So, my thoughts:
    - this patch is quite critical for --mixed filesystems
    - it (or it''s equivalent) should be considered for pulling-in to
.39 kernel (ideally), as it''s a regression since 2.6.38

Thanks!
> ---
>  fs/btrfs/extent-tree.c |   37 ++++++++++++++++++++++++++-----------
>  1 files changed, 26 insertions(+), 11 deletions(-)
> 
> diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
> index f619c3c..1b47ae4 100644
> --- a/fs/btrfs/extent-tree.c
> +++ b/fs/btrfs/extent-tree.c
> @@ -8781,23 +8781,38 @@ out:
>  int btrfs_init_space_info(struct btrfs_fs_info *fs_info)
>  {
>  	struct btrfs_space_info *space_info;
> +	struct btrfs_super_block *disk_super;
> +	u64 features;
> +	u64 flags;
> +	int mixed = 0;
>  	int ret;
>  
> -	ret = update_space_info(fs_info, BTRFS_BLOCK_GROUP_SYSTEM, 0, 0,
> -								 &space_info);
> -	if (ret)
> -		return ret;
> +	disk_super = &fs_info->super_copy;
> +	if (!btrfs_super_root(disk_super))
> +		return 1;
>  
> -	ret = update_space_info(fs_info, BTRFS_BLOCK_GROUP_METADATA, 0, 0,
> -								 &space_info);
> -	if (ret)
> -		return ret;
> +	features = btrfs_super_incompat_flags(disk_super);
> +	if (features & BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS)
> +		mixed = 1;
>  
> -	ret = update_space_info(fs_info, BTRFS_BLOCK_GROUP_DATA, 0, 0,
> -								 &space_info);
> +	flags = BTRFS_BLOCK_GROUP_SYSTEM;
> +	ret = update_space_info(fs_info, flags, 0, 0, &space_info);
>  	if (ret)
> -		return ret;
> +		goto out;
>  
> +	if (mixed) {
> +		flags = BTRFS_BLOCK_GROUP_METADATA | BTRFS_BLOCK_GROUP_DATA;
> +		ret = update_space_info(fs_info, flags, 0, 0, &space_info);
> +	} else {
> +		flags = BTRFS_BLOCK_GROUP_METADATA;
> +		ret = update_space_info(fs_info, flags, 0, 0, &space_info);
> +		if (ret)
> +			goto out;
> +
> +		flags = BTRFS_BLOCK_GROUP_DATA;
> +		ret = update_space_info(fs_info, flags, 0, 0, &space_info);
> +	}
> +out:
>  	return ret;
>  }
>  
> -- 
> 1.6.5.2
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 

  Sergei

Btrfs devel - Apr 2011 - 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

Re: 2.6.39-rc1: kernel BUG at fs/btrfs/extent-tree.c:5479!

[PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case

Re: [PATCH] Btrfs: fix easily get into ENOSPC in mixed case