Guys, I thought I share this on the list because we're using GlusterFS supplied patched fuse lib. I'm going to try the latest fuse lib now and see if it fixes things. There's nothing in the gluster logs. fuse-2.7.3glfs10 glusterfs--mainline--3.0--patch-359.>From the system log:BUG: unable to handle kernel paging request at virtual address 00100104 printing eip: f8ab3426 *pde = 254f2001 Oops: 0002 [#1] SMP Modules linked in: fuse w83627hf w83781d hwmon_vid hwmon i2c_isa i2c_piix4 i2c_i801 i2c_dev i2c_core CPU: 0 EIP: 0060:[pg0+944403494/1067394048] Not tainted VLI EFLAGS: 00010246 (2.6.18 #5) EIP is at request_end+0x3b/0xe9 [fuse] eax: 00100100 ebx: d301b320 ecx: d301b328 edx: 00200200 esi: e5e56480 edi: 00000000 ebp: e5103f10 esp: e5103ef4 ds: 007b es: 007b ss: 0068 Process glusterfs (pid: 8196, ti=e5102000 task=e6b82560 task.ti=e5102000) Stack: d301b320 e5e56480 00000000 f8ab4157 e5e56480 d301b320 00000030 e5e56480 00000001 d301b320 e5103f6c 00000000 00100008 b6cba000 c3436660 00000000 fffb9038 00000fc8 00000001 e5ce3ec0 b6cb9008 e5103fa4 00101000 f8ab41c5 Call Trace: [pg0+944406871/1067394048] fuse_dev_readv+0x1a8/0x1f1 [fuse] [pg0+944406981/1067394048] fuse_dev_read+0x25/0x29 [fuse] [vfs_read+136/286] vfs_read+0x88/0x11e [sys_read+59/99] sys_read+0x3b/0x63 [syscall_call+7/11] syscall_call+0x7/0xb Code: 8b bb 64 01 00 00 8d 4b 08 c7 83 64 01 00 00 00 00 00 00 89 02 89 50 04 c7 03 00 01 10 00 8b 43 08 c7 43 04 00 02 20 00 8b 51 04 <89> 50 04 89 02 c7 41 04 00 0 0 01 EIP: [pg0+944403494/1067394048] request_end+0x3b/0xe9 [fuse] SS:ESP 0068:e5103ef4 <3>BUG: soft lockup detected on CPU#0! [softlockup_tick+154/168] softlockup_tick+0x9a/0xa8 [update_process_times+56/93] update_process_times+0x38/0x5d [smp_apic_timer_interrupt+90/97] smp_apic_timer_interrupt+0x5a/0x61 [apic_timer_interrupt+31/36] apic_timer_interrupt+0x1f/0x24 [__func__.3+27/113] _spin_lock+0xa/0xf [pg0+944404797/1067394048] request_send+0x15/0x50 [fuse] [pg0+944416580/1067394048] fuse_flush+0xcd/0xf1 [fuse] [filp_close+53/87] filp_close+0x35/0x57 [sys_close+89/109] sys_close+0x59/0x6d [syscall_call+7/11] syscall_call+0x7/0xb BUG: soft lockup detected on CPU#1! [softlockup_tick+154/168] softlockup_tick+0x9a/0xa8 [update_process_times+56/93] update_process_times+0x38/0x5d [smp_apic_timer_interrupt+90/97] smp_apic_timer_interrupt+0x5a/0x61 [apic_timer_interrupt+31/36] apic_timer_interrupt+0x1f/0x24 [__func__.3+27/113] _spin_lock+0xa/0xf [pg0+944404873/1067394048] request_send_nowait+0x11/0x53 [fuse] [pg0+944417777/1067394048] fuse_readpages+0x7f/0xa0 [fuse] [read_pages+37/197] read_pages+0x25/0xc5 [get_page_from_freelist+112/136] get_page_from_freelist+0x70/0x88 [__alloc_pages+96/651] __alloc_pages+0x60/0x28b [__do_page_cache_readahead+277/309] __do_page_cache_readahead+0x115/0x135 [blockable_page_cache_readahead+70/149] blockable_page_cache_readahead+0x46/0x95 [make_ahead_window+120/153] make_ahead_window+0x78/0x99 [page_cache_readahead+282/312] page_cache_readahead+0x11a/0x138 [do_generic_mapping_read+282/1029] do_generic_mapping_read+0x11a/0x405 [__generic_file_aio_read+408/457] __generic_file_aio_read+0x198/0x1c9 [file_read_actor+0/234] file_read_actor+0x0/0xea [generic_file_read+172/197] generic_file_read+0xac/0xc5 [autoremove_wake_function+0/58] autoremove_wake_function+0x0/0x3a [border+32/96] mutex_lock+0xf/0x1a [generic_file_llseek+145/159] generic_file_llseek+0x91/0x9f [vfs_read+136/286] vfs_read+0x88/0x11e [sys_read+59/99] sys_read+0x3b/0x63 [syscall_call+7/11] syscall_call+0x7/0xb
Hi Arend, Can you tell your Kernel version? Regards, Amar 2008/12/2 Arend-Jan Wijtzes <ajwytzes at wise-guys.nl>> Guys, > > I thought I share this on the list because we're using GlusterFS supplied > patched > fuse lib. I'm going to try the latest fuse lib now and see if it fixes > things. > There's nothing in the gluster logs. > > > fuse-2.7.3glfs10 > glusterfs--mainline--3.0--patch-359. > > > From the system log: > > BUG: unable to handle kernel paging request at virtual address 00100104 > printing eip: > f8ab3426 > *pde = 254f2001 > Oops: 0002 [#1] > SMP > Modules linked in: fuse w83627hf w83781d hwmon_vid hwmon i2c_isa i2c_piix4 > i2c_i801 i2c_dev i2c_core > CPU: 0 > EIP: 0060:[pg0+944403494/1067394048] Not tainted VLI > EFLAGS: 00010246 (2.6.18 #5) > EIP is at request_end+0x3b/0xe9 [fuse] > eax: 00100100 ebx: d301b320 ecx: d301b328 edx: 00200200 > esi: e5e56480 edi: 00000000 ebp: e5103f10 esp: e5103ef4 > ds: 007b es: 007b ss: 0068 > Process glusterfs (pid: 8196, ti=e5102000 task=e6b82560 task.ti=e5102000) > Stack: d301b320 e5e56480 00000000 f8ab4157 e5e56480 d301b320 00000030 > e5e56480 > 00000001 d301b320 e5103f6c 00000000 00100008 b6cba000 c3436660 > 00000000 > fffb9038 00000fc8 00000001 e5ce3ec0 b6cb9008 e5103fa4 00101000 > f8ab41c5 > Call Trace: > [pg0+944406871/1067394048] fuse_dev_readv+0x1a8/0x1f1 [fuse] > [pg0+944406981/1067394048] fuse_dev_read+0x25/0x29 [fuse] > [vfs_read+136/286] vfs_read+0x88/0x11e > [sys_read+59/99] sys_read+0x3b/0x63 > [syscall_call+7/11] syscall_call+0x7/0xb > Code: 8b bb 64 01 00 00 8d 4b 08 c7 83 64 01 00 00 00 00 00 00 89 02 89 50 > 04 c7 03 00 01 10 00 8b 43 08 c7 43 04 00 02 20 00 8b 51 04 <89> 50 04 89 02 > c7 41 04 00 0 > 0 01 > EIP: [pg0+944403494/1067394048] request_end+0x3b/0xe9 [fuse] SS:ESP > 0068:e5103ef4 > <3>BUG: soft lockup detected on CPU#0! > [softlockup_tick+154/168] softlockup_tick+0x9a/0xa8 > [update_process_times+56/93] update_process_times+0x38/0x5d > [smp_apic_timer_interrupt+90/97] smp_apic_timer_interrupt+0x5a/0x61 > [apic_timer_interrupt+31/36] apic_timer_interrupt+0x1f/0x24 > [__func__.3+27/113] _spin_lock+0xa/0xf > [pg0+944404797/1067394048] request_send+0x15/0x50 [fuse] > [pg0+944416580/1067394048] fuse_flush+0xcd/0xf1 [fuse] > [filp_close+53/87] filp_close+0x35/0x57 > [sys_close+89/109] sys_close+0x59/0x6d > [syscall_call+7/11] syscall_call+0x7/0xb > BUG: soft lockup detected on CPU#1! > [softlockup_tick+154/168] softlockup_tick+0x9a/0xa8 > [update_process_times+56/93] update_process_times+0x38/0x5d > [smp_apic_timer_interrupt+90/97] smp_apic_timer_interrupt+0x5a/0x61 > [apic_timer_interrupt+31/36] apic_timer_interrupt+0x1f/0x24 > [__func__.3+27/113] _spin_lock+0xa/0xf > [pg0+944404873/1067394048] request_send_nowait+0x11/0x53 [fuse] > [pg0+944417777/1067394048] fuse_readpages+0x7f/0xa0 [fuse] > [read_pages+37/197] read_pages+0x25/0xc5 > [get_page_from_freelist+112/136] get_page_from_freelist+0x70/0x88 > [__alloc_pages+96/651] __alloc_pages+0x60/0x28b > [__do_page_cache_readahead+277/309] __do_page_cache_readahead+0x115/0x135 > [blockable_page_cache_readahead+70/149] > blockable_page_cache_readahead+0x46/0x95 > [make_ahead_window+120/153] make_ahead_window+0x78/0x99 > [page_cache_readahead+282/312] page_cache_readahead+0x11a/0x138 > [do_generic_mapping_read+282/1029] do_generic_mapping_read+0x11a/0x405 > [__generic_file_aio_read+408/457] __generic_file_aio_read+0x198/0x1c9 > [file_read_actor+0/234] file_read_actor+0x0/0xea > [generic_file_read+172/197] generic_file_read+0xac/0xc5 > [autoremove_wake_function+0/58] autoremove_wake_function+0x0/0x3a > [border+32/96] mutex_lock+0xf/0x1a > [generic_file_llseek+145/159] generic_file_llseek+0x91/0x9f > [vfs_read+136/286] vfs_read+0x88/0x11e > [sys_read+59/99] sys_read+0x3b/0x63 > [syscall_call+7/11] syscall_call+0x7/0xb > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users > >-- Amar Tumballi Gluster/GlusterFS Hacker [bulde on #gluster/irc.gnu.org] http://www.zresearch.com - Commoditizing Super Storage! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20081202/37df66a8/attachment.html>
On Tue, Dec 02, 2008 at 12:45:10PM -0800, Amar S. Tumballi wrote:> Hi Arend, > > Can you tell your Kernel version?Sure, the machine is a SuperMicro 6015B-T running 2.6.18. Aside from this fuse related crash it has been stable as a rock for quite some time, periodically also under very heavy load. regards, Arend-Jan