got this error today... what could it be? thanks ocfs2-tools 1.4.1-1 kernel 2.6.26-bpo.1-686 kernel OCFS2 1.5.0 Dec 10 16:47:12 kernel: [786537.854113] (12938,1):dlmunlock_common:128 ERROR: lockres F0000000000000000a5619c7fd2adf2: Someone is calling dlmunlock while waiting for an ast!<3>(12938,1):dlmunlock:685 ERROR: dlm status = DLM_BADPARAM Dec 10 16:47:12 kernel: [786537.854113] (12938,1):ocfs2_cancel_convert:3001 ERROR: DLM error -22 while calling ocfs2_dlm_unlock on resource F0000 000000000000a5619c7fd2adf2 Dec 10 16:47:12 kernel: [786537.854113] (12938,1):ocfs2_flock_handle_signal:1505 ERROR: status = -22 Dec 10 16:47:12 kernel: [786537.854113] (12938,1):ocfs2_do_flock:79 ERROR: status = -22 Dec 10 16:47:12 kernel: [786537.854113] ------------[ cut here ]------------ Dec 10 16:47:12 kernel: [786537.854113] kernel BUG at fs/ocfs2/dlmglue.c:678! Dec 10 16:47:12 kernel: [786537.854113] invalid opcode: 0000 [#1] SMP Dec 10 16:47:12 kernel: [786537.854113] Modules linked in: ocfs2 ipv6 ac battery ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_s tackglue configfs drbd cn dm_snapshot dm_mirror dm_log dm_mod loop i2c_i801 parport_pc snd_pcm snd_timer parport iTCO_wdt floppy snd soundcore snd_page_alloc i2c_core pcspkr e1000e container button evdev ext3 jbd mbcache raid456 md_mod async_xor async_memcpy async_tx xor sd_mod ata_generic ata_piix libata scsi_mo d dock ehci_hcd uhci_hcd usbcore thermal processor fan thermal_sys Dec 10 16:47:12 kernel: [786537.854113] Dec 10 16:47:12 kernel: [786537.854113] Pid: 4255, comm: dlm_thread Not tainted (2.6.26-bpo.1-686 #1) Dec 10 16:47:12 kernel: [786537.854113] EIP: 0060:[<f92d5f64>] EFLAGS: 00010046 CPU: 1 Dec 10 16:47:12 kernel: [786537.854113] EIP is at ocfs2_locking_ast +0x210/0x44a [ocfs2] Dec 10 16:47:12 kernel: [786537.854113] EAX: 00000241 EBX: f60c1dd4 ECX: 00000282 EDX: 00000002 Dec 10 16:47:12 kernel: [786537.854113] ESI: 00000002 EDI: f60c1ddc EBP: f6000400 ESP: f61c5f58 Dec 10 16:47:12 kernel: [786537.854113] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 Dec 10 16:47:12 kernel: [786537.854113] Process dlm_thread (pid: 4255, ti=f61c4000 task=f792a8c0 task.ti=f61c4000) Dec 10 16:47:12 kernel: [786537.854113] Stack: 00000282 d4cc0dc0 f8b8401e f6ccf200 cb4e62c0 f8b8402f f921e7fe f6ccf200 Dec 10 16:47:12 kernel: [786537.854113] d4cc0dc0 cb4e62c0 d26fe6c0 f920fb4e f6ccf200 000003e8 00000061 0000004e Dec 10 16:47:12 kernel: [786537.854113] 00000000 00000001 f6ff78d8 05050001 f6ff78e0 d8e27e98 d8e27ea0 00000000 Dec 10 16:47:12 kernel: [786537.854113] Call Trace: Dec 10 16:47:12 kernel: [786537.854113] [<f8b8401e>] o2dlm_lock_ast_wrapper+0x0/0x12 [ocfs2_stack_o2cb] Dec 10 16:47:12 kernel: [786537.854113] [<f8b8402f>] o2dlm_lock_ast_wrapper+0x11/0x12 [ocfs2_stack_o2cb] Dec 10 16:47:12 kernel: [786537.854161] [<f921e7fe>] dlm_do_local_ast +0x6c/0x71 [ocfs2_dlm] Dec 10 16:47:12 kernel: [786537.854161] [<f920fb4e>] dlm_thread +0xbb5/0xfba [ocfs2_dlm] Dec 10 16:47:12 kernel: [786537.854161] [<c0131938>] autoremove_wake_function+0x0/0x2d Dec 10 16:47:12 kernel: [786537.854161] [<f920ef99>] dlm_thread +0x0/0xfba [ocfs2_dlm] Dec 10 16:47:12 kernel: [786537.854161] [<c0131877>] kthread+0x38/0x5d Dec 10 16:47:12 kernel: [786537.854161] [<c013183f>] kthread+0x0/0x5d Dec 10 16:47:12 kernel: [786537.854161] [<c01044ff>] kernel_thread_helper+0x7/0x10 Dec 10 16:47:12 kernel: [786537.854161] ======================Dec 10 16:47:12 kernel: [786537.854161] Code: 04 30 f9 64 a1 04 40 3b c0 50 64 8b 15 00 40 3b c0 ff b2 10 01 00 00 68 4c 49 30 f9 e8 68 cf e4 c6 83 c4 14 8b 43 20 a8 02 75 04 <0f> 0b eb fe a8 01 75 04 0f 0b eb fe 83 7b 44 00 75 15 8b 43 04 Dec 10 16:47:12 kernel: [786537.854161] EIP: [<f92d5f64>] ocfs2_locking_ast+0x210/0x44a [ocfs2] SS:ESP 0068:f61c5f58 Dec 10 16:47:12 kernel: [786537.854161] ---[ end trace 4d63f8e548025b0f ]--- -- Lorenzo Milesi - lorenzo.milesi at yetopen.it YetOpen S.r.l. - http://www.yetopen.it/ C.so E. Filiberto, 74 23900 Lecco - ITALY - Tel 0341 220 205 - Fax 178 607 8199 GPG/PGP Key-Id: 0xE704E230 - http://keyserver.linux.it -------- D.Lgs. 196/2003 -------- Si avverte che tutte le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Nel caso in cui questo messaggio Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo, a non inoltrarlo a terzi e ad avvertirci non appena possibile. Grazie.
Please file a bugzilla in oss.oracle.com/bugzilla. Lorenzo Milesi wrote:> got this error today... what could it be? > thanks > > > ocfs2-tools 1.4.1-1 > kernel 2.6.26-bpo.1-686 > kernel OCFS2 1.5.0 > > > Dec 10 16:47:12 kernel: [786537.854113] (12938,1):dlmunlock_common:128 > ERROR: lockres F0000000000000000a5619c7fd2adf2: Someone is calling > dlmunlock while waiting for an ast!<3>(12938,1):dlmunlock:685 ERROR: dlm > status = DLM_BADPARAM > Dec 10 16:47:12 kernel: [786537.854113] > (12938,1):ocfs2_cancel_convert:3001 ERROR: DLM error -22 while calling > ocfs2_dlm_unlock on resource F0000 000000000000a5619c7fd2adf2 > Dec 10 16:47:12 kernel: [786537.854113] > (12938,1):ocfs2_flock_handle_signal:1505 ERROR: status = -22 > Dec 10 16:47:12 kernel: [786537.854113] (12938,1):ocfs2_do_flock:79 > ERROR: status = -22 > Dec 10 16:47:12 kernel: [786537.854113] ------------[ cut > here ]------------ > Dec 10 16:47:12 kernel: [786537.854113] kernel BUG at > fs/ocfs2/dlmglue.c:678! > Dec 10 16:47:12 kernel: [786537.854113] invalid opcode: 0000 [#1] SMP > Dec 10 16:47:12 kernel: [786537.854113] Modules linked in: ocfs2 ipv6 ac > battery ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_s > tackglue configfs drbd cn dm_snapshot dm_mirror dm_log dm_mod loop > i2c_i801 parport_pc snd_pcm snd_timer parport iTCO_wdt floppy snd > soundcore snd_page_alloc i2c_core pcspkr e1000e container button evdev > ext3 jbd mbcache raid456 md_mod async_xor async_memcpy async_tx xor > sd_mod ata_generic ata_piix libata scsi_mo d dock ehci_hcd uhci_hcd > usbcore thermal processor fan thermal_sys > Dec 10 16:47:12 kernel: [786537.854113] > Dec 10 16:47:12 kernel: [786537.854113] Pid: 4255, comm: dlm_thread Not > tainted (2.6.26-bpo.1-686 #1) > Dec 10 16:47:12 kernel: [786537.854113] EIP: 0060:[<f92d5f64>] EFLAGS: > 00010046 CPU: 1 > Dec 10 16:47:12 kernel: [786537.854113] EIP is at ocfs2_locking_ast > +0x210/0x44a [ocfs2] > Dec 10 16:47:12 kernel: [786537.854113] EAX: 00000241 EBX: f60c1dd4 ECX: > 00000282 EDX: 00000002 > Dec 10 16:47:12 kernel: [786537.854113] ESI: 00000002 EDI: f60c1ddc EBP: > f6000400 ESP: f61c5f58 > Dec 10 16:47:12 kernel: [786537.854113] DS: 007b ES: 007b FS: 00d8 GS: > 0000 SS: 0068 > Dec 10 16:47:12 kernel: [786537.854113] Process dlm_thread (pid: 4255, > ti=f61c4000 task=f792a8c0 task.ti=f61c4000) > Dec 10 16:47:12 kernel: [786537.854113] Stack: 00000282 d4cc0dc0 > f8b8401e f6ccf200 cb4e62c0 f8b8402f f921e7fe f6ccf200 > Dec 10 16:47:12 kernel: [786537.854113] d4cc0dc0 cb4e62c0 > d26fe6c0 f920fb4e f6ccf200 000003e8 00000061 0000004e > Dec 10 16:47:12 kernel: [786537.854113] 00000000 00000001 > f6ff78d8 05050001 f6ff78e0 d8e27e98 d8e27ea0 00000000 > Dec 10 16:47:12 kernel: [786537.854113] Call Trace: > Dec 10 16:47:12 kernel: [786537.854113] [<f8b8401e>] > o2dlm_lock_ast_wrapper+0x0/0x12 [ocfs2_stack_o2cb] > Dec 10 16:47:12 kernel: [786537.854113] [<f8b8402f>] > o2dlm_lock_ast_wrapper+0x11/0x12 [ocfs2_stack_o2cb] > Dec 10 16:47:12 kernel: [786537.854161] [<f921e7fe>] dlm_do_local_ast > +0x6c/0x71 [ocfs2_dlm] > Dec 10 16:47:12 kernel: [786537.854161] [<f920fb4e>] dlm_thread > +0xbb5/0xfba [ocfs2_dlm] > Dec 10 16:47:12 kernel: [786537.854161] [<c0131938>] > autoremove_wake_function+0x0/0x2d > Dec 10 16:47:12 kernel: [786537.854161] [<f920ef99>] dlm_thread > +0x0/0xfba [ocfs2_dlm] > Dec 10 16:47:12 kernel: [786537.854161] [<c0131877>] kthread+0x38/0x5d > Dec 10 16:47:12 kernel: [786537.854161] [<c013183f>] kthread+0x0/0x5d > Dec 10 16:47:12 kernel: [786537.854161] [<c01044ff>] > kernel_thread_helper+0x7/0x10 > Dec 10 16:47:12 kernel: [786537.854161] ======================> Dec 10 16:47:12 kernel: [786537.854161] Code: 04 30 f9 64 a1 04 40 3b c0 > 50 64 8b 15 00 40 3b c0 ff b2 10 01 00 00 68 4c 49 30 f9 e8 68 cf e4 c6 > 83 c4 14 8b 43 20 a8 02 75 04 <0f> 0b eb fe a8 01 75 04 0f 0b eb fe 83 > 7b 44 00 75 15 8b 43 04 > Dec 10 16:47:12 kernel: [786537.854161] EIP: [<f92d5f64>] > ocfs2_locking_ast+0x210/0x44a [ocfs2] SS:ESP 0068:f61c5f58 > Dec 10 16:47:12 kernel: [786537.854161] ---[ end trace > 4d63f8e548025b0f ]--- > >
Sunil Mushran
2008-Dec-16 21:52 UTC
[Ocfs2-users] Unsual Segfault (but reboot did not occur and node stayed offline)
$ cat /proc/sys/kernel/panic_on_oops What does this return. If 0, then that is the cause of the problem. It should be 1. David Murphy wrote:> My logs on Node Id 3: > > > Dec 16 06:44:03 web3 syslogd 1.5.0#1ubuntu1: restart. > Dec 16 08:43:31 web3 kernel: [10727560.835261] Modules linked in: vmmemctl > ocfs2 ocfs2_dlmfs ocfs2_dlm ocfs2_nodemanager configfs vmhgfs ext2 > dm_round_robin crc32c libcrc32c iscsi_tcp libiscsi scsi_transport_iscsi lp > loop ipv6 parport_pc parport psmouse evdev serio_raw pcspkr i2c_piix4 > i2c_core container ac button intel_agp agpgart dm_multipath dm_mod ext3 jbd > mbcache sr_mod cdrom sg sd_mod ata_piix pata_acpi floppy pcnet32 ata_generic > mii mptspi mptscsih mptbase scsi_transport_spi libata scsi_mod thermal > processor fan vmxnet vesafb fbcon tileblit font bitblit softcursor > Dec 16 08:43:31 web3 kernel: [10727560.843108] > Dec 16 08:43:31 web3 kernel: [10727560.843900] Pid: 4856, comm: o2net Not > tainted (2.6.24-19-virtual #1) > Dec 16 08:43:31 web3 kernel: [10727560.844724] EIP: 0062:[<f8e682bb>] > EFLAGS: 00010202 CPU: 0 > Dec 16 08:43:31 web3 kernel: [10727560.845566] EIP is at > __dlm_print_one_lock_resource+0x9db/0x9f0 [ocfs2_dlm] > Dec 16 08:43:31 web3 kernel: [10727560.846385] EAX: 00000001 EBX: 0000001f > ECX: 00000000 EDX: 00000000 > Dec 16 08:43:31 web3 kernel: [10727560.849779] ESI: f75e8c00 EDI: 00000000 > EBP: ec774700 ESP: df877d34 > Dec 16 08:43:31 web3 kernel: [10727560.851900] DS: 007b ES: 007b FS: 00d8 > GS: 0000 SS: 006a > Dec 16 08:43:31 web3 kernel: [10727560.906502] ---[ end trace > 989a5ffd1351fea4 ]--- > Dec 16 08:44:01 web3 kernel: [10727590.622434] o2net: connection to node > deploy (num 5) at 192.168.102.12:7777 has been idle for 30.0 seconds, > shutting it down. > Dec 16 08:44:01 web3 kernel: [10727590.627319] (4,0):o2net_idle_timer:1414 > here are some times that might help debug the situation: (tmr > 1229438611.731225 now 1229438641.727360 dr 1229438613.731191 adv > 1229438611.731227:1229438611.731228 func (a9b6ebe7:504) > 1229438600.868142:1229438600.868149) > Dec 16 08:44:01 web3 kernel: [10727590.629281] o2net: connection to node > app1 (num 6) at 192.168.102.10:7777 has been idle for 30.0 seconds, shutting > it down. > Dec 16 08:44:01 web3 kernel: [10727590.630630] (4,0):o2net_idle_timer:1414 > here are some times that might help debug the situation: (tmr > 1229438611.731486 now 1229438641.734226 dr 1229438634.811356 adv > 1229438611.731488:1229438611.731489 func (a9b6ebe7:502) > 1229438610.482837:1229438610.482839) > Dec 16 08:44:01 web3 kernel: [10727590.632818] o2net: connection to node > rgapp1 (num 4) at 192.168.102.11:7777 has been idle for 30.0 seconds, > shutting it down. > Dec 16 08:44:01 web3 kernel: [10727590.634937] (4,0):o2net_idle_timer:1414 > here are some times that might help debug the situation: (tmr > 1229438611.736146 now 1229438641.737771 dr 1229438613.756472 adv > 1229438611.736149:1229438611.736149 func (a9b6ebe7:503) > 1229438611.735983:1229438611.735988) > Dec 16 08:44:01 web3 kernel: [10727590.640618] o2net: connection to node > web1 (num 1) at 192.168.102.40:7777 has been idle for 30.0 seconds, shutting > it down. > Dec 16 08:44:01 web3 kernel: [10727590.642402] (4,0):o2net_idle_timer:1414 > here are some times that might help debug the situation: (tmr > 1229438611.742904 now 1229438641.745604 dr 1229438617.734942 adv > 1229438611.742907:1229438611.742907 func (a9b6ebe7:504) > 1229438611.675070:1229438611.675075) > Dec 16 08:44:01 web3 kernel: [10727590.651745] o2net: connection to node > web2 (num 2) at 192.168.102.41:7777 has been idle for 30.0 seconds, shutting > it down. > Dec 16 08:44:01 web3 kernel: [10727590.657208] (0,0):o2net_idle_timer:1414 > here are some times that might help debug the situation: (tmr > 1229438611.756791 now 1229438641.756770 dr 1229438641.756769 adv > 1229438611.756768:1229438611.756697 func (a9b6ebe7:507) > 1229438611.756792:1229438611.746230) > > > > On the other nodes they ended up locking up waiting for death notification > of Node3. > Can anyone tell me with the kernel message above means and what I can to to > keep this from occurring again > > > Thanks > David > > > _______________________________________________ > Ocfs2-users mailing list > Ocfs2-users at oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs2-users >