Ocfs2 users - Oct 2010 - Reg: ocfs2 two node cluster crashed, node2 crashed, when I rebooted node1 for maintenance.

Hi All,

We have ocfs2 node cluster with oracle 11G RAC running,

The node2 got crashed automatically, when i rebooted node one for
maintenance.

please check the log from node2 , before its got crashed.

Oct 23 15:42:25 node2 kernel: ocfs2_dlm: Nodes in domain
("029C02C993E44E90879922E268FB161A"): 2
Oct 23 15:42:29 node2 kernel: ocfs2_dlm: Node 1 leaves domain
2AB2C04A99BD482A89A7FCE9D3C9319A
Oct 23 15:42:29 node2 kernel: ocfs2_dlm: Nodes in domain
("2AB2C04A99BD482A89A7FCE9D3C9319A"): 2
Oct 23 15:42:33 node2 kernel: ocfs2_dlm: Node 1 leaves domain
B239262A386C465AA7DEE81C05F2EB93
Oct 23 15:42:33 node2 kernel: ocfs2_dlm: Nodes in domain
("B239262A386C465AA7DEE81C05F2EB93"): 2
Oct 23 15:42:38 node2 kernel: ocfs2_dlm: Node 1 leaves domain
C54B4F6991954F98AA6A37C4F3901CD8
Oct 23 15:42:38 node2 kernel: ocfs2_dlm: Nodes in domain
("C54B4F6991954F98AA6A37C4F3901CD8"): 2
Oct 23 15:42:58 node2 kernel: ocfs2_dlm: Node 1 leaves domain
D96AC8E8BDD54913AE6D8EC0EB539603
Oct 23 15:42:58 node2 kernel: ocfs2_dlm: Nodes in domain
("D96AC8E8BDD54913AE6D8EC0EB539603"): 2
Oct 23 15:44:06 node2 kernel: o2net: connection to node node1 (num 1) at
192.168.3.1:7777 has been idle for 60
.0 seconds, shutting it down.
Oct 23 15:44:06 node2 kernel: (swapper,0,15):o2net_idle_timer:1503 here are
some times that might help debug the situa
tion: (tmr 1287848586.872368 now 1287848646.872227 dr 1287848586.872346 adv
1287848586.872376:1287848586.872376 func (fb860756
:513) 1287848578.874476:1287848578.874487)
Oct 23 15:44:06 node2 kernel: o2net: no longer connected to node node1 (num
1) at 192.168.3.1:7777
Oct 23 15:45:06 node2 kernel: (o2net,14590,15):o2net_connect_expired:1664
ERROR: no connection established with node 1
 after 60.0 seconds, giving up and returning errors.
Oct 23 15:46:06 node2 kernel: (o2net,14590,15):o2net_connect_expired:1664
ERROR: no connection established with node 1
 after 60.0 seconds, giving up and returning errors.
Oct 23 15:51:34 node2 syslogd 1.4.1: restart.

Please guide me what could the issue.

Thanks
Veera.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://oss.oracle.com/pipermail/ocfs2-users/attachments/20101023/45f1c94a/attachment.html

Means that the reboot is not shutting down the services in order.
Ensure ocfs2 fs is unmounting before the network shutdown.

On 10/23/2010 11:27 AM, veeraa bose wrote:
> Hi All,
>
> We have ocfs2 node cluster with oracle 11G RAC running,
>
> The node2 got crashed automatically, when i rebooted node one for
maintenance.
>
> please check the log from node2 , before its got crashed.
>
> Oct 23 15:42:25 node2 kernel: ocfs2_dlm: Nodes in domain
("029C02C993E44E90879922E268FB161A"): 2
> Oct 23 15:42:29 node2 kernel: ocfs2_dlm: Node 1 leaves domain
2AB2C04A99BD482A89A7FCE9D3C9319A
> Oct 23 15:42:29 node2 kernel: ocfs2_dlm: Nodes in domain
("2AB2C04A99BD482A89A7FCE9D3C9319A"): 2
> Oct 23 15:42:33 node2 kernel: ocfs2_dlm: Node 1 leaves domain
B239262A386C465AA7DEE81C05F2EB93
> Oct 23 15:42:33 node2 kernel: ocfs2_dlm: Nodes in domain
("B239262A386C465AA7DEE81C05F2EB93"): 2
> Oct 23 15:42:38 node2 kernel: ocfs2_dlm: Node 1 leaves domain
C54B4F6991954F98AA6A37C4F3901CD8
> Oct 23 15:42:38 node2 kernel: ocfs2_dlm: Nodes in domain
("C54B4F6991954F98AA6A37C4F3901CD8"): 2
> Oct 23 15:42:58 node2 kernel: ocfs2_dlm: Node 1 leaves domain
D96AC8E8BDD54913AE6D8EC0EB539603
> Oct 23 15:42:58 node2 kernel: ocfs2_dlm: Nodes in domain
("D96AC8E8BDD54913AE6D8EC0EB539603"): 2
> Oct 23 15:44:06 node2 kernel: o2net: connection to node node1 (num 1) at
192.168.3.1:7777 <http://192.168.3.1:7777> has been idle for 60
> .0 seconds, shutting it down.
> Oct 23 15:44:06 node2 kernel: (swapper,0,15):o2net_idle_timer:1503 here are
some times that might help debug the situa
> tion: (tmr 1287848586.872368 now 1287848646.872227 dr 1287848586.872346 adv
1287848586.872376:1287848586.872376 func (fb860756
> :513) 1287848578.874476:1287848578.874487)
> Oct 23 15:44:06 node2 kernel: o2net: no longer connected to node node1 (num
1) at 192.168.3.1:7777 <http://192.168.3.1:7777>
> Oct 23 15:45:06 node2 kernel: (o2net,14590,15):o2net_connect_expired:1664
ERROR: no connection established with node 1
>  after 60.0 seconds, giving up and returning errors.
> Oct 23 15:46:06 node2 kernel: (o2net,14590,15):o2net_connect_expired:1664
ERROR: no connection established with node 1
>  after 60.0 seconds, giving up and returning errors.
> Oct 23 15:51:34 node2 syslogd 1.4.1: restart.
>
> Please guide me what could the issue.
>
> Thanks
> Veera.
>
>
>
> _______________________________________________
> Ocfs2-users mailing list
> Ocfs2-users at oss.oracle.com
> http://oss.oracle.com/mailman/listinfo/ocfs2-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://oss.oracle.com/pipermail/ocfs2-users/attachments/20101025/4d5f4864/attachment.html