Neil Campbell
2011-Jun-17 02:16 UTC
[Ocfs2-users] Unable to umount a filesystem - OCFS still thinks it has it mounted?
Hi all, Not sure what has happened but I am trying to remove a directory that I thought was copied from an OCFS filesystem. When I try and remove it I get the typical Device or resource busy error. This is on RHEL 4 update 8 kernel 2.6.9-89.0.26.ELsmp running OCFS2 version OCFS2 1.2.9 Mon Jan 17 07:38:31 EST 2011 (build 5e8325ec7f66b5189c65c7a8710fe8cb) Running lsof (as root) on the directory returns nothing. If I do a mount | grep ocfs I get the following (which is what I expect) # mount | grep ocfs ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw) /dev/emcpowerq1 on /d00/cfs type ocfs2 (rw,_netdev,heartbeat=local) /dev/emcpowere1 on /d00/html type ocfs2 (rw,_netdev,heartbeat=local) However, under cat /proc/fs/ocfs2 there are 3 device entries ls -l /proc/fs/ocfs2/ dr-xr-xr-x 2 root root 0 Jun 9 15:51 120_257 dr-xr-xr-x 2 root root 0 Jun 9 15:51 120_65 dr-xr-xr-x 2 root root 0 Jun 9 15:51 120_66 -r--r--r-- 1 root root 0 Jun 9 15:51 nodename -r--r--r-- 1 root root 0 Jun 9 15:51 version But on the other 3 servers in this cluster there are only 2 If I look at /dev for those devices they are as follows [root at dc-c02b01-app01 log]# ls -l /dev | grep 120 | egrep '257|65|66' brw------- 1 root root 120, 65 Feb 2 08:16 emcpowere1 brw------- 1 root root 120, 66 Feb 2 08:16 emcpowere2 brw------- 1 root root 120, 257 Feb 2 08:16 emcpowerq1 emcpowere2 should be umounted and doesn't show up when running mount, but it seems OCFS still thinks it is mounted. So what can I do to release this ? Many thanks Neil ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Downer This message is for the named person's use only. It may contain confidential, proprietary or legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you receive this message in error, please immediately delete it and all copies of it from your system, destroy any hard copies of it and notify the sender. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Downer EDI and any of its subsidiaries each reserve the right to monitor all e-mail communications through its networks. Any views expressed in this message are those of the individual sender, except where the message states otherwise and the sender is authorized to state them to be the views of any such entity. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110617/0fbfe747/attachment.html
Sunil Mushran
2011-Jun-17 06:32 UTC
[Ocfs2-users] Unable to umount a filesystem - OCFS still thinks it has it mounted?
Check /proc/mounts. That's the kernels view of the mounts. mount looks at /etc/mtab. And ocfs2 1.2 adds and removes entries /proc/fs/ocfs2 during mount/umount. Also, see if there are relevant errors in dmesg. On 06/16/2011 07:16 PM, Neil Campbell wrote:> > Hi all, > > Not sure what has happened but I am trying to remove a directory that > I thought was copied from an OCFS filesystem. > > When I try and remove it I get the typical Device or resource busy error. > > This is on RHEL 4 update 8 kernel 2.6.9-89.0.26.ELsmp running OCFS2 > version OCFS2 1.2.9 Mon Jan 17 07:38:31 EST 2011 (build > 5e8325ec7f66b5189c65c7a8710fe8cb) > > Running lsof (as root) on the directory returns nothing. > > If I do a mount | grep ocfs I get the following (which is what I expect) > > # mount | grep ocfs > ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw) > /dev/emcpowerq1 on /d00/cfs type ocfs2 (rw,_netdev,heartbeat=local) > /dev/emcpowere1 on /d00/html type ocfs2 (rw,_netdev,heartbeat=local) > > > However, under cat /proc/fs/ocfs2 there are 3 device entries > > ls -l /proc/fs/ocfs2/ > > dr-xr-xr-x 2 root root 0 Jun 9 15:51 120_257 > dr-xr-xr-x 2 root root 0 Jun 9 15:51 120_65 > dr-xr-xr-x 2 root root 0 Jun 9 15:51 120_66 > -r--r--r-- 1 root root 0 Jun 9 15:51 nodename > -r--r--r-- 1 root root 0 Jun 9 15:51 version > > > But on the other 3 servers in this cluster there are only 2 > > If I look at /dev for those devices they are as follows > > [root at dc-c02b01-app01 log]# ls -l /dev | grep 120 | egrep '257|65|66' > brw------- 1 root root 120, 65 Feb 2 08:16 emcpowere1 > brw------- 1 root root 120, 66 Feb 2 08:16 emcpowere2 > brw------- 1 root root 120, 257 Feb 2 08:16 emcpowerq1 > > > emcpowere2 should be umounted and doesn't show up when running mount, > but it seems OCFS still thinks it is mounted. > > So what can I do to release this ? > > Many thanks > Neil >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110616/a3fb46e7/attachment.html
Shave, Chris
2011-Jun-26 01:03 UTC
[Ocfs2-users] Heartbeat stays active & stops o2cb shutdown
Hi, I have an issue with shutting down o2cb & offlining the cluster, the heartbeat is staying active & is blocking any attempts to shut it down, despite there being zero ocfs2 filesystems mounted. This is what I see, even happens if using force-offline option: [root]# /etc/init.d/o2cb force-offline clustername Stopping O2CB cluster clustername Failed Unable to stop cluster as heartbeat region still active I have no ocfs2 filesystems curently mounted on either node (2 node cluster) [root]# mount | grep ocfs [root]# Versions of ocfs2 as below: [root]# rpm -qa | grep ocfs ocfs2-tools-1.4.4-1.el5.x86_64 ocfs2-tools-devel-1.4.4-1.el5.x86_64 ocfs2console-1.4.4-1.el5.x86_64 ocfs2-2.6.18-128.el5-1.4.4-1.el5.x86_64 ocfs2-tools-debuginfo-1.4.4-1.el5.x86_64 Redhat Linux kernel version: 2.6.18-128.el5 A collegue of mine stated that he usually disables all ocfs2 from the startup scripts, comments out the filesystems in /etc/fstab & reboots, is there another option to get the heartbeat offline or is this an ocfs2 or Linux bug I am encountering here?? Cheers, Christopher Shave, Global UNIX/Linux Projects Team Marsh & McLennan Companies Global Technology Infrastructure (MGTI) | Centralised Operations 555 Lonsdale Street, Level 5, Melbourne, VIC 3000, Australia +61 3 9623 5488 | Mobile +61 0402 885 057 | chris.shave at mercer.com<mailto:chris.shave at mercer.com> www.mmc.com<http://www.mmc.com/> ________________________________ This e-mail and any attachments may be confidential or legally privileged. If you received this message in error or are not the intended recipient, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained herein. Please inform us of the erroneous delivery by return e-mail. Thank you for your cooperation. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110626/00d706b3/attachment.html
Sunil Mushran
2011-Jun-27 17:15 UTC
[Ocfs2-users] Heartbeat stays active & stops o2cb shutdown
So by default, the hb is supposed to stop on umount. Do: # find /sys/kernel/config/cluster/<CLUSTERNAME>/heartbeat/* -type d | xargs basename 77D95EF51C0149D2823674FCC162CF8B This will list the active heartbeats. For each hb, do: # ocfs2_hb_ctl -I -u 77D95EF51C0149D2823674FCC162CF8B 77D95EF51C0149D2823674FCC162CF8B: 1 refs Notice the references. > 0 is active heartbeat. If you are sure there are no mounts and "ls /dlm" also has not entries, then hb failed to stop for some reason. To stop, do: # ocfs2_hb_ctl -K -u 77D95EF51C0149D2823674FCC162CF8B It could be that this is failing. What do you see? I remember we had a problem in this in tools 1.4.1. But that was fixed in 1.4.2. Sunil On 06/25/2011 06:03 PM, Shave, Chris wrote:> Hi, > I have an issue with shutting down o2cb & offlining the cluster, the heartbeat is staying active & is blocking any attempts to shut it down, despite there being zero ocfs2 filesystems mounted. > This is what I see, even happens if using force-offline option: > [root]# /etc/init.d/o2cb force-offline clustername > Stopping O2CB cluster clustername Failed > Unable to stop cluster as heartbeat region still active > I have no ocfs2 filesystems curently mounted on either node (2 node cluster) > [root]# mount | grep ocfs > [root]# > Versions of ocfs2 as below: > [root]# rpm -qa | grep ocfs > ocfs2-tools-1.4.4-1.el5.x86_64 > ocfs2-tools-devel-1.4.4-1.el5.x86_64 > ocfs2console-1.4.4-1.el5.x86_64 > ocfs2-2.6.18-128.el5-1.4.4-1.el5.x86_64 > ocfs2-tools-debuginfo-1.4.4-1.el5.x86_64 > Redhat Linux kernel version: 2.6.18-128.el5 > A collegue of mine stated that he usually disables all ocfs2 from the startup scripts, comments out the filesystems in /etc/fstab & reboots, is there another option to get the heartbeat offline or is this an ocfs2 or Linux bug I am encountering here?? > Cheers, > > *Christopher Shave*, Global UNIX/Linux Projects Team > *Marsh & McLennan Companies* > Global Technology Infrastructure (MGTI) | Centralised Operations > 555 Lonsdale Street, Level 5, Melbourne, VIC 3000, Australia > +61 3 9623 5488 | Mobile +61 0402 885 057 | _chris.shave at mercer.com_ <mailto:chris.shave at mercer.com> > _www.mmc.com_ <http://www.mmc.com/> >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://oss.oracle.com/pipermail/ocfs2-users/attachments/20110627/d558ba6a/attachment.html