Lustre discuss - Jul 2008 - Correct way to unmount a lustre client?

Hello,

what is the correct way to unmount a lustre client
when a simple "umount <filesystem>" responds with
"device is busy"? A "umount -l <filesystem>" does
unmount the filesystem however when I reboot the
machine subsequently the shutdown process hangs.
The last messages on console are:

LustreError: 131-3: Received notification of device removal
Please shutdown LNET to allow this to proceed

Best regards,
Hans Schnitzer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5750 bytes
Desc: S/MIME Cryptographic Signature
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080708/1ac74ab9/attachment.bin

Hello Hans-Juergen,

I usually try a combination of searching the process table for any running
tasks that are blocking the umount request and killing them, then doing a
''umount -k'', and then using ''lctl modules |awk
''{print $2}'' |xargs rmmod -v''
to deactivate the kernel modules. This last step sometimes leads to
complaints that the network module is busy, so you have to tell it to shut
down LNET in order to completely unload the remaining modules.

However, I don''t find this is a problem very frequently with my nodes
...
maybe you''ve got an uncooperative application that needs to be looked
at?

cheers,
Klaus

On 7/8/08 9:30 AM, "Hans-Juergen Schnitzer" <schnitzer at
rz.RWTH-Aachen.DE>did
etch on stone tablets:
> 
> Hello,
> 
> what is the correct way to unmount a lustre client
> when a simple "umount <filesystem>" responds with
> "device is busy"? A "umount -l <filesystem>" does
> unmount the filesystem however when I reboot the
> machine subsequently the shutdown process hangs.
> The last messages on console are:
> 
> LustreError: 131-3: Received notification of device removal
> Please shutdown LNET to allow this to proceed
> 
> Best regards,
> Hans Schnitzer
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

On Jul 08, 2008  18:30 +0200, Hans-Juergen Schnitzer
wrote:
> what is the correct way to unmount a lustre client
> when a simple "umount <filesystem>" responds with
> "device is busy"? A "umount -l <filesystem>" does
> unmount the filesystem however when I reboot the
> machine subsequently the shutdown process hangs.
The "umount -l" only removes the filesystem from the namespace, making
it unreachable.  It doesn''t really "unmount" the filesystem
at all.
It is likely that your "filesystem busy" is caused by processes using
Lustre for CWD or having files open there.  Use "lsof | grep
/mount/point"
to see what processes are using the filesystem.


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

lconf  --cleanup --node node_name(client''s node) config_file_name.xml
will do the needful.

To unmount the whole luster , the correct sequence would be to cleanup
client, mds and than ost.

Just check with  ''ps -ax'' after this to ensure

Dhruv

On Jul 9, 1:21 am, Andreas Dilger <adil... at sun.com>
wrote:
> On Jul 08, 2008  18:30 +0200, Hans-Juergen Schnitzer wrote:
>
> > what is the correct way to unmount a lustre client
> > when a simple "umount <filesystem>" responds with
> > "device is busy"? A "umount -l <filesystem>"
does
> > unmount the filesystem however when I reboot the
> > machine subsequently the shutdown process hangs.
>
> The "umount -l" only removes the filesystem from the namespace,
making
> it unreachable.  It doesn''t really "unmount" the
filesystem at all.
> It is likely that your "filesystem busy" is caused by processes
using
> Lustre for CWD or having files open there.  Use "lsof | grep
/mount/point"
> to see what processes are using the filesystem.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss

Andreas Dilger wrote:
> On Jul 08, 2008  18:30 +0200, Hans-Juergen Schnitzer wrote:
>> what is the correct way to unmount a lustre client
>> when a simple "umount <filesystem>" responds with
>> "device is busy"? A "umount -l <filesystem>"
does
>> unmount the filesystem however when I reboot the
>> machine subsequently the shutdown process hangs.
> 
> The "umount -l" only removes the filesystem from the namespace,
making
> it unreachable.  It doesn''t really "unmount" the
filesystem at all.
> It is likely that your "filesystem busy" is caused by processes
using
> Lustre for CWD or having files open there.  Use "lsof | grep
/mount/point"
> to see what processes are using the filesystem.
> 
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
> 
I sometimes get a "device is busy" even if lsof or fuser
doesn''t
report any processes using the filesystem. What else can I do to
force an unmount? Btw, I am using 1.6.5.

Best regards,
Hans Schnitzer

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 5750 bytes
Desc: S/MIME Cryptographic Signature
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080710/7d04aaeb/attachment.bin