I am having trouble destroying a zfs file system (device busy) and fuser isn''t telling me who has the file open: # zfs destroy files/custfs/cust12/2053699a cannot unmount ''/files/custfs/cust12/2053699a'': Device busy # zfs unmount files/custfs/cust12/2053699a cannot unmount ''/files/custfs/cust12/2053699a'': Device busy # fuser -c /files/custfs/cust12/2053699a /files/custfs/cust12/2053699a: # (cd /files/custfs/cust12/2053699a && fuser -c .) .: 10383c [fuser itself or shell] Using ''zfs unmount -f'' will unmount the file system, but I''d like to know why that was necessary. This situation appears to occur every night during a system test. The only peculiar operation on the errant file system is that another system NFS mounts it with vers=2 in a non-global zone, and then halts that zone. I haven''t been able to reproduce the problem outside the test. Both systems are running S10U4. Any clues? Fred
Fred.Oliver at Sun.COM said:> I am having trouble destroying a zfs file system (device busy) and fuser > isn''t telling me who has the file open: > . . . > This situation appears to occur every night during a system test. The only > peculiar operation on the errant file system is that another system NFS > mounts it with vers=2 in a non-global zone, and then halts that zone. I > haven''t been able to reproduce the problem outside the test.If you have a filesystem shared-out (exported) on an NFS-server, you''ll get this kind of behavior. No client need have it mounted. You must first do a "unshare /files/custfs/cust12/2053699a" in your example before trying to unmount it. Regards, Marion
Marion Hakanson wrote:> Fred.Oliver at Sun.COM said: >> I am having trouble destroying a zfs file system (device busy) and fuser >> isn''t telling me who has the file open: >> . . . >> This situation appears to occur every night during a system test. The only >> peculiar operation on the errant file system is that another system NFS >> mounts it with vers=2 in a non-global zone, and then halts that zone. I >> haven''t been able to reproduce the problem outside the test. > > If you have a filesystem shared-out (exported) on an NFS-server, you''ll get > this kind of behavior. No client need have it mounted. You must first do > a "unshare /files/custfs/cust12/2053699a" in your example before trying > to unmount it.Thanks, Marion. (a) This doesn''t help in this case. ?? # unshare /files/custfs/cust12/2053699a nfs unshare: /files/custfs/cust12/2053699a: not shared # zfs umount files/custfs/cust12/2053699a cannot unmount ''/files/custfs/cust12/2053699a'': Device busy (b) The unshare clearly isn''t necessary in the vast majority of other cases. I mean that the pattern of zfs_create, mount in zone, zone shutdown, zfs destroy works almost all of the time. What would be different in this case? --- Separately: -- lsof returns no references to this file system -- this file system is a clone, not that it should matter. -- zfs properties are attached. Fred -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: props URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080328/305b3633/attachment.ksh>
Fred Oliver wrote:> I am having trouble destroying a zfs file system (device busy) and fuser > isn''t telling me who has the file open: > > # zfs destroy files/custfs/cust12/2053699a > cannot unmount ''/files/custfs/cust12/2053699a'': Device busy > > # zfs unmount files/custfs/cust12/2053699a > cannot unmount ''/files/custfs/cust12/2053699a'': Device busy > > # fuser -c /files/custfs/cust12/2053699a > /files/custfs/cust12/2053699a: > > # (cd /files/custfs/cust12/2053699a && fuser -c .) > .: 10383c [fuser itself or shell] >shell? Is your CWD on that file system? Usually when I do this it is because I was somewhere and su''ed which leaves a shell with the CWD where I su''ed. -- richard
Richard Elling wrote:> Fred Oliver wrote: >> I am having trouble destroying a zfs file system (device busy) and >> fuser isn''t telling me who has the file open: >> >> # zfs destroy files/custfs/cust12/2053699a >> cannot unmount ''/files/custfs/cust12/2053699a'': Device busy >> >> # zfs unmount files/custfs/cust12/2053699a >> cannot unmount ''/files/custfs/cust12/2053699a'': Device busy >> >> # fuser -c /files/custfs/cust12/2053699a >> /files/custfs/cust12/2053699a: >> >> # (cd /files/custfs/cust12/2053699a && fuser -c .) >> .: 10383c [fuser itself or shell] >> > > shell? Is your CWD on that file system? Usually when I do this > it is because I was somewhere and su''ed which leaves a shell > with the CWD where I su''ed. > -- richardI just meant that in this example [cd foo && fuser], either the subshell exec''ing the fuser command or the fuser process itself is process 10383, and neither one lasts long enough to easily identify which. Fred
Fred Oliver wrote:> I am having trouble destroying a zfs file system (device busy) and fuser > isn''t telling me who has the file open: > > # zfs destroy files/custfs/cust12/2053699a > cannot unmount ''/files/custfs/cust12/2053699a'': Device busy > > # zfs unmount files/custfs/cust12/2053699a > cannot unmount ''/files/custfs/cust12/2053699a'': Device busy > > # fuser -c /files/custfs/cust12/2053699a > /files/custfs/cust12/2053699a: > >You can get this behavior (can''t unmount/fuser shows nothing) when the FS is still shared via NFS (and now CIFS presumably.) I always have to remember to run ''share'' to see what''s shared when I hit this. Though now with sharing being a ZFS property, I''ll probably have to check a different way in the future.. After I unshare it, the umount always works fine for me. -Kyle> # (cd /files/custfs/cust12/2053699a && fuser -c .) > .: 10383c [fuser itself or shell] > > Using ''zfs unmount -f'' will unmount the file system, but I''d like to > know why that was necessary. > > This situation appears to occur every night during a system test. The > only peculiar operation on the errant file system is that another system > NFS mounts it with vers=2 in a non-global zone, and then halts that > zone. I haven''t been able to reproduce the problem outside the test. > > Both systems are running S10U4. > > Any clues? > > Fred > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Fred Oliver wrote:> > Marion Hakanson wrote: >> Fred.Oliver at Sun.COM said: >>> I am having trouble destroying a zfs file system (device busy) and >>> fuser >>> isn''t telling me who has the file open: . . . >>> This situation appears to occur every night during a system test. >>> The only >>> peculiar operation on the errant file system is that another system >>> NFS >>> mounts it with vers=2 in a non-global zone, and then halts that >>> zone. I >>> haven''t been able to reproduce the problem outside the test. >> >> If you have a filesystem shared-out (exported) on an NFS-server, >> you''ll get >> this kind of behavior. No client need have it mounted. You must >> first do >> a "unshare /files/custfs/cust12/2053699a" in your example before trying >> to unmount it. > > Thanks, Marion. > > (a) This doesn''t help in this case. ?? > > # unshare /files/custfs/cust12/2053699a > nfs unshare: /files/custfs/cust12/2053699a: not shared > # zfs umount files/custfs/cust12/2053699a > cannot unmount ''/files/custfs/cust12/2053699a'': Device busy >In this case you may need to use ''zfs unshare'', since I don''t know if ''unshare'' can unshare a ZFS that was sared by ''zfs share''.> (b) The unshare clearly isn''t necessary in the vast majority of other > cases. I mean that the pattern of zfs_create, mount in zone, zone > shutdown, zfs destroy works almost all of the time. What would be > different in this case? >I don''t know why thar case would work some times. In my experience with UFS, having the fs shared out, *always* kept it busy when trying to unmount it. -Kyle> --- > > Separately: > -- lsof returns no references to this file system > -- this file system is a clone, not that it should matter. > -- zfs properties are attached. > > Fred > > > > ------------------------------------------------------------------------ > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Kyle McDonald wrote:> In this case you may need to use ''zfs unshare'', since I don''t know if > ''unshare'' can unshare a ZFS that was sared by ''zfs share''.This makes no difference: # zfs unshare files/custfs/cust12/2053699a cannot unshare ''files/custfs/cust12/2053699a'': not currently shared # zfs umount files/custfs/cust12/2053699a cannot unmount ''/files/custfs/cust12/2053699a'': Device busy I''m still looking for suggestions. Fred
Le 30 mars 08 ? 15:57, Kyle McDonald a ?crit :> Fred Oliver wrote: >> >> Marion Hakanson wrote: >>> Fred.Oliver at Sun.COM said: >>>> I am having trouble destroying a zfs file system (device busy) and >>>> fuser >>>> isn''t telling me who has the file open: . . . >>>> This situation appears to occur every night during a system test. >>>> The only >>>> peculiar operation on the errant file system is that another system >>>> NFS >>>> mounts it with vers=2 in a non-global zone, and then halts that >>>> zone. I >>>> haven''t been able to reproduce the problem outside the test. >>> >>> If you have a filesystem shared-out (exported) on an NFS-server, >>> you''ll get >>> this kind of behavior. No client need have it mounted. You must >>> first do >>> a "unshare /files/custfs/cust12/2053699a" in your example before >>> trying >>> to unmount it. >> >> Thanks, Marion. >> >> (a) This doesn''t help in this case. ?? >> >> # unshare /files/custfs/cust12/2053699a >> nfs unshare: /files/custfs/cust12/2053699a: not shared >> # zfs umount files/custfs/cust12/2053699a >> cannot unmount ''/files/custfs/cust12/2053699a'': Device busy >> > In this case you may need to use ''zfs unshare'', since I don''t know if > ''unshare'' can unshare a ZFS that was sared by ''zfs share''. >> (b) The unshare clearly isn''t necessary in the vast majority of other >> cases. I mean that the pattern of zfs_create, mount in zone, zone >> shutdown, zfs destroy works almost all of the time. What would be >> different in this case? >> > I don''t know why thar case would work some times. In my experience > with > UFS, having the fs shared out, *always* kept it busy when trying to > unmount it. >I think the zfs umount will first issue the unshare under the cover. I''ve manage to find processes holding a reference by looking at all environments and grepring for the mount point. It''s not a catch all though and I don''t know why. -r> -Kyle > >> --- >> >> Separately: >> -- lsof returns no references to this file system >> -- this file system is a clone, not that it should matter. >> -- zfs properties are attached. >> >> Fred >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss