Nathan Kroenert
2008-Mar-03 00:34 UTC
[zfs-discuss] ls -l held up when removing large files?
Hey there - whilst trying to re-create some other performance issues, I came across an ls -l that hung waiting for an rm to complete. The file was a 176GB file, and I was rm''ing it to clear some space for more tests. At the time I was rming it, I was attempting to do an ls -l and found it hung, waiting for the rm to complete. The stack looked like: > 0t1507::pid2proc |::print proc_t p_tlist |::findstack stack pointer for thread 300022d09a0: 2a101a84991 [ 000002a101a84991 cv_wait+0x38() ] 000002a101a84a41 zfs_dirent_lock+0x118() 000002a101a84b01 zfs_dirlook+0x170() 000002a101a84bd1 zfs_lookup+0x194() 000002a101a84c91 fop_lookup+0x28() 000002a101a84d51 lookuppnvp+0x344() 000002a101a84f91 lookuppnat+0x120() 000002a101a85051 lookupnameat+0x5c() 000002a101a85161 cstatat_getvp+0x198() 000002a101a85221 cstatat64_32+0x40() 000002a101a852e1 syscall_trap32+0xcc() So, we were waiting to grab a lock... I tried a few variations - different pool, different filesystem, same pool different directory etc and found the hang was only observed in the same directory as the file being rm''d. Is this expected? As it takes some minutes to remove the file, I could see this getting in the way of some application behavior, even with smaller(ish) files, and even sequentializing some operations. Any thoughts on a way to make this rm''ing of large files less pesky (or impactful - Just for you, James McP...) for other users of that directory? Does the rm operation need to hold that lock for all that time? Is there a better way? Oh - a ''find .'' from the root of that filesystem will also hang waiting for the lock. I can create new files and rm other files though, which is good. I wonder what else might be potentially impacted... Cheers! Nathan.