zfs discuss - Apr 2008 - ls -lt for links slower than for regular files

Hi...

System Config:
                 2 Intel 3 Ghz 5160 dual-core cpu''s
                 10 SATA 750 GB disks running as a ZFS RAIDZ2 pool
                 8 GB Memory
                 SunOS 5.11 snv_79a on a separate UFS mirror
                 ~150 Read I/O''s/second, ~300 Write
I/O''s/second
                    on the ZFS pool when busy
                 ARC size ~2 GB
                 No separate ARC or ZIL cache

I have a couple of large directories, ~58,000 files, where one contains all 
regular files while the other contains all links pointing back to the regular 
files.  On a busy ZFS filesystem, when I do an "ls -lat" on the
regular file
directory, it returns within a few minutes or less, whereas when I do the same 
thing on the directory of links, it can take from 15 minutes to over an hour. 
When I stop our data collection application, the regular files then list within 
a few seconds and the links list in, perhaps, 60 seconds.  Is there a 
difference in what ls has to do when listing links versus listing regular files 
in ZFS that would cause a slowdown?

                            Thanks...

                              Art

Arthur A. Person
Research Assistant, System Administrator
Penn State Department of Meteorology
email:  person at meteo.psu.edu, phone:  814-863-1563

On Tue, 8 Apr 2008, ap60 at meteo.psu.edu wrote:
> a few seconds and the links list in, perhaps, 60 seconds.  Is there a
> difference in what ls has to do when listing links versus listing regular
files
> in ZFS that would cause a slowdown?
Since you specified ''-t'' the links have to be
"dereferenced" (find the
file that is referred to) which results in opening the directory to 
see if the file exists, and what its properties are.  With 50K+ files, 
opening the directory and finding the file will take tangible time. 
If there are multiple directories in the symbolic link path, then 
these directories need to be opened as well.  Symbolic links are not 
free.

More RAM may help if it results in keeping the directory data hot in 
the cache.

If the links were hard links rather than symbolic links, then 
performance will be similar to a regular file (since it is then a 
regular file).

Bob
=====================================Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/