Hello, In ZFS the DNLC concept is gone, or is in ARC too? I mean, all the cache in ZFS is ARC right? I was thinking if we can tune the DNLC in ZFS like in UFS.. if we have too *many* files and directories, i guess we can have a better performance having all the metadata cached, and that is even more important in NFS operations. DNLC is LRU right? And ARC should be totally dynamic, but as in another thread here, i think reading a *big* file can mess with the whole thing. Can we hold an area in memory for DNLC cache, or that is not the ARC way? thanks, Leal. -- This message posted from opensolaris.org
DNLC seems to be independent.>From my laptop, which has only got ZFS file systems (Two ZPOOLs), the statsare: $ kstat -n dnlcstats module: unix instance: 0 name: dnlcstats class: misc crtime 25.772681029 dir_add_abort 0 dir_add_max 0 dir_add_no_memory 0 dir_cached_current 0 dir_cached_total 0 dir_entries_cached_current 0 dir_fini_purge 0 dir_hits 0 dir_misses 0 dir_reclaim_any 0 dir_reclaim_last 0 dir_remove_entry_fail 0 dir_remove_space_fail 0 dir_start_no_memory 0 dir_update_fail 0 double_enters 256 * enters 29871 hits 5057854 <<--- Looks Good! misses 27737* negative_cache_hits 88995 pick_free 0 pick_heuristic 0 pick_last 0 purge_all 1 purge_fs1 0 purge_total_entries 22117 purge_vfs 79 purge_vp 74 snaptime 14043.559161769 On Thu, Oct 30, 2008 at 12:50 PM, Marcelo Leal < opensolaris at posix.brte.com.br> wrote:> Hello, > In ZFS the DNLC concept is gone, or is in ARC too? I mean, all the cache > in ZFS is ARC right? > I was thinking if we can tune the DNLC in ZFS like in UFS.. if we have too > *many* files and directories, i guess we can have a better performance > having all the metadata cached, and that is even more important in NFS > operations. > DNLC is LRU right? And ARC should be totally dynamic, but as in another > thread here, i think reading a *big* file can mess with the whole thing. Can > we hold an area in memory for DNLC cache, or that is not the ARC way? > > thanks, > > Leal. > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- Any sufficiently advanced technology is indistinguishable from magic. Arthur C. Clarke My blog: http://initialprogramload.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081030/f96f1f85/attachment.html>
Leal, ZFS uses the DNLC. It still provides the fastest lookup of <directory, name> to vnode. The DNLC is kind of LRU. An async process will use a rotor to move through the hash chains and select the LRU entry but will select first negative cache entries and vnodes only referenced by the DNLC. Underlying this ZFS uses the ZAP and Fat ZAP to store the mappings. ZFS does not use the 2nd level DNLC which allows caching of directories. This is only used by UFS to avoid a linear search of large directories. Neil. On 10/30/08 04:50, Marcelo Leal wrote:> Hello, > In ZFS the DNLC concept is gone, or is in ARC too? I mean, all the cache in ZFS is ARC right? > I was thinking if we can tune the DNLC in ZFS like in UFS.. if we have too *many* files and > directories, i guess we can have a better performance having all the metadata cached, and that > is even more important in NFS operations. > DNLC is LRU right? And ARC should be totally dynamic, but as in another thread here, > i think reading a *big* file can mess with the whole thing. Can we hold an area in memory > for DNLC cache, or that is not the ARC way? > > thanks, > > Leal.
Hello Neil,> Leal, > > ZFS uses the DNLC. It still provides the fastest > lookup of <directory, name> to vnode.Ok, so the whole concept remains true? We can tune the DNLC and expect the same behaviour on ZFS?> The DNLC is kind of LRU. An async process will use a > rotor to move > through the hash chains and select the LRU entry but > will select first > negative cache entries and vnodes only referenced by > the DNLC. > Underlying this ZFS uses the ZAP and Fat ZAP to store > the mappings.Here i did not understand very well. You are saying that ZFS uses DNLC just for one level?> > ZFS does not use the 2nd level DNLC which allows > caching of directories. > This is only used by UFS to avoid a linear search of > large directories.What is the ZFS way here? One of the points of my question is exactly that... in an environment with many directories with *many* files, i think ZFS would has the *same* problems too. So, having directories cache on DNLC could be a good solution. Can you explain how ZFS handles the performance in directories with hundreds of files? There is a lot of docs around UFS/DNLC, but for now i think the only doc about ZFS/ARC and DNLC is the source code. ;-)> > Neil.Thanks a lot! I was thinking in tune DNLC to have as many metadata (directories and files) as i can, to minimize lookups/stats and etc (in NFS there is a lot of getattr ops). So we could have *all* the metadata cached, and use what remains in memory to cache data. Maybe that kind of tuning would be usefull for just a few workloads, but could be a *huge* enhancement for that workloads. Leal -- posix rules -- [http://www.posix.brte.com.br/blog]> > On 10/30/08 04:50, Marcelo Leal wrote: > > Hello, > > In ZFS the DNLC concept is gone, or is in ARC too? > I mean, all the cache in ZFS is ARC right? > > I was thinking if we can tune the DNLC in ZFS like > in UFS.. if we have too *many* files and > > directories, i guess we can have a better > performance having all the metadata cached, and that > > is even more important in NFS operations. > > DNLC is LRU right? And ARC should be totally > dynamic, but as in another thread here, > > i think reading a *big* file can mess with the > whole thing. Can we hold an area in memory > > for DNLC cache, or that is not the ARC way? > > > > thanks, > > > > Leal. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ss-- This message posted from opensolaris.org
On 10/30/08 11:00, Marcelo Leal wrote:> Hello Neil, > >> Leal, >> >> ZFS uses the DNLC. It still provides the fastest >> lookup of <directory, name> to vnode. > > Ok, so the whole concept remains true? We can tune the DNLC and expect the same behaviour on ZFS?Yes.> >> The DNLC is kind of LRU. An async process will use a >> rotor to move >> through the hash chains and select the LRU entry but >> will select first >> negative cache entries and vnodes only referenced by >> the DNLC. >> Underlying this ZFS uses the ZAP and Fat ZAP to store >> the mappings. > > Here i did not understand very well. You are saying that ZFS uses DNLC just for one level?Yes, the DNLC also supports entire directory caches, however ZFS doesn''t use this as it''s better organised on disk not to be linear. Normally name lookups check the normal/original (1st level) DNLC then if that fails the entire directory name cache (2nd level) is checked.> >> ZFS does not use the 2nd level DNLC which allows >> caching of directories. >> This is only used by UFS to avoid a linear search of >> large directories. > > What is the ZFS way here? One of the points of my question is exactly that... in an environment with many directories with *many* files, i think ZFS would has the *same* problems too. > So, having directories cache on DNLC could be a good solution. Can you explain how ZFS handles the performance in directories with hundreds of files? > There is a lot of docs around UFS/DNLC, but for now i think the only doc about ZFS/ARC and DNLC is the source code. ;-) > >> Neil. > > Thanks a lot! > I was thinking in tune DNLC to have as many metadata (directories and files) as i can, to minimize lookups/stats and etc (in NFS there is a lot of getattr ops). So we could have *all* the metadata cached, and use what remains in memory to cache data. > Maybe that kind of tuning would be usefull for just a few workloads, but could be a *huge* enhancement for that workloads. > > Leal > -- posix rules -- > [http://www.posix.brte.com.br/blog] > >> On 10/30/08 04:50, Marcelo Leal wrote: >>> Hello, >>> In ZFS the DNLC concept is gone, or is in ARC too? >> I mean, all the cache in ZFS is ARC right? >>> I was thinking if we can tune the DNLC in ZFS like >> in UFS.. if we have too *many* files and >>> directories, i guess we can have a better >> performance having all the metadata cached, and that >>> is even more important in NFS operations. >>> DNLC is LRU right? And ARC should be totally >> dynamic, but as in another thread here, >>> i think reading a *big* file can mess with the >> whole thing. Can we hold an area in memory >>> for DNLC cache, or that is not the ARC way? >>> >>> thanks, >>> >>> Leal. >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discu >> ss