Hi, On an idle server, when I do a recursive ''/usr/bin/ls'' on a folder, I see a lot of disk activity. This makes sense because the results (metadata/data) may not have been cached. When I do a second "ls" on the same folder right after the first one finished, I do see disk activity again. Can someone explain why the results are not cached in ARC? I am running this on a thumper, b118 and default zpool settings. The box has 64GB of RAM and arc size is around 24GB. No ARC tuning. -- This message posted from opensolaris.org
Hello John, On Oct 30, 2009, at 9:03 PM, John wrote:> Hi, > > On an idle server, when I do a recursive ''/usr/bin/ls'' on a folder, > I see a lot of disk activity. This makes sense because the results > (metadata/data) may not have been cached. > When I do a second "ls" on the same folder right after the first one > finished, I do see disk activity again. > > Can someone explain why the results are not cached in ARC?You would have disk access again unless you have turned set atime to off for that filesystem. I did posted something similar a few days back and write a summary of the ARC-part of my findings: http://sparcv9.blogspot.com/2009/10/curious-case-of-strange-arc.html Here is the whole thread: http://opensolaris.org/jive/thread.jspa?messageID=430385 If that does not explain it you should probably provide some more data, how many files, some ARC statistics etc. Regards Henrik Henrik http://sparcv9.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20091030/6e207526/attachment.html>
> <div id="jive-html-wrapper-div"> > Hello John,<div><br><div><div><div>On Oct 30, 2009, > at 9:03 PM, John wrote:</div><br > class="Apple-interchange-newline"><blockquote > type="cite"><div>Hi,<br><br>On an idle server, when I > do a recursive ''/usr/bin/ls'' on a folder, I see a lot > of disk activity. This makes sense because the > results (metadata/data) may not have been > cached.<br>When I do a second "ls" on the same folder > right after the first one finished, I do see disk > activity again.<br><br>Can someone explain why the > results are not cached in > ARC?<br></div></blockquote><br></div><div>You would > have disk access again unless you have turned set > atime to off for that filesystem. I did posted > something similar a few days back and write a summary > of the ARC-part of my findings: <a > href="http://sparcv9.blogspot.com/2009/10/curious-case > -of-strange-arc.html">http://sparcv9.blogspot.com/2009 > /10/curious-case-of-strange-arc.html</a></div><div><br > ></div><div>Here is the whole thread: <a > href="http://opensolaris.org/jive/thread.jspa?messageI > D=430385">http://opensolaris.org/jive/thread.jspa?mess > ageID=430385</a></div><div><br></div><div>If that > does not explain it you should probably provide some > more data, how many files, some ARC statistics > etc.</div><div><br></div><div>Regards</div><div>Henrik > </div><br><div> > <span class="Apple-style-span" > style="border-collapse: separate; color: rgb(0, 0, > 0); font-family: Helvetica; font-size: medium; > font-style: normal; font-variant: normal; > font-weight: normal; letter-spacing: normal; > line-height: normal; orphans: 2; text-align: auto; > text-indent: 0px; text-transform: none; white-space: > normal; widows: 2; word-spacing: 0px; > -webkit-border-horizontal-spacing: 0px; > -webkit-border-vertical-spacing: 0px; > -webkit-text-decorations-in-effect: none; > -webkit-text-size-adjust: auto; > -webkit-text-stroke-width: 0px; "><div>Henrik<div><a > href="http://sparcv9.blogspot.com/">http://sparcv9.blo > gspot.com</a></div></div></span> > </div> > > <br></div></div> > </div>_______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ssthanks Henrik. This makes perfect sense. More questions. arc_meta_limit is set to a quarter of the ARC size. what is arc_meta_max ? On some systems, I have arc_meta_max > arc_meta_limit. Example: arc_meta_used = 29427 MB arc_meta_limit = 16125 MB arc_meta_max = 29427 MB Example 2: arc_meta_used = 5885 MB arc_meta_limit = 5885 MB arc_meta_max = 17443 MB -- This message posted from opensolaris.org
On Oct 30, 2009, at 10:20 PM, John wrote:> > thanks Henrik. This makes perfect sense. More questions. > arc_meta_limit is set to a quarter of the ARC size. > what is arc_meta_max ? > On some systems, I have arc_meta_max > arc_meta_limit. > > Example: > arc_meta_used = 29427 MB > arc_meta_limit = 16125 MB > arc_meta_max = 29427 MB > > Example 2: > arc_meta_used = 5885 MB > arc_meta_limit = 5885 MB > arc_meta_max = 17443 MB > --That looks very strange, the source says: if (arc_meta_max < arc_meta_used) arc_meta_max = arc_meta_used; So arc_meta_max should be the maximum amount of that arc_meta_used has ever reached. The limit on the metadata is not enforced synchronously, but that seems to be quite a bit over the limit. What are these machines doing, are they quickly processing large numbers files/directories? I do not know the exact implementation of this but perhaps new metadata is added to the cache faster than it gets purged. Maybe someone else knows more exactly how this works? Henrik http://sparcv9.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20091030/857c95c6/attachment.html>