freebsd stable - Aug 2013 - Suggest changing dirhash defaults for FreeBSD 9.2.

I have been experimenting with dirhash settings, and have scoured the internet 
for other peoples' experience with it.
(I found the performance improvement in compiling has forestalled the need to 
add an SSD drive. ;-)

I believe that increasing the following values by 10 would benefit most FreeBSD 
users without disadvantage.

vfs.ufs.dirhash_maxmem: 2097152 to 20971520

vfs.ufs.dirhash_reclaimage: 5 to 50 or 60

FreeBSD administrators who have adjusted vfs.ufs.dirhash_maxmem for file server 
use have found
8 megs to be about minimum to satisfy vfs.ufs.dirhash_mem usage,
while I have found compiling larger packages such as Firefox brings 
vfs.ufs.dirhash_mem up to about 13 megs.

Setting vfs.ufs.dirhash_maxmem much larger than 20 megs results in 
vfs.ufs.dirhash_lowmemcount events
scavenging excessive amounts of the directory cache, at 10% per event.

I believe vfs.ufs.dirhash_reclaimage of about a minute is sufficient to reclaim 
memory without losing active directory cache,
particularly when compiling; however my experience in operations leads me to 
believe this is also an optimal default for file servers.

As the worst case scenario is that the directory cache will be over cleansed and
the memory returned to the kernel,
I would like to suggest that these defaults be raised by a factor of 10 for 
FreeBSD 9.2 or 10.0.

I've also toyed with dirhash on a few servers and received favorable
results. I've no idea where the defaults currently come from, but I'm
guessing probably around 1999 ;-)

On 27 August 2013 16:41, Robert Burmeister
<Robert.Burmeister at utoledo.edu> wrote:
>
> I have been experimenting with dirhash settings, and have scoured the
internet for other peoples' experience with it.
> (I found the performance improvement in compiling has forestalled the need
to add an SSD drive. ;-)
>
> I believe that increasing the following values by 10 would benefit most
FreeBSD users without disadvantage.
>
> vfs.ufs.dirhash_maxmem: 2097152 to 20971520
>
> vfs.ufs.dirhash_reclaimage: 5 to 50 or 60
vfs.ufs.dirhash_maxmem is further autotuned based on available physical memory.
See r214359 for details.

-- 
wbr,
pluknet

For previous benchmarks on the effect of the dirhash cache see:

https://wiki.freebsd.org/DirhashDynamicMemory




--
View this message in context:
http://freebsd.1045724.n5.nabble.com/Suggest-changing-dirhash-defaults-for-FreeBSD-9-2-tp5839351p5839775.html
Sent from the freebsd-stable mailing list archive at Nabble.com.

Here is a more recent dialog between the developers.

<quote
Nick Barkas
http://markmail.org/message/3sufphda2exjmhnq#query:+page:1+mid:3sufphda2exjmhnq+state:results

On Wed, May 27, 2009 at 12:36:49PM +0200, Nick Barkas wrote:

Some time during the next week or so, I plan on committing the attached
patch. It adds a vm_lowmem event handler to the dirhash code in UFS2 so that
dirhashes will be deleted when the system is low on memory. This allows one
to increase the maximum amount of memory available for dirhash on machines
that have memory to spare (via the vfs.ufs.dirhash_maxmem sysctl), and
hopefully just improving behaviour in low memory situations. I worked on
this last year for the summer of code with David Malone as my mentor.

cool! do you have any performance numbers? graphs? :) what value do you
recommend for the dirhash_maxmem sysctl?

Oh yes, I have many graphs: http://wiki.freebsd.org/DirhashDynamicMemory
When I ran those tests a few months ago, I used 64MB for dirhash_maxmem on a
system with 1GB of memory. I have not tried other amounts of memory besides
that, at least that I can recall, so please let me know what you find if you
experiment with other values. Performance improvements and sometimes
degradations changed depending on the type of work load, and the results on
7.x were also sometimes quite different from -current. I feel that the tests
I did were pretty artificial though, so it would be great to hear about any
results found with more realistic testing.

Nick 
&lt;/quote>

<quote
David Malone
http://markmail.org/message/3sufphda2exjmhnq#query:+page:1+mid:aerievrnrmkezehk+state:results

On Wed, May 27, 2009 at 09:57:31AM -0700, Julian Elischer wrote:

I was initially impressed by the numbers until I saw the scales.. a
difference between 475.5 and 474 is not that significant, but if your graph
scale is from 473 to 477, it looks at first glace very impressive.

I think we felt the real gains here would be the ability to set the dirhash
memory limits to a much larger value without having to worry about it
chewing all your memory. The results basically confirm that we haven't
introduced any serious regressions ;-)

David. 
&lt;/quote>



--
View this message in context:
http://freebsd.1045724.n5.nabble.com/Suggest-changing-dirhash-defaults-for-FreeBSD-9-2-tp5839351p5840090.html
Sent from the freebsd-stable mailing list archive at Nabble.com.

After scouring the internet, 
it seems that no one else has done a great deal of testing of UFS2 dirhash
defaults lately.

As the dirhash feature has effectively been tested for regressions,
I would like to propose setting the default dirhash values to my original
recommendation:

vfs.ufs.dirhash_maxmem: 20971520

vfs.ufs.dirhash_reclaimage: 60

The last benchmarks from 2008 only tested dirhash_maxmem at 2 and 64 megs.

I found in my testing that setting maxmem much over 32 megs makes the UFS
dirhash dysfunctional,
as scavenging 10% of maxmem on each vm_lowmem event becomes too aggressive.

While autotuning defaults is highly desirable, autotuning dirhash_maxmem
opens the can of worms of also tuning the hard coded 10% scavenging.

It may be better to hard code the maxmem default.


It may be appropriate to set the reclaimage to 5 minutes rather than 5
seconds;
I consider 60 seconds to still be a very conservative value, but at least an
effective one.




--
View this message in context:
http://freebsd.1045724.n5.nabble.com/Suggest-changing-dirhash-defaults-for-FreeBSD-9-2-tp5839351p5840115.html
Sent from the freebsd-stable mailing list archive at Nabble.com.

Presentation describing the logic behind adding dynamic memory allocation to
UFS dirhash
can be found at:

"EuroBSDCon 2008 - Nick Barkas - Dynamic memory allocation for dirhash in
UFS2"
http://www.za.freebsd.org/multimedia/tag-nick_barkas.html




--
View this message in context:
http://freebsd.1045724.n5.nabble.com/Suggest-changing-dirhash-defaults-for-FreeBSD-9-2-tp5839351p5851625.html
Sent from the freebsd-stable mailing list archive at Nabble.com.