On Dec 16, 2005 11:48 -0800, Jeffrey W. Baker wrote:> Normally when I list a directory with ls, I get the results quickly, but > if I then use du in the same directory, this is slow, and the following > ls is also slow: > > $ time ls | wc > 415 415 4565 > ls 0.00s user 0.01s system 79% cpu 0.018 total > > $ time du -hs > 17G . > du -hs 0.00s user 0.18s system 2% cpu 6.459 total > > t$ time ls | wc > 415 415 4565 > ls 0.00s user 0.02s system 0% cpu 9.802 total > > Leaving aside the problem of needing 10 seconds to list 415 entries, > what causes the difference between the first and the last ls? Or the > slow du? Is there some kind of cache that becomes flushed? > > This is Lustre 1.4.5The client LRU size is 100 entries. Servers can generally handle at least 100k locks, probably a lot more (depends on RAM size). If you have "login" or interactive nodes increasing the LRU size of the MDC up to 10k and OSCs up to 1k can improve responsiveness dramatically. Do this with "echo $LRU_SIZE > /proc/fs/lustre/ldlm/namespaces/$NAME/lru_size" Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
On Fri, 2005-12-16 at 19:08 -0700, Andreas Dilger wrote:> On Dec 16, 2005 11:48 -0800, Jeffrey W. Baker wrote: > > Normally when I list a directory with ls, I get the results quickly, but > > if I then use du in the same directory, this is slow, and the following > > ls is also slow: > > > > $ time ls | wc > > 415 415 4565 > > ls 0.00s user 0.01s system 79% cpu 0.018 total > > > > $ time du -hs > > 17G . > > du -hs 0.00s user 0.18s system 2% cpu 6.459 total > > > > t$ time ls | wc > > 415 415 4565 > > ls 0.00s user 0.02s system 0% cpu 9.802 total > > > > Leaving aside the problem of needing 10 seconds to list 415 entries, > > what causes the difference between the first and the last ls? Or the > > slow du? Is there some kind of cache that becomes flushed? > > > > This is Lustre 1.4.5 > > The client LRU size is 100 entries. Servers can generally handle at > least 100k locks, probably a lot more (depends on RAM size). If you > have "login" or interactive nodes increasing the LRU size of the MDC > up to 10k and OSCs up to 1k can improve responsiveness dramatically. > > Do this with "echo $LRU_SIZE > /proc/fs/lustre/ldlm/namespaces/$NAME/lru_size"Thanks for the tip. Should this be done before Lustre is started, or can it be done safely at any time? -jwb
Jeffrey W. Baker writes: > On Fri, 2005-12-16 at 19:08 -0700, Andreas Dilger wrote: [...] > > > > Do this with "echo $LRU_SIZE > /proc/fs/lustre/ldlm/namespaces/$NAME/lru_size" > > Thanks for the tip. Should this be done before Lustre is started, or > can it be done safely at any time? This tunable can be changed at any time. By the way, /proc/fs/lustre files appear only after lustre is started. > > -jwb Nikita.
Normally when I list a directory with ls, I get the results quickly, but
if I then use du in the same directory, this is slow, and the following
ls is also slow:
$ time ls | wc
415 415 4565
ls 0.00s user 0.01s system 79% cpu 0.018 total
$ time du -hs
17G .
du -hs 0.00s user 0.18s system 2% cpu 6.459 total
t$ time ls | wc
415 415 4565
ls 0.00s user 0.02s system 0% cpu 9.802 total
Leaving aside the problem of needing 10 seconds to list 415 entries,
what causes the difference between the first and the last ls? Or the
slow du? Is there some kind of cache that becomes flushed?
This is Lustre 1.4.5
-jwb