Lustre devel - Dec 2006 - [Bug 11289] Slow read performance for Lustre on XT3 service nodes

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11289



Created an attachment (id=9152)
Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
 --> (https://bugzilla.lustre.org/attachment.cgi?id=9152&action=view)
Benchmark example code

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11289



Created an attachment (id=9153)
Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
 --> (https://bugzilla.lustre.org/attachment.cgi?id=9153&action=view)
Readahead stats before running benchmark

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11289



Created an attachment (id=9154)
Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
 --> (https://bugzilla.lustre.org/attachment.cgi?id=9154&action=view)
Readahead stats after running benchmark

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11289



I''m going to post several files...

mybench.c is the benchmark I''m using.  I run it with -w on nodeA then
run it
with -r on nodeB.

I will also post before and after readahead stats.

Here is the output from the run for a 104MB file...
canon@rizzo2:~/benchmarks/sio> ./mybench /lustre/scratch/canon/test/testfile
-r
Ct   time(ms) Bytes  Rate(MB/s)
  0      116    8388608    69
  1      725    8388608    11
  2     1529    8388608     5
  3     2369    8388608     3
  4     3231    8388608     2
  5       21    8388608   381
  6       16    8388608   500
  7       17    8388608   471
  8       11    8388608   727
  9       11    8388608   727
 10        9    8388608   889
 11       26    8388608   308
 12       33    8388608   242
 13        1          0     0
Total: 104 MB      13 MB/s


As you can see in cts 2-4, things get very slow.  After that it picks up and
runs pretty well.  The only noticeable thing in the stats is that the failed
grab_cache_page seems to be incrementing quite a bit.