Lustre devel - Dec 2006 - [Bug 11270] eliminate client locks in face of contention

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11270



Created an attachment (id=9166)
Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
 --> (https://bugzilla.lustre.org/attachment.cgi?id=9166&action=view)
a draft patch

a draft patch for b1_5 implementing the lockless i/o, it is compilable and it 
passes simple copy/write tests as well as fsx -RW. 

these tunables need to be altered to trigger lockless i/o:

for x in /proc/fs/lustre/llite/* ; do
echo 4096 >>  $x/min_nolock_size
echo 40960 >> $x/max_nolock_size
done
echo 40960 >> /proc/fs/lustre/ost/OSS/max_nolock_size

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11270



Wow, nice to get this.  Shouldn''t this become the default setting?  I
am
surprised that the server doesn''t enforce this, it seems you are
setting client
options?

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11270



(In reply to comment #19)
> Wow, nice to get this.  Shouldn''t this become the default setting?
too small to be default i think. anyway no performance testing is done yet.
>  I am
> surprised that the server doesn''t enforce this, it seems you are
setting
> client options?
client makes its own decisions based on the request size.  according the HLD 
client doesn''t talk with the server if the request size is smaller than
the
min_nolock_size.

more precisely client behaves differently for small, medium and large 
requests:

1. small requests. request size < min_nolock_size

client _always_ goes lockless i/o, the server is not asked.

2. medium size requests. min_nolock_size <= rq. size <= max_nolock_size

client asks the server whether lockless i/o is used and caches a positive 
answer for contention_time seconds -- subsequent medium size requests will use 
lockless i/o. 

3. large requests. rq.size > max_nolock_size

always ask the server, do not cache the answer.

Please don''t reply to lustre-devel. Instead, comment in Bugzilla by
using the following link:
https://bugzilla.lustre.org/show_bug.cgi?id=11270



(In reply to comment #21)
> - The io size heuristics are not acceptable.  For example, precisely with 
small
> IO the benefits of a cache can be enormous - this is just a shot in the 
dark,

well, I have 2 questions:

1. Does it mean that only server makes decision about 
whether client or server should acquire the locks?
Except the i/o size heuristics the client code caches server 
answer about using server locks.

2. the algorithm uses a simple measure of lock contention.  it is just
a count of conflicting locks in the "granted" and "waiting"
queues.  I think
it would not work well in a case when number of concurrent processes accessing
one file is low, but lock revocation rate is high.

Having something like "average lock revocation rate" counter would
solve
this problem. The function (prototype) below calculates an average value of 
lock revocation rate in a similar way Linux calculates load averate counters:

#define CONT_CALC_SHIFT 4UL
static unsigned ldlm_calc_contention(struct ldlm_lock *lock, int 
num_ast_to_send)
{
        struct ldlm_resource *res = lock->l_resource; 
        unsigned long now = jiffies;
        unsigned long t = now - es->lr_contention_time;
        unsigned interval = 
lock->l_export->exp_obd->u.ost.ost_contention_interval;
        unsigned long val = 0;

        if (interval == 0)
                return 0;

        if (t > interval)
                t = interval;
        else
                val = res->lr_contention_value;
        val = ((val * (interval - t) + (num_ast_to_send * t << 
CONT_CALC_SHIFT))
               / interval) >> CONT_CALC_SHIFT;
        res->lr_contention_time = now;
        res->lr_contention_value = val;
        return val >> CONT_CALC_SHIFT;
}
> and I''d prefer to haven nothing than something ill conceived.
> 
> - Locks for truncate must not be given out either
it is done now the same way as in liblustre -- ll_truncate uses 
OBD_FL_TRUNCLOCK if the server supports it.
> 
> - An application where all clients executed the following on one file: 
{open;
> truncate; write in page one; close } was the killer for liblustre with 
locks. 
> Please prove that this application will run without lock bouncing.