Linux Virtualization - Apr 2014 - [PATCH v9 04/19] qspinlock: Extract out the exchange of tail code word

On Thu, Apr 17, 2014 at 11:03:56AM -0400, Waiman Long
wrote:
> @@ -192,36 +220,25 @@ void queue_spin_lock_slowpath(struct qspinlock *lock,
u32 val)
>  	node->next = NULL;
>  
>  	/*
> +	 * We touched a (possibly) cold cacheline; attempt the trylock once
> +	 * more in the hope someone let go while we weren't watching as long
> +	 * as no one was queuing.
>  	 */
> +	if (!(val & _Q_TAIL_MASK) && queue_spin_trylock(lock))
> +		goto release;
But you just did a potentially very expensive op; @val isn't
representative anymore!

On 04/17/2014 11:49 AM, Peter Zijlstra wrote:
> On Thu, Apr 17, 2014 at 11:03:56AM -0400, Waiman Long wrote:
>> @@ -192,36 +220,25 @@ void queue_spin_lock_slowpath(struct qspinlock
*lock, u32 val)
>>   	node->next = NULL;
>>
>>   	/*
>> +	 * We touched a (possibly) cold cacheline; attempt the trylock once
>> +	 * more in the hope someone let go while we weren't watching as
long
>> +	 * as no one was queuing.
>>   	 */
>> +	if (!(val&  _Q_TAIL_MASK)&&  queue_spin_trylock(lock))
>> +		goto release;
> But you just did a potentially very expensive op; @val isn't
> representative anymore!
That is not true. I pass in a pointer to val to trylock_pending() (the 
pointer thing) so that it will store the latest value that it reads from 
the lock back into val. I did miss one in the PV qspinlock exit loop. I 
will add it back when I do the next version.

-Longman

On Thu, Apr 17, 2014 at 05:28:17PM -0400, Waiman Long
wrote:
> On 04/17/2014 11:49 AM, Peter Zijlstra wrote:
> >On Thu, Apr 17, 2014 at 11:03:56AM -0400, Waiman Long wrote:
> >>@@ -192,36 +220,25 @@ void queue_spin_lock_slowpath(struct
qspinlock *lock, u32 val)
> >>  	node->next = NULL;
> >>
> >>  	/*
> >>+	 * We touched a (possibly) cold cacheline; attempt the trylock
once
> >>+	 * more in the hope someone let go while we weren't watching
as long
> >>+	 * as no one was queuing.
> >>  	 */
> >>+	if (!(val&  _Q_TAIL_MASK)&& 
queue_spin_trylock(lock))
> >>+		goto release;
> >But you just did a potentially very expensive op; @val isn't
> >representative anymore!
> 
> That is not true. I pass in a pointer to val to trylock_pending() (the
> pointer thing) so that it will store the latest value that it reads from
the
> lock back into val. I did miss one in the PV qspinlock exit loop. I will
add
> it back when I do the next version.
But you did that read _before_ you touched a cold cacheline, that's 100s
of cycles. Whatever value you read back then is now complete nonsense.