search for: lockval

On 12/03/14 18:54, Waiman Long wrote: > This patch adds para-virtualization support to the queue spinlock in > the same way as was done in the PV ticket lock code. In essence, the > lock waiters will spin for a specified number of times (QSPIN_THRESHOLD > = 2^14) and then halted itself. The queue head waiter will spins > 2*QSPIN_THRESHOLD times before halting itself. When it has

On 12/03/14 18:54, Waiman Long wrote: > This patch adds para-virtualization support to the queue spinlock in > the same way as was done in the PV ticket lock code. In essence, the > lock waiters will spin for a specified number of times (QSPIN_THRESHOLD > = 2^14) and then halted itself. The queue head waiter will spins > 2*QSPIN_THRESHOLD times before halting itself. When it has

...); // pv_kick_node: if (pv->cpustate != PV_CPU_HALTED) return; ACCESS_ONCE(pv->cpustate) = PV_CPU_KICKED; __queue_kick_cpu(pv->mycpu, PV_KICK_QUEUE_HEAD); Waiter ------------------------------------------- // pv_head_spin_check ACCESS_ONCE(pv->cpustate) = PV_CPU_HALTED; lockval = cmpxchg(&qlock->lock, _QSPINLOCK_LOCKED, _QSPINLOCK_LOCKED_SLOWPATH); if (lockval == 0) { /* * Can exit now as the lock is free */ ACCESS_ONCE(pv->cpustate) = PV_CPU_ACTIVE; *count = 0; return; } __queue_hibernate(); Nothing protects from writing q...

...n; > ACCESS_ONCE(pv->cpustate) = PV_CPU_KICKED; > __queue_kick_cpu(pv->mycpu, PV_KICK_QUEUE_HEAD); > > Waiter ------------------------------------------- > > // pv_head_spin_check > ACCESS_ONCE(pv->cpustate) = PV_CPU_HALTED; > lockval = cmpxchg(&qlock->lock, > _QSPINLOCK_LOCKED, > _QSPINLOCK_LOCKED_SLOWPATH); > if (lockval == 0) { > /* > * Can exit now as the lock is free > */ > ACCESS_ONCE(pv->cpustate) =...

...n; > ACCESS_ONCE(pv->cpustate) = PV_CPU_KICKED; > __queue_kick_cpu(pv->mycpu, PV_KICK_QUEUE_HEAD); > > Waiter ------------------------------------------- > > // pv_head_spin_check > ACCESS_ONCE(pv->cpustate) = PV_CPU_HALTED; > lockval = cmpxchg(&qlock->lock, > _QSPINLOCK_LOCKED, > _QSPINLOCK_LOCKED_SLOWPATH); > if (lockval == 0) { > /* > * Can exit now as the lock is free > */ > ACCESS_ONCE(pv->cpustate) =...

...al) { unsigned short sat = 0xff, tmp = var; asm ("addb %[val], %b[var];" "cmovc %[sat], %[var];" : [var] "+r" (tmp) : [val] "ir" (val), [sat] "r" (sat) ); return tmp; } Another thing to try is, instead of threading that lockval throughout the thing, simply: #define _Q_LOCKED_VAL this_cpu_read_stable(cpu_sat) or combined with the above #define _Q_LOCKED_VAL adds(this_cpu_read_stable(cpu_number), 2) and see if the compiler really makes a mess of things.

...e the actual lock holder. + */ + int cpu = (qcode >> (_QCODE_VAL_OFFSET + 2)) - 1; + + ACCESS_ONCE(pv->cpustate) = PV_CPU_KICKING; + __queue_kick_cpu(cpu, PV_KICK_LOCK_HOLDER); + ACCESS_ONCE(pv->cpustate) = PV_CPU_ACTIVE; + } + if (unlikely(*count >= 2*QSPIN_THRESHOLD)) { + u8 lockval; + + /* + * Set the lock byte to _QSPINLOCK_LOCKED_SLOWPATH before + * trying to hibernate itself. It is possible that the + * lock byte had been set to _QSPINLOCK_LOCKED_SLOWPATH + * already. In this case, just proceeds to sleeping. + */ + ACCESS_ONCE(pv->cpustate) = PV_CPU_HALTED;...

...v_qnode *)mcs; + + if (!static_key_false(&paravirt_spinlocks_enabled)) + return; + + (*count)++; + if (pv->cpustate == PV_CPU_KICKED) { + /* + * Reset count and flag + */ + pv->cpustate = PV_CPU_ACTIVE; + *count = 0; + + } else if (unlikely(*count >= 2*QSPIN_THRESHOLD)) { + u8 lockval; + s8 oldstate; + + /* + * Set the lock byte to _Q_LOCKED_SLOWPATH before + * trying to halt itself. It is possible that the + * lock byte had been set to _Q_LOCKED_SLOWPATH + * already (spurious wakeup of queue head after a halt + * or opportunistic setting in pv_halt_check()). + * I...

On Thu, Jul 23, 2020 at 02:32:36PM -0400, Waiman Long wrote: > BTW, do you have any comment on my v2 lock holder cpu info qspinlock patch? > I will have to update the patch to fix the reported 0-day test problem, but > I want to collect other feedback before sending out v3. I want to say I hate it all, it adds instructions to a path we spend an aweful lot of time optimizing without

On Thu, Jul 23, 2020 at 02:32:36PM -0400, Waiman Long wrote: > BTW, do you have any comment on my v2 lock holder cpu info qspinlock patch? > I will have to update the patch to fix the reported 0-day test problem, but > I want to collect other feedback before sending out v3. I want to say I hate it all, it adds instructions to a path we spend an aweful lot of time optimizing without

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit

v5->v6: - Change the optimized 2-task contending code to make it fairer at the expense of a bit of performance. - Add a patch to support unfair queue spinlock for Xen. - Modify the PV qspinlock code to follow what was done in the PV ticketlock. - Add performance data for the unfair lock as well as the PV support code. v4->v5: - Move the optimized 2-task contending code to the

v5->v6: - Change the optimized 2-task contending code to make it fairer at the expense of a bit of performance. - Add a patch to support unfair queue spinlock for Xen. - Modify the PV qspinlock code to follow what was done in the PV ticketlock. - Add performance data for the unfair lock as well as the PV support code. v4->v5: - Move the optimized 2-task contending code to the

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the