search for: pv_head_spin_check

...pvticketlocks, where the *unlock* > primitive wakes up a sleeping VCPU. It is more similar to PLE > (pause-loop exiting). Adding to the discussion, I see there are two possibilities here, considering that in undercommit cases we should not exceed HEAD_SPIN_THRESHOLD, 1. the looping vcpu in pv_head_spin_check() should do halt() considering that we have done enough spinning (more than typical lock-hold time), and hence we are in potential overcommit. 2. multiplex kick_cpu to do directed yield in qspinlock case. But this may result in some ping ponging?

...pvticketlocks, where the *unlock* > primitive wakes up a sleeping VCPU. It is more similar to PLE > (pause-loop exiting). Adding to the discussion, I see there are two possibilities here, considering that in undercommit cases we should not exceed HEAD_SPIN_THRESHOLD, 1. the looping vcpu in pv_head_spin_check() should do halt() considering that we have done enough spinning (more than typical lock-hold time), and hence we are in potential overcommit. 2. multiplex kick_cpu to do directed yield in qspinlock case. But this may result in some ping ponging?

...s long as the + * other nodes before it puts itself to halt. The reason for that is to + * increase halting chance of heavily contended locks to favor lightly + * contended locks (queue depth of 1 or less). + * + * There are 2 places where races can happen: + * 1) Halting of the queue head CPU (in pv_head_spin_check) and the CPU + * kicking by the lock holder in the unlock path (in pv_kick_node). + * 2) Halting of the queue node CPU (in pv_queue_spin_check) and the + * the status check by the previous queue head (in pv_halt_check). + * See the comments on those functions to see how the races are being...

...fields in struct pv_qvars + * @pv : pointer to struct pv_qvars + * @cpu: current CPU number + */ +static __always_inline void pv_init_vars(struct pv_qvars *pv, int cpu) +{ + pv->cpustate = PV_CPU_ACTIVE; + pv->prev = NULL; + pv->nxtcpu_p1 = 0; + pv->mycpu = cpu; +} + +/** + * pv_head_spin_check - perform para-virtualization checks for queue head + * @pv : pointer to struct pv_qvars + * @count : loop count + * @qcode : queue code of the supposed lock holder + * @lock : pointer to the qspinlock structure + * + * The following checks will be done: + * 1) Attempt to kick the lock holder,...

...to kick the lock holder + * 2) Set the prehead flag of the next node + * 3) Clear the active flag of the next node periodically + * 4) If the active flag is not set after a while, assume the CPU of the + * next-in-line node is offline and kick it back up again. + */ +static __always_inline void +pv_head_spin_check(int *count, u32 qcode, int nxtcpu, void *next, int offset) +{ + if (!static_key_false(&paravirt_spinlocks_enabled)) + return; + if ((++(*count) == HEAD_SPIN_THRESHOLD) && qcode) { + /* + * Get the CPU number of the lock holder & kick it + * The lock may have been stealed by an...

On 12/03/14 18:54, Waiman Long wrote: > This patch adds para-virtualization support to the queue spinlock in > the same way as was done in the PV ticket lock code. In essence, the > lock waiters will spin for a specified number of times (QSPIN_THRESHOLD > = 2^14) and then halted itself. The queue head waiter will spins > 2*QSPIN_THRESHOLD times before halting itself. When it has

On 12/03/14 18:54, Waiman Long wrote: > This patch adds para-virtualization support to the queue spinlock in > the same way as was done in the PV ticket lock code. In essence, the > lock waiters will spin for a specified number of times (QSPIN_THRESHOLD > = 2^14) and then halted itself. The queue head waiter will spins > 2*QSPIN_THRESHOLD times before halting itself. When it has

...t;> primitive wakes up a sleeping VCPU. It is more similar to PLE >> (pause-loop exiting). > > Adding to the discussion, I see there are two possibilities here, > considering that in undercommit cases we should not exceed > HEAD_SPIN_THRESHOLD, > > 1. the looping vcpu in pv_head_spin_check() should do halt() > considering that we have done enough spinning (more than typical > lock-hold time), and hence we are in potential overcommit. > > 2. multiplex kick_cpu to do directed yield in qspinlock case. > But this may result in some ping ponging? Actually, I think the qspi...

...e_spin_unlock barrier(); ACCESS_ONCE(qlock->lock) = 0; barrier(); // pv_kick_node: if (pv->cpustate != PV_CPU_HALTED) return; ACCESS_ONCE(pv->cpustate) = PV_CPU_KICKED; __queue_kick_cpu(pv->mycpu, PV_KICK_QUEUE_HEAD); Waiter ------------------------------------------- // pv_head_spin_check ACCESS_ONCE(pv->cpustate) = PV_CPU_HALTED; lockval = cmpxchg(&qlock->lock, _QSPINLOCK_LOCKED, _QSPINLOCK_LOCKED_SLOWPATH); if (lockval == 0) { /* * Can exit now as the lock is free */ ACCESS_ONCE(pv->cpustate) = PV_CPU_ACTIVE; *count = 0; return;...

On 27/02/14 13:11, Paolo Bonzini wrote: > Il 27/02/2014 13:11, David Vrabel ha scritto: >>> > This patch adds para-virtualization support to the queue spinlock code >>> > by enabling the queue head to kick the lock holder CPU, if known, >>> > in when the lock isn't released for a certain amount of time. It >>> > also enables the mutual

On 27/02/14 13:11, Paolo Bonzini wrote: > Il 27/02/2014 13:11, David Vrabel ha scritto: >>> > This patch adds para-virtualization support to the queue spinlock code >>> > by enabling the queue head to kick the lock holder CPU, if known, >>> > in when the lock isn't released for a certain amount of time. It >>> > also enables the mutual

...> // pv_kick_node: > if (pv->cpustate != PV_CPU_HALTED) > return; > ACCESS_ONCE(pv->cpustate) = PV_CPU_KICKED; > __queue_kick_cpu(pv->mycpu, PV_KICK_QUEUE_HEAD); > > Waiter ------------------------------------------- > > // pv_head_spin_check > ACCESS_ONCE(pv->cpustate) = PV_CPU_HALTED; > lockval = cmpxchg(&qlock->lock, > _QSPINLOCK_LOCKED, > _QSPINLOCK_LOCKED_SLOWPATH); > if (lockval == 0) { > /* > * Can exit now as the l...

...> // pv_kick_node: > if (pv->cpustate != PV_CPU_HALTED) > return; > ACCESS_ONCE(pv->cpustate) = PV_CPU_KICKED; > __queue_kick_cpu(pv->mycpu, PV_KICK_QUEUE_HEAD); > > Waiter ------------------------------------------- > > // pv_head_spin_check > ACCESS_ONCE(pv->cpustate) = PV_CPU_HALTED; > lockval = cmpxchg(&qlock->lock, > _QSPINLOCK_LOCKED, > _QSPINLOCK_LOCKED_SLOWPATH); > if (lockval == 0) { > /* > * Can exit now as the l...

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.