search for: _q_locked_slowpath

...t lock code. In essence, the lock waiters will spin for a specified number of times (QSPIN_THRESHOLD = 2^14) and then halted itself. The queue head waiter, unlike the other waiter, will spins 2*QSPIN_THRESHOLD times before halting itself. Before being halted, the queue head waiter will set a flag (_Q_LOCKED_SLOWPATH) in the lock byte to indicate that the unlock slowpath has to be invoked. In the unlock slowpath, the current lock holder will find the queue head by following the previous node pointer links stored in the queue node structure until it finds one that has the qhead flag turned on. It then attempt t...

...uct qspinlock *lock) > +#else > static inline void queue_spin_unlock(struct qspinlock *lock) > +#endif > { > barrier(); > ACCESS_ONCE(*(u8 *)lock) = 0; > barrier(); > } > > +#ifdef CONFIG_PARAVIRT_SPINLOCKS > +/* > + * The lock byte can have a value of _Q_LOCKED_SLOWPATH to indicate > + * that it needs to go through the slowpath to do the unlocking. > + */ > +#define _Q_LOCKED_SLOWPATH (_Q_LOCKED_VAL | 2) > + > +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); > + > +static inline void queue_spin_unlock(struct qspinlock *lock) &g...

...uct qspinlock *lock) > +#else > static inline void queue_spin_unlock(struct qspinlock *lock) > +#endif > { > barrier(); > ACCESS_ONCE(*(u8 *)lock) = 0; > barrier(); > } > > +#ifdef CONFIG_PARAVIRT_SPINLOCKS > +/* > + * The lock byte can have a value of _Q_LOCKED_SLOWPATH to indicate > + * that it needs to go through the slowpath to do the unlocking. > + */ > +#define _Q_LOCKED_SLOWPATH (_Q_LOCKED_VAL | 2) > + > +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); > + > +static inline void queue_spin_unlock(struct qspinlock *lock) &g...

...icate that + * racing condition may happen. This flag is set when the + * loop count is getting close to the halting threshold. + * + * When that happens, a 2 variables (cpustate & node->locked + * handshake is used to make sure that pv_wait_check() won't + * miss setting the _Q_LOCKED_SLOWPATH when the CPU is about + * to be halted. + * + * pv_wait_check pv_link_and_wait_node + * ------------- --------------------- + * [1] node->locked = true [3] cpustate = PV_CPU_HALTED + * smp_mb() smp_mb() + * [2] if (cpustate [4] if (node->locked) + * == PV_C...

...icate that + * racing condition may happen. This flag is set when the + * loop count is getting close to the halting threshold. + * + * When that happens, a 2 variables (cpustate & node->locked + * handshake is used to make sure that pv_wait_check() won't + * miss setting the _Q_LOCKED_SLOWPATH when the CPU is about + * to be halted. + * + * pv_wait_check pv_link_and_wait_node + * ------------- --------------------- + * [1] node->locked = true [3] cpustate = PV_CPU_HALTED + * smp_mb() smp_mb() + * [2] if (cpustate [4] if (node->locked) + * == PV_C...

...icate that + * racing condition may happen. This flag is set when the + * loop count is getting close to the halting threshold. + * + * When that happens, a 2 variables (cpustate & node->locked + * handshake is used to make sure that pv_wait_check() won't + * miss setting the _Q_LOCKED_SLOWPATH when the CPU is about + * to be halted. + * + * pv_wait_check pv_link_and_wait_node + * ------------- --------------------- + * [1] node->locked = true [3] cpustate = PV_CPU_HALTED + * smp_mb() smp_mb() + * [2] if (cpustate [4] if (node->locked) + * == PV_C...

...icate that + * racing condition may happen. This flag is set when the + * loop count is getting close to the halting threshold. + * + * When that happens, a 2 variables (cpustate & node->locked + * handshake is used to make sure that pv_wait_check() won't + * miss setting the _Q_LOCKED_SLOWPATH when the CPU is about + * to be halted. + * + * pv_wait_check pv_link_and_wait_node + * ------------- --------------------- + * [1] node->locked = true [3] cpustate = PV_CPU_HALTED + * smp_mb() smp_mb() + * [2] if (cpustate [4] if (node->locked) + * == PV_C...

...ic inline void queue_spin_unlock(struct qspinlock *lock) >> +#endif >> { >> barrier(); >> ACCESS_ONCE(*(u8 *)lock) = 0; >> barrier(); >> } >> >> +#ifdef CONFIG_PARAVIRT_SPINLOCKS >> +/* >> + * The lock byte can have a value of _Q_LOCKED_SLOWPATH to indicate >> + * that it needs to go through the slowpath to do the unlocking. >> + */ >> +#define _Q_LOCKED_SLOWPATH (_Q_LOCKED_VAL | 2) >> + >> +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); >> + >> +static inline void queue_spin_unloc...

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the

v11->v12: - Based on PeterZ's version of the qspinlock patch (https://lkml.org/lkml/2014/6/15/63). - Incorporated many of the review comments from Konrad Wilk and Paolo Bonzini. - The pvqspinlock code is largely from my previous version with PeterZ's way of going from queue tail to head and his idea of using callee saved calls to KVM and XEN codes. v10->v11: - Use a

v11->v12: - Based on PeterZ's version of the qspinlock patch (https://lkml.org/lkml/2014/6/15/63). - Incorporated many of the review comments from Konrad Wilk and Paolo Bonzini. - The pvqspinlock code is largely from my previous version with PeterZ's way of going from queue tail to head and his idea of using callee saved calls to KVM and XEN codes. v10->v11: - Use a

v13->v14: - Patches 1 & 2: Add queue_spin_unlock_wait() to accommodate commit 78bff1c86 from Oleg Nesterov. - Fix the system hang problem when using PV qspinlock in an over-committed guest due to a racing condition in the pv_set_head_in_tail() function. - Increase the MAYHALT_THRESHOLD from 10 to 1024. - Change kick_cpu into a regular function pointer instead of a

v13->v14: - Patches 1 & 2: Add queue_spin_unlock_wait() to accommodate commit 78bff1c86 from Oleg Nesterov. - Fix the system hang problem when using PV qspinlock in an over-committed guest due to a racing condition in the pv_set_head_in_tail() function. - Increase the MAYHALT_THRESHOLD from 10 to 1024. - Change kick_cpu into a regular function pointer instead of a

v12->v13: - Change patch 9 to generate separate versions of the queue_spin_lock_slowpath functions for bare metal and PV guest. This reduces the performance impact of the PV code on bare metal systems. v11->v12: - Based on PeterZ's version of the qspinlock patch (https://lkml.org/lkml/2014/6/15/63). - Incorporated many of the review comments from Konrad Wilk and Paolo

v12->v13: - Change patch 9 to generate separate versions of the queue_spin_lock_slowpath functions for bare metal and PV guest. This reduces the performance impact of the PV code on bare metal systems. v11->v12: - Based on PeterZ's version of the qspinlock patch (https://lkml.org/lkml/2014/6/15/63). - Incorporated many of the review comments from Konrad Wilk and Paolo

v8->v9: - Integrate PeterZ's version of the queue spinlock patch with some modification: http://lkml.kernel.org/r/20140310154236.038181843 at infradead.org - Break the more complex patches into smaller ones to ease review effort. - Fix a racing condition in the PV qspinlock code. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving

v8->v9: - Integrate PeterZ's version of the queue spinlock patch with some modification: http://lkml.kernel.org/r/20140310154236.038181843 at infradead.org - Break the more complex patches into smaller ones to ease review effort. - Fix a racing condition in the PV qspinlock code. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving

v9->v10: - Make some minor changes to qspinlock.c to accommodate review feedback. - Change author to PeterZ for 2 of the patches. - Include Raghavendra KT's test results in patch 18. v8->v9: - Integrate PeterZ's version of the queue spinlock patch with some modification: http://lkml.kernel.org/r/20140310154236.038181843 at infradead.org - Break the more complex

v9->v10: - Make some minor changes to qspinlock.c to accommodate review feedback. - Change author to PeterZ for 2 of the patches. - Include Raghavendra KT's test results in patch 18. v8->v9: - Integrate PeterZ's version of the queue spinlock patch with some modification: http://lkml.kernel.org/r/20140310154236.038181843 at infradead.org - Break the more complex