search for: paravirt_spinlocks_en

From: Waiman Long <Waiman.Long at hp.com> This patch renames the paravirt_ticketlocks_enabled static key to a more generic paravirt_spinlocks_enabled name. Signed-off-by: Waiman Long <Waiman.Long at hp.com> Signed-off-by: Peter Zijlstra <peterz at infradead.org> --- arch/x86/include/asm/spinlock.h | 4 ++-- arch/x86/kernel/kvm.c | 2 +- arch/x86/kernel/paravirt-spinlocks.c | 4 ++-- arch/x86/xen/sp...

This patch renames the paravirt_ticketlocks_enabled static key to a more generic paravirt_spinlocks_enabled name. Signed-off-by: Waiman Long <Waiman.Long at hp.com> Signed-off-by: Peter Zijlstra <peterz at infradead.org> --- arch/x86/include/asm/spinlock.h | 4 ++-- arch/x86/kernel/kvm.c | 2 +- arch/x86/kernel/paravirt-spinlocks.c | 4 ++-- arch/x86/xen/sp...

This patch renames the paravirt_ticketlocks_enabled static key to a more generic paravirt_spinlocks_enabled name. Signed-off-by: Waiman Long <Waiman.Long at hp.com> Signed-off-by: Peter Zijlstra <peterz at infradead.org> --- arch/x86/include/asm/spinlock.h | 4 ++-- arch/x86/kernel/kvm.c | 2 +- arch/x86/kernel/paravirt-spinlocks.c | 4 ++-- arch/x86/xen/sp...

On 10/27/2014 02:04 PM, Peter Zijlstra wrote: > On Mon, Oct 27, 2014 at 01:38:20PM -0400, Waiman Long wrote: >> On 10/24/2014 04:54 AM, Peter Zijlstra wrote: >>> On Thu, Oct 16, 2014 at 02:10:38PM -0400, Waiman Long wrote: >>> >>>> Since enabling paravirt spinlock will disable unlock function inlining, >>>> a jump label can be added to the unlock

On 10/27/2014 02:04 PM, Peter Zijlstra wrote: > On Mon, Oct 27, 2014 at 01:38:20PM -0400, Waiman Long wrote: >> On 10/24/2014 04:54 AM, Peter Zijlstra wrote: >>> On Thu, Oct 16, 2014 at 02:10:38PM -0400, Waiman Long wrote: >>> >>>> Since enabling paravirt spinlock will disable unlock function inlining, >>>> a jump label can be added to the unlock

...mcs: pointer to struct mcs_spinlock + * @cpu: current CPU number + */ +static inline void pv_init_vars(struct mcs_spinlock *node, int cpu) +{ + struct pv_qnode *pv = (struct pv_qnode *)node; + + BUILD_BUG_ON(sizeof(struct pv_qnode) > 5*sizeof(struct mcs_spinlock)); + + if (!static_key_false(&paravirt_spinlocks_enabled)) + return; + + pv->cpustate = PV_CPU_ACTIVE; + pv->prev = NULL; + pv->mayhalt = false; + pv->mycpu = cpu; +} + +/** + * pv_head_spin_check - perform para-virtualization checks for queue head + * @mcs : pointer to the mcs_spinlock structure + * @lock : pointer to the qspi...

...wpath to do the unlocking. > + */ > +#define _Q_LOCKED_SLOWPATH (_Q_LOCKED_VAL | 2) > + > +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); > + > +static inline void queue_spin_unlock(struct qspinlock *lock) > +{ > + barrier(); > + if (static_key_false(&paravirt_spinlocks_enabled)) { > + /* > + * Need to atomically clear the lock byte to avoid racing with > + * queue head waiter trying to set _QLOCK_LOCKED_SLOWPATH. > + */ > + if (likely(cmpxchg((u8 *)lock, _Q_LOCKED_VAL, 0) > + == _Q_LOCKED_VAL)) > + return; > + else > + queu...

...wpath to do the unlocking. > + */ > +#define _Q_LOCKED_SLOWPATH (_Q_LOCKED_VAL | 2) > + > +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); > + > +static inline void queue_spin_unlock(struct qspinlock *lock) > +{ > + barrier(); > + if (static_key_false(&paravirt_spinlocks_enabled)) { > + /* > + * Need to atomically clear the lock byte to avoid racing with > + * queue head waiter trying to set _QLOCK_LOCKED_SLOWPATH. > + */ > + if (likely(cmpxchg((u8 *)lock, _Q_LOCKED_VAL, 0) > + == _Q_LOCKED_VAL)) > + return; > + else > + queu...

...+ */ >> +#define _Q_LOCKED_SLOWPATH (_Q_LOCKED_VAL | 2) >> + >> +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); >> + >> +static inline void queue_spin_unlock(struct qspinlock *lock) >> +{ >> + barrier(); >> + if (static_key_false(&paravirt_spinlocks_enabled)) { >> + /* >> + * Need to atomically clear the lock byte to avoid racing with >> + * queue head waiter trying to set _QLOCK_LOCKED_SLOWPATH. >> + */ >> + if (likely(cmpxchg((u8 *)lock, _Q_LOCKED_VAL, 0) >> + == _Q_LOCKED_VAL)) >> + return...

...pile time fail for that, not a comment. > + */ > +static __always_inline void queue_spin_lock(struct qspinlock *lock) > +{ > + u32 val; > + > + val = atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL); > + if (likely(val == 0)) > + return; > + if (static_key_false(&paravirt_spinlocks_enabled)) > + pv_queue_spin_lock_slowpath(lock, val); > + else > + queue_spin_lock_slowpath(lock, val); > +} No, this is just vile.. _that_ is what we have PV ops for. And at that point its the same function it was before the PV stuff, so that whole inline thing is then gone. > +ext...

...t is > worthwhile to reduce the performance delta between the PV and non-PV > kernel on bare metal. I am sorry that the unlock call sites patching code doesn't work in a virtual guest. Your pvqspinlock patch did an unconditional patching even in a virtual guest. I added check for the paravirt_spinlocks_enabled, but it turned out that some spin_unlock() seemed to be called before paravirt_spinlocks_enabled is set. As a result, some call sites were still patched resulting in missed wake up's and system hang. At this point, I am going to leave out that change from my patch set until we can fig...

...1) Attempt to kick the lock holder, if known, after QSPIN_THRESHOLD + * 2) Halt itself if lock is still not available after 2*QSPIN_THRESHOLD + */ +static __always_inline void pv_head_spin_check(struct pv_qvars *pv, int *count, + u32 qcode, struct qspinlock *lock) +{ + if (!static_key_false(&paravirt_spinlocks_enabled)) + return; + if (unlikely((++(*count) == QSPIN_THRESHOLD) && qcode)) { + /* + * Get the CPU number of the lock holder & kick it + * The lock may have been stealed by another CPU + * if PARAVIRT_UNFAIR_LOCKS is set, so the computed + * CPU number may not be the actual loc...

...lder --------------------------------------- > > // queue_spin_unlock > barrier(); > ACCESS_ONCE(qlock->lock) = 0; > barrier(); > This is not the unlock code that is used when PV spinlock is enabled. The right unlock code is if (static_key_false(&paravirt_spinlocks_enabled)) { /* * Need to atomically clear the lock byte to avoid racing with * queue head waiter trying to set _QSPINLOCK_LOCKED_SLOWPATH. */ if (likely(cmpxchg(&qlock->lock, _QSPINLOCK_LOCKED, 0)...

...lder --------------------------------------- > > // queue_spin_unlock > barrier(); > ACCESS_ONCE(qlock->lock) = 0; > barrier(); > This is not the unlock code that is used when PV spinlock is enabled. The right unlock code is if (static_key_false(&paravirt_spinlocks_enabled)) { /* * Need to atomically clear the lock byte to avoid racing with * queue head waiter trying to set _QSPINLOCK_LOCKED_SLOWPATH. */ if (likely(cmpxchg(&qlock->lock, _QSPINLOCK_LOCKED, 0)...

...ure + * + * N.B. INLINE_SPIN_LOCK should not be enabled when PARAVIRT_SPINLOCK is on. + */ +static __always_inline void queue_spin_lock(struct qspinlock *lock) +{ + u32 val; + + val = atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL); + if (likely(val == 0)) + return; + if (static_key_false(&paravirt_spinlocks_enabled)) + pv_queue_spin_lock_slowpath(lock, val); + else + queue_spin_lock_slowpath(lock, val); +} + +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); + /** * queue_spin_unlock - release a queue spinlock * @lock : Pointer to queue spinlock structure * * An effective smp_sto...

...ure + * + * N.B. INLINE_SPIN_LOCK should not be enabled when PARAVIRT_SPINLOCK is on. + */ +static __always_inline void queue_spin_lock(struct qspinlock *lock) +{ + u32 val; + + val = atomic_cmpxchg(&lock->val, 0, _Q_LOCKED_VAL); + if (likely(val == 0)) + return; + if (static_key_false(&paravirt_spinlocks_enabled)) + pv_queue_spin_lock_slowpath(lock, val); + else + queue_spin_lock_slowpath(lock, val); +} + +extern void queue_spin_unlock_slowpath(struct qspinlock *lock); + /** * queue_spin_unlock - release a queue spinlock * @lock : Pointer to queue spinlock structure * * An effective smp_sto...

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the

v10->v11: - Use a simple test-and-set unfair lock to simplify the code, but performance may suffer a bit for large guest with many CPUs. - Take out Raghavendra KT's test results as the unfair lock changes may render some of his results invalid. - Add PV support without increasing the size of the core queue node structure. - Other minor changes to address some of the

Since Waiman seems incapable of doing simple things; here's my take on the paravirt crap. The first few patches are taken from Waiman's latest series, but the virt support is completely new. Its primary aim is to not mess up the native code. I've not stress tested it, but the virt and paravirt (kvm) cases boot on simple smp guests. I've not done Xen, but the patch should be

Since Waiman seems incapable of doing simple things; here's my take on the paravirt crap. The first few patches are taken from Waiman's latest series, but the virt support is completely new. Its primary aim is to not mess up the native code. I've not stress tested it, but the virt and paravirt (kvm) cases boot on simple smp guests. I've not done Xen, but the patch should be