search for: cpu_nr

On 02/26, Waiman Long wrote: > > +void queue_spin_lock_slowpath(struct qspinlock *lock, int qsval) > +{ > + unsigned int cpu_nr, qn_idx; > + struct qnode *node, *next; > + u32 prev_qcode, my_qcode; > + > + /* > + * Get the queue node > + */ > + cpu_nr = smp_processor_id(); > + node = get_qnode(&qn_idx); > + > + /* > + * It should never happen that all the queue nodes are being used....

On 02/26, Waiman Long wrote: > > +void queue_spin_lock_slowpath(struct qspinlock *lock, int qsval) > +{ > + unsigned int cpu_nr, qn_idx; > + struct qnode *node, *next; > + u32 prev_qcode, my_qcode; > + > + /* > + * Get the queue node > + */ > + cpu_nr = smp_processor_id(); > + node = get_qnode(&qn_idx); > + > + /* > + * It should never happen that all the queue nodes are being used....

...el/locking/osq_lock.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c index 05a3785..39d1385 100644 --- a/kernel/locking/osq_lock.c +++ b/kernel/locking/osq_lock.c @@ -21,6 +21,11 @@ static inline int encode_cpu(int cpu_nr) return cpu_nr + 1; } +static inline int node_cpu(struct optimistic_spin_node *node) +{ + return node->cpu - 1; +} + static inline struct optimistic_spin_node *decode_cpu(int encoded_cpu_val) { int cpu_nr = encoded_cpu_val - 1; @@ -118,8 +123,11 @@ bool osq_lock(struct optimistic_spin_q...

...d_clr_qcode(struct qspinlock *lock, u32 qcode) +{ + return atomic_cmpxchg(&lock->qlcode, qcode, _QLOCK_LOCKED) == qcode; +} +#endif /* queue_spin_trylock_and_clr_qcode */ + +#ifndef queue_encode_qcode +/** + * queue_encode_qcode - Encode the CPU number & node index into a qnode code + * @cpu_nr: CPU number + * @qn_idx: Queue node index + * Return : A qnode code that can be saved into the qspinlock structure + */ +static inline u32 queue_encode_qcode(u32 cpu_nr, u8 qn_idx) +{ + return ((cpu_nr + 1) << (_QCODE_VAL_OFFSET + 2)) | + (qn_idx << _QCODE_VAL_OFFSET); +} +#endif /* qu...

...r_qcode(struct qspinlock *lock, u32 qcode) +{ + return atomic_cmpxchg(&lock->qlcode, qcode, _QSPINLOCK_LOCKED) == qcode; +} +#endif /* queue_spin_trylock_and_clr_qcode */ + +#ifndef queue_encode_qcode +/** + * queue_encode_qcode - Encode the CPU number & node index into a qnode code + * @cpu_nr: CPU number + * @qn_idx: Queue node index + * Return : A qnode code that can be saved into the qspinlock structure + * + * The lock bit is set in the encoded 32-bit value as the need to encode + * a qnode means that the lock should have been taken. + */ +static u32 queue_encode_qcode(u32 cpu_nr, u8...

...r_qcode(struct qspinlock *lock, u32 qcode) +{ + return atomic_cmpxchg(&lock->qlcode, qcode, _QSPINLOCK_LOCKED) == qcode; +} +#endif /* queue_spin_trylock_and_clr_qcode */ + +#ifndef queue_encode_qcode +/** + * queue_encode_qcode - Encode the CPU number & node index into a qnode code + * @cpu_nr: CPU number + * @qn_idx: Queue node index + * Return : A qnode code that can be saved into the qspinlock structure + * + * The lock bit is set in the encoded 32-bit value as the need to encode + * a qnode means that the lock should have been taken. + */ +static u32 queue_encode_qcode(u32 cpu_nr, u8...

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue

...@@ */ /* + * Para-virtualized queue spinlock support + */ +#ifdef CONFIG_PARAVIRT_SPINLOCKS +#include <asm/pvqspinlock.h> +#else + +#define PV_SET_VAR(type, var, val) +#define PV_VAR(var) 0 + +struct qnode; +struct pv_qvars {}; +static inline void pv_init_vars(struct pv_qvars *pv, int cpu_nr) {} +static inline void pv_set_vars(struct pv_qvars *pv, struct pv_qvars *ppv, + int cpu, struct qnode *prev) {} +static inline void pv_head_spin_check(struct pv_qvars *pv, int *count, + u32 qcode, struct qspinlock *lock) {} +static inline void pv_queue_spin_check(struct pv_qvars *pv, int *c...

change from v2: no code change, fix typos, update some comments change from v1: a simplier definition of default vcpu_is_preempted skip mahcine type check on ppc, and add config. remove dedicated macro. add one patch to drop overload of rwsem_spin_on_owner and mutex_spin_on_owner. add more comments thanks boqun and Peter's suggestion. This patch set aims to fix lock holder preemption

change from v2: no code change, fix typos, update some comments change from v1: a simplier definition of default vcpu_is_preempted skip mahcine type check on ppc, and add config. remove dedicated macro. add one patch to drop overload of rwsem_spin_on_owner and mutex_spin_on_owner. add more comments thanks boqun and Peter's suggestion. This patch set aims to fix lock holder preemption

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

v5->v6: - Change the optimized 2-task contending code to make it fairer at the expense of a bit of performance. - Add a patch to support unfair queue spinlock for Xen. - Modify the PV qspinlock code to follow what was done in the PV ticketlock. - Add performance data for the unfair lock as well as the PV support code. v4->v5: - Move the optimized 2-task contending code to the

v5->v6: - Change the optimized 2-task contending code to make it fairer at the expense of a bit of performance. - Add a patch to support unfair queue spinlock for Xen. - Modify the PV qspinlock code to follow what was done in the PV ticketlock. - Add performance data for the unfair lock as well as the PV support code. v4->v5: - Move the optimized 2-task contending code to the

Excerpts from Waiman Long's message of July 8, 2020 1:33 pm: > On 7/7/20 1:57 AM, Nicholas Piggin wrote: >> Yes, powerpc could certainly get more performance out of the slow >> paths, and then there are a few parameters to tune. >> >> We don't have a good alternate patching for function calls yet, but >> that would be something to do for native vs pv.

Excerpts from Waiman Long's message of July 8, 2020 1:33 pm: > On 7/7/20 1:57 AM, Nicholas Piggin wrote: >> Yes, powerpc could certainly get more performance out of the slow >> paths, and then there are a few parameters to tune. >> >> We don't have a good alternate patching for function calls yet, but >> that would be something to do for native vs pv.

...etry_queue_wait: arch_mutex_cpu_relax(); arch_mcs_spin_unlock_contended(&next->locked); + pv_halt_check(next, lock); } /** @@ -358,7 +394,7 @@ void queue_spin_lock_slowpath(struct qspinlock *lock, u32 val) { struct mcs_spinlock *node; u32 new, old, tail; - int idx; + int idx, cpu_nr; int retry = INT_MAX; /* Retry count, queue if <= 0 */ BUILD_BUG_ON(CONFIG_NR_CPUS >= (1U << _Q_TAIL_CPU_BITS)); @@ -470,11 +506,12 @@ void queue_spin_lock_slowpath(struct qspinlock *lock, u32 val) queue: node = this_cpu_ptr(&mcs_nodes[0]); idx = node->count++; - tail...

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit