search for: _qcode_offset

...AITING (1 << _QSPINLOCK_WAIT_SHIFT) +/* Masks for lock & wait bits */ +#define _QSPINLOCK_LWMASK (_QSPINLOCK_WAITING | _QSPINLOCK_LOCKED) + #define queue_encode_qcode(cpu, idx) (((cpu) + 1) << 2 | (idx)) +#define queue_get_qcode(lock) (atomic_read(&(lock)->qlcode) >> _QCODE_OFFSET) + +#define queue_spin_trylock_quick queue_spin_trylock_quick +/** + * queue_spin_trylock_quick - quick spinning on the queue spinlock + * @lock : Pointer to queue spinlock structure + * @qsval: Old queue spinlock value + * Return: 1 if lock acquired, 0 if failed + * + * This is an optimized conten...

...ric/qspinlock_types.h b/include/asm-generic/qspinlock_types.h index df981d0..3a02a9e 100644 --- a/include/asm-generic/qspinlock_types.h +++ b/include/asm-generic/qspinlock_types.h @@ -48,7 +48,13 @@ typedef struct qspinlock { atomic_t qlcode; /* Lock + queue code */ } arch_spinlock_t; -#define _QCODE_OFFSET 8 +#if CONFIG_NR_CPUS >= (1 << 14) +# define _Q_MANY_CPUS +# define _QCODE_OFFSET 8 +#else +# define _QCODE_OFFSET 16 +#endif + #define _QSPINLOCK_LOCKED 1U #define _QSPINLOCK_LOCK_MASK 0xff diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c index ed5efa7..22a63fa 10...

...ric/qspinlock_types.h b/include/asm-generic/qspinlock_types.h index df981d0..3a02a9e 100644 --- a/include/asm-generic/qspinlock_types.h +++ b/include/asm-generic/qspinlock_types.h @@ -48,7 +48,13 @@ typedef struct qspinlock { atomic_t qlcode; /* Lock + queue code */ } arch_spinlock_t; -#define _QCODE_OFFSET 8 +#if CONFIG_NR_CPUS >= (1 << 14) +# define _Q_MANY_CPUS +# define _QCODE_OFFSET 8 +#else +# define _QCODE_OFFSET 16 +#endif + #define _QSPINLOCK_LOCKED 1U #define _QSPINLOCK_LOCK_MASK 0xff diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c index ed5efa7..22a63fa 10...

..._nr, qn_idx); > + > + /* > + * Initialize the queue node > + */ > + node->wait = true; > + node->next = NULL; > + > + /* > + * The lock may be available at this point, try again if no task was > + * waiting in the queue. > + */ > + if (!(qsval >> _QCODE_OFFSET) && queue_spin_trylock(lock)) { > + put_qnode(); > + return; > + } Cosmetic, but probably "goto release_node" would be more consistent. And I am wondering how much this "qsval >> _QCODE_OFFSET" check can help. Note that this is the only usage of this a...

..._nr, qn_idx); > + > + /* > + * Initialize the queue node > + */ > + node->wait = true; > + node->next = NULL; > + > + /* > + * The lock may be available at this point, try again if no task was > + * waiting in the queue. > + */ > + if (!(qsval >> _QCODE_OFFSET) && queue_spin_trylock(lock)) { > + put_qnode(); > + return; > + } Cosmetic, but probably "goto release_node" would be more consistent. And I am wondering how much this "qsval >> _QCODE_OFFSET" check can help. Note that this is the only usage of this a...

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue

v5->v6: - Change the optimized 2-task contending code to make it fairer at the expense of a bit of performance. - Add a patch to support unfair queue spinlock for Xen. - Modify the PV qspinlock code to follow what was done in the PV ticketlock. - Add performance data for the unfair lock as well as the PV support code. v4->v5: - Move the optimized 2-task contending code to the

v5->v6: - Change the optimized 2-task contending code to make it fairer at the expense of a bit of performance. - Add a patch to support unfair queue spinlock for Xen. - Modify the PV qspinlock code to follow what was done in the PV ticketlock. - Add performance data for the unfair lock as well as the PV support code. v4->v5: - Move the optimized 2-task contending code to the

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

....h> +#endif + +/* + * The queue spinlock data structure - a 32-bit word + * + * The bits assignment are: + * Bit 0 : Set if locked + * Bits 1-7 : Not used + * Bits 8-31: Queue code + */ +typedef struct qspinlock { + atomic_t qlcode; /* Lock + queue code */ +} arch_spinlock_t; + +#define _QCODE_OFFSET 8 +#define _QLOCK_LOCKED 1U +#define _QLOCK_LOCK_MASK 0xff + +#endif /* __ASM_GENERIC_QSPINLOCK_TYPES_H */ diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index d2b32ac..f185584 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -223,3 +223,10 @@ endif config MUTEX_SPIN_ON_...

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

...PUS < 16K, the bits assignment are: + * Bit 0 : Set if locked + * Bits 1-7 : Not used + * Bits 8-15: Reserved for architecture specific optimization + * Bits 16-31: Queue code + */ +typedef struct qspinlock { + atomic_t qlcode; /* Lock + queue code */ +} arch_spinlock_t; + +#define _QCODE_OFFSET 8 +#define _QSPINLOCK_LOCKED 1U +#define _QSPINLOCK_LOCK_MASK 0xff + +#endif /* __ASM_GENERIC_QSPINLOCK_TYPES_H */ diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index d2b32ac..f185584 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -223,3 +223,10 @@ endif config MUTEX_S...

...PUS < 16K, the bits assignment are: + * Bit 0 : Set if locked + * Bits 1-7 : Not used + * Bits 8-15: Reserved for architecture specific optimization + * Bits 16-31: Queue code + */ +typedef struct qspinlock { + atomic_t qlcode; /* Lock + queue code */ +} arch_spinlock_t; + +#define _QCODE_OFFSET 8 +#define _QSPINLOCK_LOCKED 1U +#define _QSPINLOCK_LOCK_MASK 0xff + +#endif /* __ASM_GENERIC_QSPINLOCK_TYPES_H */ diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index d2b32ac..f185584 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -223,3 +223,10 @@ endif config MUTEX_S...

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue

v4->v5: - Move the optimized 2-task contending code to the generic file to enable more architectures to use it without code duplication. - Address some of the style-related comments by PeterZ. - Allow the use of unfair queue spinlock in a real para-virtualized execution environment. - Add para-virtualization support to the qspinlock code by ensuring that the lock holder and queue