search for: release_node

...> + > + /* > + * The lock may be available at this point, try again if no task was > + * waiting in the queue. > + */ > + if (!(qsval >> _QCODE_OFFSET) && queue_spin_trylock(lock)) { > + put_qnode(); > + return; > + } Cosmetic, but probably "goto release_node" would be more consistent. And I am wondering how much this "qsval >> _QCODE_OFFSET" check can help. Note that this is the only usage of this arg, perhaps it would be better to simply remove it and shrink the caller's code a bit? It is also used in 3/8, but we can read the...

...> + > + /* > + * The lock may be available at this point, try again if no task was > + * waiting in the queue. > + */ > + if (!(qsval >> _QCODE_OFFSET) && queue_spin_trylock(lock)) { > + put_qnode(); > + return; > + } Cosmetic, but probably "goto release_node" would be more consistent. And I am wondering how much this "qsval >> _QCODE_OFFSET" check can help. Note that this is the only usage of this arg, perhaps it would be better to simply remove it and shrink the caller's code a bit? It is also used in 3/8, but we can read the...

...00008 > [..] > CPU: 15 PID: 1414 Comm: lt-libndctl Tainted: G OE > 5.2.0-rc3+ #3399 > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015 > RIP: 0010:percpu_ref_kill_and_confirm+0x1e/0x180 > [..] > Call Trace: > release_nodes+0x234/0x280 > device_release_driver_internal+0xe8/0x1b0 > bus_remove_device+0xf2/0x160 > device_del+0x166/0x370 > unregister_dev_dax+0x23/0x50 > release_nodes+0x234/0x280 > device_release_driver_internal+0xe8/0x1b0 > unbind_store+0x94/0x120 &...

...e need to add + * 1 to it before putting it into the queue code. + */ +#define MAX_QNODES 4 +#ifndef _QCODE_VAL_OFFSET +#define _QCODE_VAL_OFFSET _QCODE_OFFSET +#endif + +/* + * Function exit status + */ +enum exitval { + NORMAL_EXIT = 0, + NOTIFY_NEXT , /* Notify the next waiting node CPU */ + RELEASE_NODE /* Release current node directly */ +}; + +/* + * The queue node structure + * + * This structure is essentially the same as the mcs_spinlock structure + * in mcs_spinlock.h file. It is retained for future extension where new + * fields may be added. + */ +struct qnode { + u32 qhead; /* Queu...

...ointer dereference, address: 0000000000000008 [..] CPU: 15 PID: 1414 Comm: lt-libndctl Tainted: G OE 5.2.0-rc3+ #3399 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015 RIP: 0010:percpu_ref_kill_and_confirm+0x1e/0x180 [..] Call Trace: release_nodes+0x234/0x280 device_release_driver_internal+0xe8/0x1b0 bus_remove_device+0xf2/0x160 device_del+0x166/0x370 unregister_dev_dax+0x23/0x50 release_nodes+0x234/0x280 device_release_driver_internal+0xe8/0x1b0 unbind_store+0x94/0x120 kernfs_fop_write+0xf0/0x1a0...

...+0x4c/0x88 [ 3.608400] brk_handler+0x11c/0x218 [ 3.613430] do_debug_exception+0xe8/0x318 [ 3.627370] el1_dbg+0x18/0x78 [ 3.634037] __vunmap+0x1b8/0x220 [ 3.648747] vunmap+0x6c/0xc0 [ 3.653864] __iounmap+0x44/0x58 [ 3.659771] devm_ioremap_release+0x34/0x68 [ 3.672983] release_nodes+0x404/0x880 [ 3.683543] devres_release_all+0x6c/0xe8 [ 3.695692] driver_probe_device+0x250/0x828 [ 3.706187] __driver_attach+0x190/0x210 [ 3.717645] bus_for_each_dev+0x14c/0x1f0 [ 3.728633] driver_attach+0x48/0x78 [ 3.740249] bus_add_driver+0x26c/0x5b8 [ 3.752248] drive...

...+0x4c/0x88 [ 3.608400] brk_handler+0x11c/0x218 [ 3.613430] do_debug_exception+0xe8/0x318 [ 3.627370] el1_dbg+0x18/0x78 [ 3.634037] __vunmap+0x1b8/0x220 [ 3.648747] vunmap+0x6c/0xc0 [ 3.653864] __iounmap+0x44/0x58 [ 3.659771] devm_ioremap_release+0x34/0x68 [ 3.672983] release_nodes+0x404/0x880 [ 3.683543] devres_release_all+0x6c/0xe8 [ 3.695692] driver_probe_device+0x250/0x828 [ 3.706187] __driver_attach+0x190/0x210 [ 3.717645] bus_for_each_dev+0x14c/0x1f0 [ 3.728633] driver_attach+0x48/0x78 [ 3.740249] bus_add_driver+0x26c/0x5b8 [ 3.752248] drive...

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

v6->v7: - Remove an atomic operation from the 2-task contending code - Shorten the names of some macros - Make the queue waiter to attempt to steal lock when unfair lock is enabled. - Remove lock holder kick from the PV code and fix a race condition - Run the unfair lock & PV code on overcommitted KVM guests to collect performance data. v5->v6: - Change the optimized

...uct devres is allocated together with the memory, since alloc_dr() does: size_t tot_size = sizeof(struct devres) + size; struct devres *dr; dr = kmalloc_node_track_caller(tot_size, gfp, nid); return dr->data; ... where dr->data points at the memory after the struct devres. Later, in release_nodes() we do: list_for_each_entry_safe_reverse(dr, tmp, &todo, node.entry) { devres_log(dev, &dr->node, "REL"); dr->node.release(dev, dr->data); kfree(dr); } ... which will invoke the no-op devm_kmalloc_release, then free the devres allocation, including the dr-...

...uct devres is allocated together with the memory, since alloc_dr() does: size_t tot_size = sizeof(struct devres) + size; struct devres *dr; dr = kmalloc_node_track_caller(tot_size, gfp, nid); return dr->data; ... where dr->data points at the memory after the struct devres. Later, in release_nodes() we do: list_for_each_entry_safe_reverse(dr, tmp, &todo, node.entry) { devres_log(dev, &dr->node, "REL"); dr->node.release(dev, dr->data); kfree(dr); } ... which will invoke the no-op devm_kmalloc_release, then free the devres allocation, including the dr-...

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit to enable/disable unfair lock. - Reduce unfair lock slowpath lock stealing frequency depending on its distance from the queue head. - Add performance data for IvyBridge-EX CPU.

...3430] do_debug_exception+0xe8/0x318 > > [ 3.627370] el1_dbg+0x18/0x78 > > [ 3.634037] __vunmap+0x1b8/0x220 > > [ 3.648747] vunmap+0x6c/0xc0 > > [ 3.653864] __iounmap+0x44/0x58 > > [ 3.659771] devm_ioremap_release+0x34/0x68 > > [ 3.672983] release_nodes+0x404/0x880 > > [ 3.683543] devres_release_all+0x6c/0xe8 > > [ 3.695692] driver_probe_device+0x250/0x828 > > [ 3.706187] __driver_attach+0x190/0x210 > > [ 3.717645] bus_for_each_dev+0x14c/0x1f0 > > [ 3.728633] driver_attach+0x48/0x78 > > [...

...3430] do_debug_exception+0xe8/0x318 > > [ 3.627370] el1_dbg+0x18/0x78 > > [ 3.634037] __vunmap+0x1b8/0x220 > > [ 3.648747] vunmap+0x6c/0xc0 > > [ 3.653864] __iounmap+0x44/0x58 > > [ 3.659771] devm_ioremap_release+0x34/0x68 > > [ 3.672983] release_nodes+0x404/0x880 > > [ 3.683543] devres_release_all+0x6c/0xe8 > > [ 3.695692] driver_probe_device+0x250/0x828 > > [ 3.706187] __driver_attach+0x190/0x210 > > [ 3.717645] bus_for_each_dev+0x14c/0x1f0 > > [ 3.728633] driver_attach+0x48/0x78 > > [...

...+0x4c/0x88 [ 3.608400] brk_handler+0x11c/0x218 [ 3.613430] do_debug_exception+0xe8/0x318 [ 3.627370] el1_dbg+0x18/0x78 [ 3.634037] __vunmap+0x1b8/0x220 [ 3.648747] vunmap+0x6c/0xc0 [ 3.653864] __iounmap+0x44/0x58 [ 3.659771] devm_ioremap_release+0x34/0x68 [ 3.672983] release_nodes+0x404/0x880 [ 3.683543] devres_release_all+0x6c/0xe8 [ 3.695692] driver_probe_device+0x250/0x828 [ 3.706187] __driver_attach+0x190/0x210 [ 3.717645] bus_for_each_dev+0x14c/0x1f0 [ 3.728633] driver_attach+0x48/0x78 [ 3.740249] bus_add_driver+0x26c/0x5b8 [ 3.752248] drive...

...+0x4c/0x88 [ 3.608400] brk_handler+0x11c/0x218 [ 3.613430] do_debug_exception+0xe8/0x318 [ 3.627370] el1_dbg+0x18/0x78 [ 3.634037] __vunmap+0x1b8/0x220 [ 3.648747] vunmap+0x6c/0xc0 [ 3.653864] __iounmap+0x44/0x58 [ 3.659771] devm_ioremap_release+0x34/0x68 [ 3.672983] release_nodes+0x404/0x880 [ 3.683543] devres_release_all+0x6c/0xe8 [ 3.695692] driver_probe_device+0x250/0x828 [ 3.706187] __driver_attach+0x190/0x210 [ 3.717645] bus_for_each_dev+0x14c/0x1f0 [ 3.728633] driver_attach+0x48/0x78 [ 3.740249] bus_add_driver+0x26c/0x5b8 [ 3.752248] drive...

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit

N.B. Sorry for the duplicate. This patch series were resent as the original one was rejected by the vger.kernel.org list server due to long header. There is no change in content. v7->v8: - Remove one unneeded atomic operation from the slowpath, thus improving performance. - Simplify some of the codes and add more comments. - Test for X86_FEATURE_HYPERVISOR CPU feature bit

...ely(!(prev_qcode & _QSPINLOCK_LOCKED))) { + if (prev_qcode == 0) { + /* + * Got the lock since it is at the head of the queue + * Now try to atomically clear the queue code. + */ + if (atomic_cmpxchg(&lock->qlcode, my_qcode, + _QSPINLOCK_LOCKED) == my_qcode) + goto release_node; + /* + * The cmpxchg fails only if one or more tasks + * are added to the queue. In this case, we need to + * notify the next one to be the head of the queue. + */ + goto notify_next; + } + /* + * Accidentally steal the lock, release the lock and + * let the queue head get it...