similar to: [patch] btrfs: fix gfp flags masking

Nicholas Piggin <npiggin at gmail.com> writes: > Signed-off-by: Nicholas Piggin <npiggin at gmail.com> > --- > arch/powerpc/include/asm/paravirt.h | 28 ++++++++ > arch/powerpc/include/asm/qspinlock.h | 66 +++++++++++++++++++ > arch/powerpc/include/asm/qspinlock_paravirt.h | 7 ++ > arch/powerpc/platforms/pseries/Kconfig | 5 ++ >

Nicholas Piggin <npiggin at gmail.com> writes: > Signed-off-by: Nicholas Piggin <npiggin at gmail.com> > --- > arch/powerpc/include/asm/paravirt.h | 28 ++++++++ > arch/powerpc/include/asm/qspinlock.h | 66 +++++++++++++++++++ > arch/powerpc/include/asm/qspinlock_paravirt.h | 7 ++ > arch/powerpc/platforms/pseries/Kconfig | 5 ++ >

Nicholas Piggin <npiggin at gmail.com> writes: > These have shown significantly improved performance and fairness when > spinlock contention is moderate to high on very large systems. > > [ Numbers hopefully forthcoming after more testing, but initial > results look good ] Would be good to have something here, even if it's preliminary. > Thanks to the fast path,

v3 is updated to use __pv_queued_spin_unlock, noticed by Waiman (thank you). Thanks, Nick Nicholas Piggin (6): powerpc/powernv: must include hvcall.h to get PAPR defines powerpc/pseries: move some PAPR paravirt functions to their own file powerpc: move spinlock implementation to simple_spinlock powerpc/64s: implement queued spinlocks and rwlocks powerpc/pseries: implement paravirt

v3 is updated to use __pv_queued_spin_unlock, noticed by Waiman (thank you). Thanks, Nick Nicholas Piggin (6): powerpc/powernv: must include hvcall.h to get PAPR defines powerpc/pseries: move some PAPR paravirt functions to their own file powerpc: move spinlock implementation to simple_spinlock powerpc/64s: implement queued spinlocks and rwlocks powerpc/pseries: implement paravirt

Nick Piggin <npiggin@suse.de> wrote: > > Also, for non-wb memory. I don't think the Intel document referenced > says anything about this, but the AMD document says that loads can pass > loads (page 8, rule b). > > This is why our rmb() is still an lfence. BTW, Xen (in particular, the code in drivers/xen) uses mb/rmb/wmb instead of smp_mb/smp_rmb/smp_wmb when it

Nick Piggin <npiggin@suse.de> wrote: > > Also, for non-wb memory. I don't think the Intel document referenced > says anything about this, but the AMD document says that loads can pass > loads (page 8, rule b). > > This is why our rmb() is still an lfence. BTW, Xen (in particular, the code in drivers/xen) uses mb/rmb/wmb instead of smp_mb/smp_rmb/smp_wmb when it

On 7/3/20 3:35 AM, Nicholas Piggin wrote: > Signed-off-by: Nicholas Piggin <npiggin at gmail.com> > --- > arch/powerpc/include/asm/paravirt.h | 28 ++++++++++ > arch/powerpc/include/asm/qspinlock.h | 55 +++++++++++++++++++ > arch/powerpc/include/asm/qspinlock_paravirt.h | 5 ++ > arch/powerpc/platforms/pseries/Kconfig | 5 ++ >

v2 is updated to account for feedback from Will, Peter, and Waiman (thank you), and trims off a couple of RFC and unrelated patches. Thanks, Nick Nicholas Piggin (6): powerpc/powernv: must include hvcall.h to get PAPR defines powerpc/pseries: move some PAPR paravirt functions to their own file powerpc: move spinlock implementation to simple_spinlock powerpc/64s: implement queued

On Thu, Jul 02, 2020 at 08:25:43PM +1000, Nicholas Piggin wrote: > Excerpts from Will Deacon's message of July 2, 2020 6:02 pm: > > On Thu, Jul 02, 2020 at 05:48:36PM +1000, Nicholas Piggin wrote: > >> diff --git a/arch/powerpc/include/asm/qspinlock.h b/arch/powerpc/include/asm/qspinlock.h > >> new file mode 100644 > >> index 000000000000..f84da77b6bb7 >

On Thu, Jul 02, 2020 at 08:25:43PM +1000, Nicholas Piggin wrote: > Excerpts from Will Deacon's message of July 2, 2020 6:02 pm: > > On Thu, Jul 02, 2020 at 05:48:36PM +1000, Nicholas Piggin wrote: > >> diff --git a/arch/powerpc/include/asm/qspinlock.h b/arch/powerpc/include/asm/qspinlock.h > >> new file mode 100644 > >> index 000000000000..f84da77b6bb7 >

This series adds an option to use queued spinlocks for powerpc, and makes it the default for the Book3S-64 subarch. This effort starts with the generic code so it's very simple but still very performant. There are optimisations that can be made to slowpaths, but I think it's better to attack those incrementally if/when we find things, and try to add the improvements to generic code as

Updated with everybody's feedback (thanks all), and more performance results. What I've found is I might have been measuring the worst load point for the paravirt case, and by looking at a range of loads it's clear that queued spinlocks are overall better even on PV, doubly so when you look at the generally much improved worst case latencies. I have defaulted it to N even though

On 7/9/20 6:53 AM, Michael Ellerman wrote: > Nicholas Piggin <npiggin at gmail.com> writes: > >> Signed-off-by: Nicholas Piggin <npiggin at gmail.com> >> --- >> arch/powerpc/include/asm/paravirt.h | 28 ++++++++ >> arch/powerpc/include/asm/qspinlock.h | 66 +++++++++++++++++++ >> arch/powerpc/include/asm/qspinlock_paravirt.h |

On Tue, Oct 10, 2017 at 07:47:37PM +0900, Tetsuo Handa wrote: > In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to > serialize against fill_balloon(). But in fill_balloon(), > alloc_page(GFP_HIGHUSER[_MOVABLE] | __GFP_NOMEMALLOC | __GFP_NORETRY) is > called with vb->balloon_lock mutex held. Since GFP_HIGHUSER[_MOVABLE] > implies __GFP_DIRECT_RECLAIM |

On Tue, Oct 10, 2017 at 07:47:37PM +0900, Tetsuo Handa wrote: > In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to > serialize against fill_balloon(). But in fill_balloon(), > alloc_page(GFP_HIGHUSER[_MOVABLE] | __GFP_NOMEMALLOC | __GFP_NORETRY) is > called with vb->balloon_lock mutex held. Since GFP_HIGHUSER[_MOVABLE] > implies __GFP_DIRECT_RECLAIM |

There is no need for rmb(), this allows faster lwsync here. Signed-off-by: Nicholas Piggin <npiggin at gmail.com> --- arch/powerpc/lib/locks.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/lib/locks.c b/arch/powerpc/lib/locks.c index 6440d5943c00..47a530de733e 100644 --- a/arch/powerpc/lib/locks.c +++ b/arch/powerpc/lib/locks.c @@ -30,7 +30,7 @@ void

An include goes away in future patches which breaks compilation without this. Signed-off-by: Nicholas Piggin <npiggin at gmail.com> --- arch/powerpc/platforms/powernv/pci-ioda-tce.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/powerpc/platforms/powernv/pci-ioda-tce.c b/arch/powerpc/platforms/powernv/pci-ioda-tce.c index f923359d8afc..8eba6ece7808 100644 ---

This implements smp_cond_load_relaed with the slowpath busy loop using the preferred SMT priority pattern. Signed-off-by: Nicholas Piggin <npiggin at gmail.com> --- arch/powerpc/include/asm/barrier.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/powerpc/include/asm/barrier.h b/arch/powerpc/include/asm/barrier.h index 123adcefd40f..9b4671d38674 100644 ---

fill_balloon doing memory allocations under balloon_lock can cause a deadlock when leak_balloon is called from virtballoon_oom_notify and tries to take same lock. To fix, split page allocation and enqueue and do allocations outside the lock. Here's a detailed analysis of the deadlock by Tetsuo Handa: In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to serialize