search for: __read_once

The meat and potatoes of READ_ONCE() is defined by the __READ_ONCE() macro, which uses a volatile casts in an attempt to avoid tearing of byte, halfword, word and double-word accesses. Allow this to be overridden by the architecture code in the case that things like memory barriers are also required. Acked-by: Paul E. McKenney <paulmck at kernel.org> Signed...

...t provides the + * required ordering. + */ +#ifndef __ASM_GENERIC_RWONCE_H +#define __ASM_GENERIC_RWONCE_H + +#ifndef __ASSEMBLY__ + +#include <linux/compiler_types.h> +#include <linux/kasan-checks.h> +#include <linux/kcsan-checks.h> + +#include <asm/barrier.h> + +/* + * Use __READ_ONCE() instead of READ_ONCE() if you do not require any + * atomicity or dependency ordering guarantees. Note that this may result + * in tears! + */ +#define __READ_ONCE(x) (*(const volatile __unqual_scalar_typeof(x) *)&(x)) + +#define __READ_ONCE_SCALAR(x) \ +({ \ + __unqual_scalar_ty...

...t provides the + * required ordering. + */ +#ifndef __ASM_GENERIC_RWONCE_H +#define __ASM_GENERIC_RWONCE_H + +#ifndef __ASSEMBLY__ + +#include <linux/compiler_types.h> +#include <linux/kasan-checks.h> +#include <linux/kcsan-checks.h> + +#include <asm/barrier.h> + +/* + * Use __READ_ONCE() instead of READ_ONCE() if you do not require any + * atomicity or dependency ordering guarantees. Note that this may result + * in tears! + */ +#define __READ_ONCE(x) (*(const volatile __unqual_scalar_typeof(x) *)&(x)) + +#define __READ_ONCE_SCALAR(x) \ +({ \ + __unqual_scalar_ty...

...NERIC_RWONCE_H > +#define __ASM_GENERIC_RWONCE_H > + > +#ifndef __ASSEMBLY__ > + > +#include <linux/compiler_types.h> > +#include <linux/kasan-checks.h> > +#include <linux/kcsan-checks.h> > + > +#include <asm/barrier.h> > + > +/* > + * Use __READ_ONCE() instead of READ_ONCE() if you do not require any > + * atomicity or dependency ordering guarantees. Note that this may result > + * in tears! > + */ > +#define __READ_ONCE(x) (*(const volatile __unqual_scalar_typeof(x) *)&(x)) > + > +#define __READ_ONCE_SCALAR(x) \ >...

On Tue, Jun 30, 2020 at 06:37:20PM +0100, Will Deacon wrote: > Rather then relying on the core code to use smp_read_barrier_depends() > as part of the READ_ONCE() definition, instead override __READ_ONCE() > in the Alpha code so that it is treated the same way as > smp_load_acquire(). > > Acked-by: Paul E. McKenney <paulmck at kernel.org> > Signed-off-by: Will Deacon <will at kernel.org> > --- > arch/alpha/include/asm/barrier.h | 61 ++++---------------------------...

On Tue, Jun 30, 2020 at 06:37:20PM +0100, Will Deacon wrote: > Rather then relying on the core code to use smp_read_barrier_depends() > as part of the READ_ONCE() definition, instead override __READ_ONCE() > in the Alpha code so that it is treated the same way as > smp_load_acquire(). > > Acked-by: Paul E. McKenney <paulmck at kernel.org> > Signed-off-by: Will Deacon <will at kernel.org> > --- > arch/alpha/include/asm/barrier.h | 61 ++++---------------------------...

...provides the + * required ordering. + */ +#ifndef __ASM_GENERIC_RWONCE_H +#define __ASM_GENERIC_RWONCE_H + +#ifndef __ASSEMBLY__ + +#include <linux/compiler_attributes.h> +#include <linux/kasan-checks.h> + +#include <uapi/linux/types.h> + +#include <asm/barrier.h> + +#define __READ_ONCE_SIZE \ +({ \ + switch (size) { \ + case 1: *(__u8 *)res = *(volatile __u8 *)p; break; \ + case 2: *(__u16 *)res = *(volatile __u16 *)p; break; \ + case 4: *(__u32 *)res = *(volatile __u32 *)p; break; \ + case 8: *(__u64 *)res = *(volatile __u64 *)p; break; \ + default:...

...posted here: v1: https://lore.kernel.org/lkml/20191108170120.22331-1-will at kernel.org/ v2: https://lore.kernel.org/r/20200630173734.14057-1-will at kernel.org Changes since v2 include: * Actually add the barrier in READ_ONCE() for Alpha! * Implement Alpha's smp_load_acquire() using __READ_ONCE(), rather than the other way around. * Further untangling of header files * Use CONFIG_LTO instead of CONFIG_CLANG_LTO I have booted this on arm64, and build-tested as follows: - arm64 allnoconfig, defconfig (also bisected) and allmodconfig - arm32 allnoconfig, defconfig and allmodcon...

...+#include <asm/barrier.h> > > + > > +/* > > + * Alpha is apparently daft enough to reorder address-dependent loads > > + * on some CPU implementations. Knock some common sense into it with > > + * a memory barrier in READ_ONCE(). > > + */ > > +#define __READ_ONCE(x) __smp_load_acquire(&(x)) > > As above, I don't see a memory barrier implied here, so this doesn't > look quite right. You're right, and Peter spotted the same thing off-list. I've reworked locally so that the mb() ends up in __READ_ONCE() and __smp_load_acquire() c...

...ill at kernel.org/ I ended up parking the series while the READ_ONCE() implementation was being overhauled, but with that merged during the recent merge window and LTO patches being posted again [1], it was time for a refresh. The patches allow architectures to provide their own implementation of __READ_ONCE(). This serves two main purposes: 1. It finally allows us to remove [smp_]read_barrier_depends() from the Linux memory model and make it an implementation detail of the Alpha back-end. 2. It allows arm64 to upgrade __READ_ONCE() to have RCpc acquire semantics when compiling wit...

...ill at kernel.org/ I ended up parking the series while the READ_ONCE() implementation was being overhauled, but with that merged during the recent merge window and LTO patches being posted again [1], it was time for a refresh. The patches allow architectures to provide their own implementation of __READ_ONCE(). This serves two main purposes: 1. It finally allows us to remove [smp_]read_barrier_depends() from the Linux memory model and make it an implementation detail of the Alpha back-end. 2. It allows arm64 to upgrade __READ_ONCE() to have RCpc acquire semantics when compiling wit...

On Tue, Jun 30, 2020 at 7:39 PM Will Deacon <will at kernel.org> wrote: > +#define __READ_ONCE(x) \ > +({ \ > + int atomic = 1; \ > + union { __unqual_scalar_typeof(x) __val; char __c[1]; } __u; \ > +...

On Tue, Jun 30, 2020 at 7:39 PM Will Deacon <will at kernel.org> wrote: > +#define __READ_ONCE(x) \ > +({ \ > + int atomic = 1; \ > + union { __unqual_scalar_typeof(x) __val; char __c[1]; } __u; \ > +...

Rather then relying on the core code to use smp_read_barrier_depends() as part of the READ_ONCE() definition, instead override __READ_ONCE() in the Alpha code so that it is treated the same way as smp_load_acquire(). Acked-by: Paul E. McKenney <paulmck at kernel.org> Signed-off-by: Will Deacon <will at kernel.org> --- arch/alpha/include/asm/barrier.h | 61 ++++---------------------------- arch/alpha/include/asm/rwonce.h...

...: Joe Perches <joe at perches.com> Cc: Boqun Feng <boqun.feng at gmail.com> Cc: linux-alpha at vger.kernel.org Cc: virtualization at lists.linux-foundation.org --->8 Will Deacon (13): compiler.h: Split {READ,WRITE}_ONCE definitions out into rwonce.h READ_ONCE: Undefine internal __READ_ONCE_SIZE macro after use READ_ONCE: Allow __READ_ONCE_SIZE cases to be overridden by the architecture vhost: Remove redundant use of read_barrier_depends() barrier alpha: Override READ_ONCE() with barriered implementation READ_ONCE: Remove smp_read_barrier_depends() invocation alpha: Repl...

On Tue, Jun 30, 2020 at 09:25:03PM +0200, Arnd Bergmann wrote: > On Tue, Jun 30, 2020 at 7:39 PM Will Deacon <will at kernel.org> wrote: > > +#define __READ_ONCE(x) \ > > +({ \ > > + int atomic = 1; \ > > + union { __unqual_scalar_typeof(x) __val; char __c[1]; } __u;...

...\ + "ldar" #sfx "\t" #regs, \ + ".arch_extension rcpc\n" \ + "ldapr" #sfx "\t" #regs, \ + ARM64_HAS_LDAPR) +#else +#define __LOAD_RCPC(sfx, regs...) "ldar" #sfx "\t" #regs +#endif /* CONFIG_AS_HAS_LDAPR */ + +#define __READ_ONCE(x) \ +({ \ + int atomic = 1; \ + union { __unqual_scalar_typeof(x) __val; char __c[1]; } __u; \ + typeof(&(x)) __x = &(x); \ + switch (sizeof(x)) { \ + case 1: \ + asm volatile(__LOAD_RCPC(b, %w0, %1) \ + : "=r" (*(__u8 *)__u.__c) \ + :...

...\ + "ldar" #sfx "\t" #regs, \ + ".arch_extension rcpc\n" \ + "ldapr" #sfx "\t" #regs, \ + ARM64_HAS_LDAPR) +#else +#define __LOAD_RCPC(sfx, regs...) "ldar" #sfx "\t" #regs +#endif /* CONFIG_AS_HAS_LDAPR */ + +#define __READ_ONCE(x) \ +({ \ + typeof(&(x)) __x = &(x); \ + int atomic = 1; \ + union { __unqual_scalar_typeof(*__x) __val; char __c[1]; } __u; \ + switch (sizeof(x)) { \ + case 1: \ + asm volatile(__LOAD_RCPC(b, %w0, %1) \ + : "=r" (*(__u8 *)__u.__c) \ +...

...neric/rwonce.h b/include/asm-generic/rwonce.h index cc810f1f18ca..cd0302746fb4 100644 --- a/include/asm-generic/rwonce.h +++ b/include/asm-generic/rwonce.h @@ -26,8 +26,6 @@ #include <linux/kasan-checks.h> #include <linux/kcsan-checks.h> -#include <asm/barrier.h> - /* * Use __READ_ONCE() instead of READ_ONCE() if you do not require any * atomicity. Note that this may result in tears! diff --git a/include/linux/nospec.h b/include/linux/nospec.h index 0c5ef54fd416..c1e79f72cd89 100644 --- a/include/linux/nospec.h +++ b/include/linux/nospec.h @@ -5,6 +5,8 @@ #ifndef _LINUX_NOSP...

...dex cc810f1f18ca..cd0302746fb4 100644 > --- a/include/asm-generic/rwonce.h > +++ b/include/asm-generic/rwonce.h > @@ -26,8 +26,6 @@ > #include <linux/kasan-checks.h> > #include <linux/kcsan-checks.h> > > -#include <asm/barrier.h> > - > /* > * Use __READ_ONCE() instead of READ_ONCE() if you do not require any > * atomicity. Note that this may result in tears! > diff --git a/include/linux/nospec.h b/include/linux/nospec.h > index 0c5ef54fd416..c1e79f72cd89 100644 > --- a/include/linux/nospec.h > +++ b/include/linux/nospec.h > @@ -5,6...