search for: __builtin_memcpy

...se two macros will also work on aggregate data types like structs or + * unions. If the size of the accessed data type exceeds the word size of + * the machine (e.g., 32 bits or 64 bits) READ_ONCE() and WRITE_ONCE() will + * fall back to memcpy(). There's at least two memcpy()s: one for the + * __builtin_memcpy() and then one for the macro doing the copy of variable + * - '__u' allocated on the stack. + * + * Their two major use cases are: (1) Mediating communication between + * process-level code and irq/NMI handlers, all running on the same CPU, + * and (2) Ensuring that the compiler does not fo...

...Mark Rutland wrote: >> On Fri, Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote: > >>> What are use cases for such primitive that won't be OK with "read once >>> _and_ atomically"? >> >> I have none to hand. > > Whatever triggers the __builtin_memcpy() paths, and even the size==8 > paths on 32bit. > > You could put a WARN in there to easily find them. There were several cases that I found during writing the *ONCE stuff. For example there are some 32bit ppc variants with 64bit PTEs. Some for others (I think sparc). And the mm/ code is...

...Mark Rutland wrote: >> On Fri, Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote: > >>> What are use cases for such primitive that won't be OK with "read once >>> _and_ atomically"? >> >> I have none to hand. > > Whatever triggers the __builtin_memcpy() paths, and even the size==8 > paths on 32bit. > > You could put a WARN in there to easily find them. There were several cases that I found during writing the *ONCE stuff. For example there are some 32bit ppc variants with 64bit PTEs. Some for others (I think sparc). And the mm/ code is...

On Fri, Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote: > > READ/WRITE_ONCE imply atomicity. Even if their names don't spell it (a > function name doesn't have to spell all of its guarantees). Most of > the uses of READ/WRITE_ONCE will be broken if they are not atomic. In practice, this is certainly the assumption made by many/most users of the *_ONCE() accessors.

...a buffer that is clearly too small to be used in a meaningful way, as the 'sizeof(*args) + argc > sizeof(stack)' will always fail: In function 'memcpy', inlined from 'nvif_vmm_map' at drivers/gpu/drm/nouveau/nvif/vmm.c:55:2: include/linux/string.h:353:9: error: '__builtin_memcpy' offset 40 is out of the bounds [0, 16] of object 'stack' with type 'u8[16]' {aka 'unsigned char[16]'} [-Werror=array-bounds] return __builtin_memcpy(p, q, size); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/gpu/drm/nouveau/nvif/vmm.c: In function 'nvif_vmm_map...

...ed int *)p; break; \ + case 8: *(unsigned long long *)res = *(volatile unsigned long long *)p; break; \ + default: \ + barrier(); \ + __builtin_memcpy((void *)res, (const void *)p, size); \ + barrier(); \ + } \ +} + +static __always_inline void __write_once_size(volatile void *p, void *res, int size) +{ + switch (size...

...ing spilled to the stack in some cases, so I think it would still help to simplify it. We probably don't want the exact ACCESS_ONCE() implementation back that existed before, but rather something that implements the stricter READ_ONCE() and WRITE_ONCE(). I'd probably also want to avoid the __builtin_memcpy() exception for odd-sized accesses and instead have a separate way to do those. Arnd

...au/nvif/outp.c: In function 'nvif_outp_acquire_dp': ../include/linux/fortify-string.h:57:33: warning: array subscript 'unsigned char[16][0]' is partly outside array bounds of 'u8[15]' {aka 'unsigned char[15]'} [-Warray-bounds=] 57 | #define __underlying_memcpy __builtin_memcpy | ^ ... ../drivers/gpu/drm/nouveau/nvif/outp.c:140:9: note: in expansion of macro 'memcpy' 140 | memcpy(args.dp.dpcd, dpcd, sizeof(args.dp.dpcd)); | ^~~~~~ ../drivers/gpu/drm/nouveau/nvif/outp.c:130:49: note: object 'dpcd'...

...Unfortunately this is not always the case: https://godbolt.org/z/mHpAYe. In addition `-fno-builtin-memcpy` prevents the compiler from understanding that a piece of code has the memory copy semantic but does not prevent the compiler from generating calls to libc's `memcpy`, for instance: Using `__builtin_memcpy`: https://godbolt.org/z/O0sjIl Passing big structs by value: https://godbolt.org/z/4BUDc0 In both cases, the generated `@llvm.memcpy` IR intrinsic is lowered into a libc `memcpy` call. We would like to use `__builtin_memcpy` to communicate the semantic to the compiler but prevent it from generati...

On Fri, Nov 25, 2016 at 5:17 PM, Peter Zijlstra <peterz at infradead.org> wrote: >> > What are use cases for such primitive that won't be OK with "read once >> > _and_ atomically"? >> >> I have none to hand. > > Whatever triggers the __builtin_memcpy() paths, and even the size==8 > paths on 32bit. > > You could put a WARN in there to easily find them. > > The advantage of introducing the SINGLE_{LOAD,STORE}() helpers is that > they compiletime validate this the size is 'right' and can runtime check > alignment const...

On Fri, Nov 25, 2016 at 5:17 PM, Peter Zijlstra <peterz at infradead.org> wrote: >> > What are use cases for such primitive that won't be OK with "read once >> > _and_ atomically"? >> >> I have none to hand. > > Whatever triggers the __builtin_memcpy() paths, and even the size==8 > paths on 32bit. > > You could put a WARN in there to easily find them. > > The advantage of introducing the SINGLE_{LOAD,STORE}() helpers is that > they compiletime validate this the size is 'right' and can runtime check > alignment const...

Hi all, Although [smp_]read_barrier_depends() became part of READ_ONCE() in commit 76ebbe78f739 ("locking/barriers: Add implicit smp_read_barrier_depends() to READ_ONCE()"), it still limps on in the Linux memory model with the sinister hope of attracting innocent new users so that it becomes impossible to remove altogether. Let's strike before it's too late: there's only

...v 25, 2016 at 04:10:04PM +0000, Mark Rutland wrote: > On Fri, Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote: > > What are use cases for such primitive that won't be OK with "read once > > _and_ atomically"? > > I have none to hand. Whatever triggers the __builtin_memcpy() paths, and even the size==8 paths on 32bit. You could put a WARN in there to easily find them. The advantage of introducing the SINGLE_{LOAD,STORE}() helpers is that they compiletime validate this the size is 'right' and can runtime check alignment constraints. IE, they are strictly st...

...4:21:39PM +0100, Dmitry Vyukov wrote: >>> >>>>> What are use cases for such primitive that won't be OK with "read once >>>>> _and_ atomically"? >>>> >>>> I have none to hand. >>> >>> Whatever triggers the __builtin_memcpy() paths, and even the size==8 >>> paths on 32bit. >>> >>> You could put a WARN in there to easily find them. >> >> There were several cases that I found during writing the *ONCE stuff. >> For example there are some 32bit ppc variants with 64bit PTEs. Som...

..., Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote: > > > >>> What are use cases for such primitive that won't be OK with "read once > >>> _and_ atomically"? > >> > >> I have none to hand. > > > > Whatever triggers the __builtin_memcpy() paths, and even the size==8 > > paths on 32bit. > > > > You could put a WARN in there to easily find them. > > There were several cases that I found during writing the *ONCE stuff. > For example there are some 32bit ppc variants with 64bit PTEs. Some for > others (...

...case 1: *(volatile __u8 *)p = *(__u8 *)res; break; case 2: *(volatile __u16 *)p = *(__u16 *)res; break; case 4: *(volatile __u32 *)p = *(__u32 *)res; break; case 8: *(volatile __u64 *)p = *(__u64 *)res; break; default: barrier(); __builtin_memcpy((void *)p, (const void *)res, size); barrier(); } } #define WRITE_ONCE(x, val) \ ({ \ union { typeof(x) __val; char __c[1]; } __u = \ { .__val = (__force typeof(x)) (val) }; \ __write_onc...

..., Nov 25, 2016 at 04:21:39PM +0100, Dmitry Vyukov wrote: > > > >>> What are use cases for such primitive that won't be OK with "read once > >>> _and_ atomically"? > >> > >> I have none to hand. > > > > Whatever triggers the __builtin_memcpy() paths, and even the size==8 > > paths on 32bit. > > > > You could put a WARN in there to easily find them. > > There were several cases that I found during writing the *ONCE stuff. > For example there are some 32bit ppc variants with 64bit PTEs. Some for > others (...

On Fri, Nov 25, 2016 at 3:56 PM, Boqun Feng <boqun.feng at gmail.com> wrote: > On Fri, Nov 25, 2016 at 01:44:04PM +0100, Peter Zijlstra wrote: >> On Fri, Nov 25, 2016 at 01:40:44PM +0100, Peter Zijlstra wrote: >> > #define SINGLE_LOAD(x) \ >> > {( \ >> >

On Fri, Nov 25, 2016 at 3:56 PM, Boqun Feng <boqun.feng at gmail.com> wrote: > On Fri, Nov 25, 2016 at 01:44:04PM +0100, Peter Zijlstra wrote: >> On Fri, Nov 25, 2016 at 01:40:44PM +0100, Peter Zijlstra wrote: >> > #define SINGLE_LOAD(x) \ >> > {( \ >> >

On Tue, Dec 05, 2017 at 08:31:20PM +0200, Michael S. Tsirkin wrote: > Apropos, READ_ONCE is now asymmetrical with WRITE_ONCE. > > I can read a pointer with READ_ONCE and be sure the value > is sane, but only if I also remember to put in smp_wmb before > WRITE_ONCE. Otherwise the pointer is ok but no guarantees > about the data pointed to. That was already the case on