Evgenii Stepanov via llvm-dev
2015-Sep-25 17:38 UTC
[llvm-dev] Dynamic VMA in Sanitizers for AArch64
Jakub makes a good point, are you sure that there is no single shadow offset value that works for all VMA variants? What exactly breaks when 1<<36 is used on 42-bit VMA? On Fri, Sep 25, 2015 at 3:28 AM, Yury Gribov via llvm-dev <llvm-dev at lists.llvm.org> wrote:> On 09/25/2015 01:27 PM, Yury Gribov wrote: >> >> On 09/25/2015 11:53 AM, Jakub Jelinek via llvm-dev wrote: >>> >>> On Fri, Sep 25, 2015 at 01:19:48AM -0700, Renato Golin wrote: >>>> >>>> After long talks with lots of people, I think we have a winning >>>> strategy to deal with the variable nature of VMA address in AArch64. >>>> It seems that the best way forward is to try the dynamic calculation >>>> at runtime, evaluate the performance, and then, only if the hit is too >>>> great, think about compile-time alternatives. I'd like to know if >>>> everyone is in agreement, so we could get cracking. >>> >>> >>> You mean you want a dynamic shadow offset on aarch64 as opposed to >>> fixed kAArch64_ShadowOffset64 (1UL << 36) one? >>> I think this is completely unnecessary, all you need to change is >>> libsanitizer internals IMNSHO, all that is needed is to make runtime >>> decisions during libasan initialization on the memory layout, and >>> also (because 39 bit VMA is too slow), dynamic decision whether to use >>> 32-bit or 64-bit allocator. See >>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64435 >>> for details. >> >> >> Added kcc. >> >> FYI optional dynamic offset would also help in not-so-rare situations >> when ASan's shadow range is stolen by early constructors in unsanitized >> libraries. > > > And - we'll finally be able to run ASan under Valgrind) > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Jakub Jelinek via llvm-dev
2015-Sep-25 19:11 UTC
[llvm-dev] Dynamic VMA in Sanitizers for AArch64
On Fri, Sep 25, 2015 at 10:38:59AM -0700, Evgenii Stepanov wrote:> Jakub makes a good point, are you sure that there is no single shadow > offset value that works for all VMA variants? What exactly breaks when > 1<<36 is used on 42-bit VMA?Note, in our distros we are shipping 42-bit VMA and are using patch on top of vanilla libsanitizer (with the 1UL << 36 shadow offset) and I don't remember any bugs reported against this not working (and the testsuite works too). So, assuming 39-bit VMA works too, that would show that at least those two settings work, the question is if 48-bit VMA (or how many) works too, and if it does, the next thing is tweaking the library so that it can perhaps with some small but still acceptable performance hit decide between those at runtime (e.g. kAllocatorSpace/kAllocatorSize could be turned into non-const variables for aarch64, harder would be to add some allocator that at runtime picks if it uses 32-bit or 64-bit allocator. --- libsanitizer/asan/asan_allocator.h (revision 219833) +++ libsanitizer/asan/asan_allocator.h (working copy) @@ -100,6 +100,10 @@ # if defined(__powerpc64__) const uptr kAllocatorSpace = 0xa0000000000ULL; const uptr kAllocatorSize = 0x20000000000ULL; // 2T. +# elif defined(__aarch64__) +// Valid only for 42-bit VA +const uptr kAllocatorSpace = 0x10000000000ULL; +const uptr kAllocatorSize = 0x10000000000ULL; // 1T. # else const uptr kAllocatorSpace = 0x600000000000ULL; const uptr kAllocatorSize = 0x40000000000ULL; // 4T. --- libsanitizer/sanitizer_common/sanitizer_platform.h (revision 219833) +++ libsanitizer/sanitizer_common/sanitizer_platform.h (working copy) @@ -79,7 +79,7 @@ // For such platforms build this code with -DSANITIZER_CAN_USE_ALLOCATOR64=0 or // change the definition of SANITIZER_CAN_USE_ALLOCATOR64 here. #ifndef SANITIZER_CAN_USE_ALLOCATOR64 -# if defined(__aarch64__) || defined(__mips64) +# if defined(__mips64) # define SANITIZER_CAN_USE_ALLOCATOR64 0 # else # define SANITIZER_CAN_USE_ALLOCATOR64 (SANITIZER_WORDSIZE == 64) @@ -88,10 +88,10 @@ // The range of addresses which can be returned my mmap. // FIXME: this value should be different on different platforms, -// e.g. on AArch64 it is most likely (1ULL << 39). Larger values will still work +// e.g. on AArch64 it is most likely (1ULL << 42). Larger values will still work // but will consume more memory for TwoLevelByteMap. #if defined(__aarch64__) -# define SANITIZER_MMAP_RANGE_SIZE FIRST_32_SECOND_64(1ULL << 32, 1ULL << 39) +# define SANITIZER_MMAP_RANGE_SIZE FIRST_32_SECOND_64(1ULL << 32, 1ULL << 42) #else # define SANITIZER_MMAP_RANGE_SIZE FIRST_32_SECOND_64(1ULL << 32, 1ULL << 47) #endif Jakub
Renato Golin via llvm-dev
2015-Sep-29 13:38 UTC
[llvm-dev] Dynamic VMA in Sanitizers for AArch64
On 25 September 2015 at 20:11, Jakub Jelinek <jakub at redhat.com> wrote:> Note, in our distros we are shipping 42-bit VMA and are using patch on > top of vanilla libsanitizer (with the 1UL << 36 shadow offset) and I don't > remember any bugs reported against this not working (and the testsuite works > too). So, assuming 39-bit VMA works too, that would show that at least > those two settings work, the question is if 48-bit VMA (or how many) works > too, and if it does, the next thing is tweaking the library so that it can > perhaps with some small but still acceptable performance hit decide between > those at runtime (e.g. kAllocatorSpace/kAllocatorSize could be turned into > non-const variables for aarch64, harder would be to add some allocator that > at runtime picks if it uses 32-bit or 64-bit allocator.Hi Jakub, My assumption is based on what I understood from my talks with various people in sanitizers, libraries, GCC, LLVM and the kernel. Please, correct me if I'm wrong. IIUC, using larger VMAs on kernels with smaller VMAs work, but restrict the space that you can use as a shadow region, thus limiting the size of programs you can run with the sanitizers. Also, the higher you go, the higher is the levels of indirection you need to use memory, so using higher VMAs may be more of a performance hit than it should. This may not be such a big deal for 42 vs. 39 bits, but the new kernels will come with 48 bits, and there's talks to push it up to 56 bits in the near future. So, there are only three paths we can take: 1. Keep it constant, today at 42, and increase the constant slowly, as kernels start popping in with higher VMAs. This could slow down for large values of VMA and low values on kernel. 2. Create a compiler flag -mvma=NN. This would be as fast as native when chosen correctly, but could break if lower than the machine's VMA. 3. Make it dynamic, so that the VMA value doesn't matter. I really have no idea on what the impact of dynamic VMA will have on the sanitizers, nor I have on what it would be if we choose an arbitrarily large VMA value (say, 56), and run on the lowest VMA (39). We need to benchmark those things. My email was just a summary of my discussions and a look forward. We believe we can implement the dynamic VMA with very little impact, but before going there, we need to understand what's the impact of using higher than necessary VMA values. This is all part of the investigation process that we'll start now. All our changes until now will have no impact on the GCC port. If anything, it'll make your life easier, since now you don't need a patch to move it to 42-bits, it's just a compiler flag. That's why I wanted to involve you in this discussion from now on, since we'll be taking decisions that *will* affect you, and we need to be sure they're the right decisions on both static and dynamic cases. But we can't take any decision without hard data to go by, and that's why we'll be investigating the performance compromises of each model. cheers, --renato