thr3ads.net - llvm dev - [llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity [Nov 2017]

If this information is useful, please help other people find it:
Share via:

Walter Lee via llvm-dev

2017-Nov-08 00:50 UTC

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

I've finished my initial set of patches to make 32-byte shadow
granularity work on x86.  Here is a summary of the changes from last
week:

- As discussed, I added a full redzone after every stack variable.

- We discussed adding a -fsanitize-address-granularity=N flag, but I
  found the following existing flag has been sufficient for my
  purposes: -asan-mapping-scale N.  If anyone thinks I should add the
  flag anyways, possibly replacing the latter, please let me know.

- I've modified the build so that we always run the ASan
  instrumentation test suite for shadow scale values of 3 and 5.

- I've gone through the asan test suites to make them run cleanly for
  both shadow scale=3 and shadow scale=5.  Here are the tests I have
  disabled, grouped by categories:

  - Instrumentation/AddressSanitizer tests.  Most tests work out of
    the box, and I ported some basic tests, leaving the following:

    llvm/test/Instrumentation/AddressSanitizer/lifetime-throw.ll
    llvm/test/Instrumentation/AddressSanitizer/lifetime-uar-uas.ll
    llvm/test/Instrumentation/AddressSanitizer/lifetime.ll
   
llvm/test/Instrumentation/AddressSanitizer/stack-poisoning-and-lifetime-be.ll
    llvm/test/Instrumentation/AddressSanitizer/stack-poisoning-and-lifetime.ll
    llvm/test/Instrumentation/AddressSanitizer/stack-poisoning.ll
    llvm/test/Instrumentation/AddressSanitizer/stack_layout.ll

  - Asm instrumentation not supported:
    Instrumentation/AddressSanitizer/X86/*
    compiler-rt/lib/asan/tests/asan_asm_test.cc
    compiler-rt/test/asan/TestCases/Linux/asan-asm-stacktrace-test.cc

  - Prelinking not supported:
    compiler-rt/test/asan/TestCases/Linux/asan_prelink_test.cc

  - Intra-object padding not supported:
    compiler-rt/test/asan/TestCases/intra-object-overflow.cc

  - Calls __asan_poison_memory_region in middle of shadow byte:
    compiler-rt/lib/asan/tests/asan_interface_test.cc
      SimplePoisonMemoryRegionTest
      OverlappingPoisonMemoryRegionTest
      PoisoningStressTest
    compiler-rt/test/asan/TestCases/small_memcpy_test.cc
    compiler-rt/test/asan/TestCases/strtol_strict.c
    compiler-rt/test/asan/TestCases/strtoll_strict.c

  - Has hardwired memory map:
    compiler-rt/test/asan/TestCases/Linux/cuda_test.cc
    compiler-rt/test/asan/TestCases/Linux/kernel-area.cc
    compiler-rt/test/asan/TestCases/Linux/nohugepage_test.cc

  - Miscellaneous:
    compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc
      This test assumes amount of available memory.
    compiler-rt/test/asan/TestCases/stack-buffer-overflow-with-position.cc
      This fails because we don't have enough redzones to disambiguate
      overflow of one stack object vs underflow of the next stack
      object.

- Here is the full list of revisions.  I'll add review requests
  shortly.

[asan] Add CMake hook to override default shadow scale
       https://reviews.llvm.org/D39469
[asan] Fix size/alignment issues with non-default shadow scale
       https://reviews.llvm.org/D39470
[asan] Fix small X86_64 ShadowOffset for non-default shadow scale
       https://reviews.llvm.org/D39471
[asan] Ensure that the minimum redzone is at least SHADOW_GRANULARITY
       https://reviews.llvm.org/D39472
[sanitizers] Increase alignment of low level allocator
       https://reviews.llvm.org/D39473
[asan] Avoid assert failure for non-default shadow scale
       https://reviews.llvm.org/D39474
[asan] Add full redzone after every stack variable
       https://reviews.llvm.org/D39475
[gtest] Increase stack size for child process in EXPECT_DEATH implementation
       https://reviews.llvm.org/D39771
[asan] Add lit feature for custom shadow scale
       https://reviews.llvm.org/D39772
[asan] Port tests to shadow scale of 5
       https://reviews.llvm.org/D39773
[asan] Disable unsupported tests for custom shadow scale
       https://reviews.llvm.org/D39774
[asan] Test ASan instrumentation for shadow scale value of 5
       https://reviews.llvm.org/D39775

Thanks,

Walter

Evgenii Stepanov via llvm-dev

2017-Nov-08 21:10 UTC

head link

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

On Tue, Nov 7, 2017 at 4:50 PM, Walter Lee <waltl at google.com>
wrote:> I've finished my initial set of patches to make 32-byte shadow
> granularity work on x86.  Here is a summary of the changes from last
> week:
>
> - As discussed, I added a full redzone after every stack variable.
>
> - We discussed adding a -fsanitize-address-granularity=N flag, but I
>   found the following existing flag has been sufficient for my
>   purposes: -asan-mapping-scale N.  If anyone thinks I should add the
>   flag anyways, possibly replacing the latter, please let me know.
IMO a clang flag would mean that non-standard setting for address
granularity is a supported configuration. That would require the
driver to link correct runtime library, which means we either build
two copies of libclang_rt.asan for each platform and somehow encode
the granularity value in the library name; or export that value from
instrumented code through a global, but then it stops being a
compile-time constant, and that may have effect on performance. Either
way would be an ABI break.

I think what you really want is to test shadow scale = 5 on
linux/x86_64 as a substitute for testing on the real hardware. For
that, a cmake variable in compiler-rt and an LLVM flag
(asan-mapping-scale) is more than enough.
>
> - I've modified the build so that we always run the ASan
>   instrumentation test suite for shadow scale values of 3 and 5.
>
> - I've gone through the asan test suites to make them run cleanly for
>   both shadow scale=3 and shadow scale=5.  Here are the tests I have
>   disabled, grouped by categories:
>
>   - Instrumentation/AddressSanitizer tests.  Most tests work out of
>     the box, and I ported some basic tests, leaving the following:
>
>     llvm/test/Instrumentation/AddressSanitizer/lifetime-throw.ll
>     llvm/test/Instrumentation/AddressSanitizer/lifetime-uar-uas.ll
>     llvm/test/Instrumentation/AddressSanitizer/lifetime.ll
>    
llvm/test/Instrumentation/AddressSanitizer/stack-poisoning-and-lifetime-be.ll
>    
llvm/test/Instrumentation/AddressSanitizer/stack-poisoning-and-lifetime.ll
>     llvm/test/Instrumentation/AddressSanitizer/stack-poisoning.ll
>     llvm/test/Instrumentation/AddressSanitizer/stack_layout.ll
>
>   - Asm instrumentation not supported:
>     Instrumentation/AddressSanitizer/X86/*
>     compiler-rt/lib/asan/tests/asan_asm_test.cc
>     compiler-rt/test/asan/TestCases/Linux/asan-asm-stacktrace-test.cc
>
>   - Prelinking not supported:
>     compiler-rt/test/asan/TestCases/Linux/asan_prelink_test.cc
>
>   - Intra-object padding not supported:
>     compiler-rt/test/asan/TestCases/intra-object-overflow.cc
>
>   - Calls __asan_poison_memory_region in middle of shadow byte:
>     compiler-rt/lib/asan/tests/asan_interface_test.cc
>       SimplePoisonMemoryRegionTest
>       OverlappingPoisonMemoryRegionTest
>       PoisoningStressTest
>     compiler-rt/test/asan/TestCases/small_memcpy_test.cc
>     compiler-rt/test/asan/TestCases/strtol_strict.c
>     compiler-rt/test/asan/TestCases/strtoll_strict.c
>
>   - Has hardwired memory map:
>     compiler-rt/test/asan/TestCases/Linux/cuda_test.cc
>     compiler-rt/test/asan/TestCases/Linux/kernel-area.cc
>     compiler-rt/test/asan/TestCases/Linux/nohugepage_test.cc
>
>   - Miscellaneous:
>     compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc
>       This test assumes amount of available memory.
>     compiler-rt/test/asan/TestCases/stack-buffer-overflow-with-position.cc
>       This fails because we don't have enough redzones to disambiguate
>       overflow of one stack object vs underflow of the next stack
>       object.
>
> - Here is the full list of revisions.  I'll add review requests
>   shortly.
>
> [asan] Add CMake hook to override default shadow scale
>        https://reviews.llvm.org/D39469
> [asan] Fix size/alignment issues with non-default shadow scale
>        https://reviews.llvm.org/D39470
> [asan] Fix small X86_64 ShadowOffset for non-default shadow scale
>        https://reviews.llvm.org/D39471
> [asan] Ensure that the minimum redzone is at least SHADOW_GRANULARITY
>        https://reviews.llvm.org/D39472
> [sanitizers] Increase alignment of low level allocator
>        https://reviews.llvm.org/D39473
> [asan] Avoid assert failure for non-default shadow scale
>        https://reviews.llvm.org/D39474
> [asan] Add full redzone after every stack variable
>        https://reviews.llvm.org/D39475
> [gtest] Increase stack size for child process in EXPECT_DEATH
implementation
>        https://reviews.llvm.org/D39771
> [asan] Add lit feature for custom shadow scale
>        https://reviews.llvm.org/D39772
> [asan] Port tests to shadow scale of 5
>        https://reviews.llvm.org/D39773
> [asan] Disable unsupported tests for custom shadow scale
>        https://reviews.llvm.org/D39774
> [asan] Test ASan instrumentation for shadow scale value of 5
>        https://reviews.llvm.org/D39775
>
> Thanks,
>
> Walter

Walter Lee via llvm-dev

2017-Nov-10 17:34 UTC

head link

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

On Wed, Nov 8, 2017 at 4:10 PM, Evgenii Stepanov <eugenis at google.com>
wrote:> I think what you really want is to test shadow scale = 5 on
> linux/x86_64 as a substitute for testing on the real hardware. For
> that, a cmake variable in compiler-rt and an LLVM flag
> (asan-mapping-scale) is more than enough.
Thanks for the feedback.  I've made the changes accordingly and
uploaded them for reviews.

Walter
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20171110/684692f5/attachment.html>

Walter Lee via llvm-dev

2017-Nov-21 01:30 UTC

head link

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

On Tue, Nov 7, 2017 at 7:50 PM Walter Lee <waltl at google.com>
wrote:>
> I've finished my initial set of patches to make 32-byte shadow
> granularity work on x86.  Here is a summary of the changes from last
> week:
Thanks everyone for the feedback and reviews (especially Vitaly who did
the bulk of it).  The patch series have been committed.

Walter

llvm dev - Nov 2017 - [RFC] ASan: patches to support 32-byte shadow granularity

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity

[llvm-dev] [RFC] ASan: patches to support 32-byte shadow granularity