thr3ads.net - llvm dev - [llvm-dev] ASan port for Myriad RTEMS [May 2018]

If this information is useful, please help other people find it:
Share via:

Walter Lee via llvm-dev

2018-May-04 21:00 UTC

[llvm-dev] ASan port for Myriad RTEMS

I have ported ASan in LLVM to Myriad RTEMS, and I would like to
upstream the port.  Below is the design doc.  Feedback welcome.

https://docs.google.com/document/d/1oxmk0xUojybDaQDAuTEVpHVMi5xQX74cJPyMJbaSaRM

The port is expected to work with modified versions of RTEMS and
newlib.  I have a git repo with changes to those projects, that I can
make available if there is interest.

Here is the patch series:
https://reviews.llvm.org/D46451 [asan] Add instrumentation support for
Myriad
https://reviews.llvm.org/D46452 [sanitizer] Don't add --export-dynamic for
Myriad
https://reviews.llvm.org/D46453 [sanitizer] Add definitions for Myriad
RTEMS platform
https://reviews.llvm.org/D46454 [sanitizer] Trivial portion of the port to
Myriad RTEMS
https://reviews.llvm.org/D46456 [asan] Add support for Myriad RTEMS memory
map
https://reviews.llvm.org/D46457 [asan] Add a magic shadow value for shadw
gap
https://reviews.llvm.org/D46459 [asan] On RTEMS, checks for asan_inited
before entering ASan run-time
https://reviews.llvm.org/D46461 [asan] Set flags appropriately for RTEMS
https://reviews.llvm.org/D46462 [sanitizer] Allow Fuchsia symbolizer to be
reused by Myriad RTEMS
https://reviews.llvm.org/D46463 [sanitizer] On RTEMS, avoid recursion when
reporting Mmap failure
https://reviews.llvm.org/D46465 [asan] Port asan_malloc_linux.cc to RTEMS
https://reviews.llvm.org/D46466 [asan] Add AsanThread::Restart() to support
thread restart
https://reviews.llvm.org/D46467 [asan] Add argument to allow fake stack to
be initialized during thread init
https://reviews.llvm.org/D46468 [asan] Add target-specific files for Myriad
RTEMS port
https://reviews.llvm.org/D46469 [sanitizer] Port fast stack unwinder to
sparcv8

Thanks,

Walter

Jake Ehrlich via llvm-dev

2018-May-04 21:36 UTC

head link

[llvm-dev] ASan port for Myriad RTEMS

Hey,

I work on fuchsia symbolizer stuff. I don't know if you guys already have
an external symbolizer but I'm working on making one right now and I plan
on making one backed by LLVM that can be run host-side or target-side. I'd
like to contribute that back to llvm ideally. What do you guys have so far?
I have a prototype in golang that just spins up an instance of
llvm-symbolizer currently. It isn't fully functional at the moment and it
has some fuchsia specific stuff as well:
https://fuchsia.googlesource.com/tools/+/master/symbolize/ but it should
usable soon. The full spec we'd like to implement can be read here:
https://fuchsia.googlesource.com/zircon/+/master/docs/symbolizer_markup.md

Best,
Jake

On Fri, May 4, 2018 at 2:00 PM Walter Lee via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I have ported ASan in LLVM to Myriad RTEMS, and I would like to
> upstream the port.  Below is the design doc.  Feedback welcome.
>
>
>
https://docs.google.com/document/d/1oxmk0xUojybDaQDAuTEVpHVMi5xQX74cJPyMJbaSaRM
>
> The port is expected to work with modified versions of RTEMS and
> newlib.  I have a git repo with changes to those projects, that I can
> make available if there is interest.
>
> Here is the patch series:
> https://reviews.llvm.org/D46451 [asan] Add instrumentation support for
> Myriad
> https://reviews.llvm.org/D46452 [sanitizer] Don't add --export-dynamic
for
> Myriad
> https://reviews.llvm.org/D46453 [sanitizer] Add definitions for Myriad
> RTEMS platform
> https://reviews.llvm.org/D46454 [sanitizer] Trivial portion of the port to
> Myriad RTEMS
> https://reviews.llvm.org/D46456 [asan] Add support for Myriad RTEMS memory
> map
> https://reviews.llvm.org/D46457 [asan] Add a magic shadow value for shadw
> gap
> https://reviews.llvm.org/D46459 [asan] On RTEMS, checks for asan_inited
> before entering ASan run-time
> https://reviews.llvm.org/D46461 [asan] Set flags appropriately for RTEMS
> https://reviews.llvm.org/D46462 [sanitizer] Allow Fuchsia symbolizer to be
> reused by Myriad RTEMS
> https://reviews.llvm.org/D46463 [sanitizer] On RTEMS, avoid recursion when
> reporting Mmap failure
> https://reviews.llvm.org/D46465 [asan] Port asan_malloc_linux.cc to RTEMS
> https://reviews.llvm.org/D46466 [asan] Add AsanThread::Restart() to
> support
> thread restart
> https://reviews.llvm.org/D46467 [asan] Add argument to allow fake stack to
> be initialized during thread init
> https://reviews.llvm.org/D46468 [asan] Add target-specific files for
> Myriad
> RTEMS port
> https://reviews.llvm.org/D46469 [sanitizer] Port fast stack unwinder to
> sparcv8
>
> Thanks,
>
> Walter
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180504/0fc3101c/attachment.html>

Kostya Serebryany via llvm-dev

2018-May-04 22:10 UTC

head link

[llvm-dev] ASan port for Myriad RTEMS

Hi Walter,

I've done a first quick scan.
Overall looks reasonable, but I'd like to try reducing the number of newly
introduced platform-specific ifs.

Vitaly, please also take a look (once my initial comments are addressed).

One outstanding issue is your problem with initialization vs checking,
which requires you to insert so many ifs.
Is there any chance you can avoid this?
If you control the OS, surely you can do at least minimal initialization of
the shadow at a proper moment...

--kcc

On Fri, May 4, 2018 at 2:00 PM Walter Lee via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I have ported ASan in LLVM to Myriad RTEMS, and I would like to
> upstream the port.  Below is the design doc.  Feedback welcome.
>
>
>
https://docs.google.com/document/d/1oxmk0xUojybDaQDAuTEVpHVMi5xQX74cJPyMJbaSaRM
>
> The port is expected to work with modified versions of RTEMS and
> newlib.  I have a git repo with changes to those projects, that I can
> make available if there is interest.
>
> Here is the patch series:
> https://reviews.llvm.org/D46451 [asan] Add instrumentation support for
> Myriad
> https://reviews.llvm.org/D46452 [sanitizer] Don't add --export-dynamic
for
> Myriad
> https://reviews.llvm.org/D46453 [sanitizer] Add definitions for Myriad
> RTEMS platform
> https://reviews.llvm.org/D46454 [sanitizer] Trivial portion of the port to
> Myriad RTEMS
> https://reviews.llvm.org/D46456 [asan] Add support for Myriad RTEMS memory
> map
> https://reviews.llvm.org/D46457 [asan] Add a magic shadow value for shadw
> gap
> https://reviews.llvm.org/D46459 [asan] On RTEMS, checks for asan_inited
> before entering ASan run-time
> https://reviews.llvm.org/D46461 [asan] Set flags appropriately for RTEMS
> https://reviews.llvm.org/D46462 [sanitizer] Allow Fuchsia symbolizer to be
> reused by Myriad RTEMS
> https://reviews.llvm.org/D46463 [sanitizer] On RTEMS, avoid recursion when
> reporting Mmap failure
> https://reviews.llvm.org/D46465 [asan] Port asan_malloc_linux.cc to RTEMS
> https://reviews.llvm.org/D46466 [asan] Add AsanThread::Restart() to
> support
> thread restart
> https://reviews.llvm.org/D46467 [asan] Add argument to allow fake stack to
> be initialized during thread init
> https://reviews.llvm.org/D46468 [asan] Add target-specific files for
> Myriad
> RTEMS port
> https://reviews.llvm.org/D46469 [sanitizer] Port fast stack unwinder to
> sparcv8
>
> Thanks,
>
> Walter
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180504/1be7e050/attachment.html>

Kostya Serebryany via llvm-dev

2018-May-04 22:20 UTC

head link

[llvm-dev] ASan port for Myriad RTEMS

On RAM...

You chose the 32-byte shadow granularity to reduce the RAM overhead,
but I am afraid this will actually increase it due to extra alignment
requirements,
especially if an average allocation on your typical application is small.

The pointers are 32-bit, right?
Given how RAM-constrained your environment is, maybe you should consider
something more like HWASAN instead of ASAN.
https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html
But you may not have enough address bits. :(

--kcc



On Fri, May 4, 2018 at 3:10 PM Kostya Serebryany <kcc at google.com>
wrote:
> Hi Walter,
>
> I've done a first quick scan.
> Overall looks reasonable, but I'd like to try reducing the number of
newly
> introduced platform-specific ifs.
>
> Vitaly, please also take a look (once my initial comments are addressed).
>
> One outstanding issue is your problem with initialization vs checking,
> which requires you to insert so many ifs.
> Is there any chance you can avoid this?
> If you control the OS, surely you can do at least minimal initialization
> of the shadow at a proper moment...
>
> --kcc
>
> On Fri, May 4, 2018 at 2:00 PM Walter Lee via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> I have ported ASan in LLVM to Myriad RTEMS, and I would like to
>> upstream the port.  Below is the design doc.  Feedback welcome.
>>
>>
>>
https://docs.google.com/document/d/1oxmk0xUojybDaQDAuTEVpHVMi5xQX74cJPyMJbaSaRM
>>
>> The port is expected to work with modified versions of RTEMS and
>> newlib.  I have a git repo with changes to those projects, that I can
>> make available if there is interest.
>>
>> Here is the patch series:
>> https://reviews.llvm.org/D46451 [asan] Add instrumentation support for
>> Myriad
>> https://reviews.llvm.org/D46452 [sanitizer] Don't add
--export-dynamic
>> for
>> Myriad
>> https://reviews.llvm.org/D46453 [sanitizer] Add definitions for Myriad
>> RTEMS platform
>> https://reviews.llvm.org/D46454 [sanitizer] Trivial portion of the port
>> to
>> Myriad RTEMS
>> https://reviews.llvm.org/D46456 [asan] Add support for Myriad RTEMS
>> memory
>> map
>> https://reviews.llvm.org/D46457 [asan] Add a magic shadow value for
shadw
>> gap
>> https://reviews.llvm.org/D46459 [asan] On RTEMS, checks for asan_inited
>> before entering ASan run-time
>> https://reviews.llvm.org/D46461 [asan] Set flags appropriately for
RTEMS
>> https://reviews.llvm.org/D46462 [sanitizer] Allow Fuchsia symbolizer to
>> be
>> reused by Myriad RTEMS
>> https://reviews.llvm.org/D46463 [sanitizer] On RTEMS, avoid recursion
>> when
>> reporting Mmap failure
>> https://reviews.llvm.org/D46465 [asan] Port asan_malloc_linux.cc to
RTEMS
>> https://reviews.llvm.org/D46466 [asan] Add AsanThread::Restart() to
>> support
>> thread restart
>> https://reviews.llvm.org/D46467 [asan] Add argument to allow fake stack
>> to
>> be initialized during thread init
>> https://reviews.llvm.org/D46468 [asan] Add target-specific files for
>> Myriad
>> RTEMS port
>> https://reviews.llvm.org/D46469 [sanitizer] Port fast stack unwinder to
>> sparcv8
>>
>> Thanks,
>>
>> Walter
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180504/b1384239/attachment.html>

Walter Lee via llvm-dev

2018-May-05 01:16 UTC

head link

[llvm-dev] ASan port for Myriad RTEMS

Hi Jake.  Thanks for the info.  Where can I keep up to date on the
symbolizer status?

Our symbolizer is provided by the Myriad vendor and integrated into its
host test environment.  It doesn't do much: just look for PC string
patterns and symbolize them using addr2line.

Thanks,

Walter

On Fri, May 4, 2018 at 5:36 PM Jake Ehrlich <jakehehrlich at google.com>
wrote:
> Hey,
> I work on fuchsia symbolizer stuff. I don't know if you guys already
havean external symbolizer but I'm working on making one right now and I plan
on making one backed by LLVM that can be run host-side or target-side. I'd
like to contribute that back to llvm ideally. What do you guys have so far?
I have a prototype in golang that just spins up an instance of
llvm-symbolizer currently. It isn't fully functional at the moment and it
has some fuchsia specific stuff as well:
https://fuchsia.googlesource.com/tools/+/master/symbolize/ but it should
usable soon. The full spec we'd like to implement can be read here:
https://fuchsia.googlesource.com/zircon/+/master/docs/symbolizer_markup.md
> Best,
> Jake
> On Fri, May 4, 2018 at 2:00 PM Walter Lee via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>> I have ported ASan in LLVM to Myriad RTEMS, and I would like to
>> upstream the port.  Below is the design doc.  Feedback welcome.

https://docs.google.com/document/d/1oxmk0xUojybDaQDAuTEVpHVMi5xQX74cJPyMJbaSaRM
>> The port is expected to work with modified versions of RTEMS and
>> newlib.  I have a git repo with changes to those projects, that I can
>> make available if there is interest.
>> Here is the patch series:
>> https://reviews.llvm.org/D46451 [asan] Add instrumentation support for
>> Myriad
>> https://reviews.llvm.org/D46452 [sanitizer] Don't add
--export-dynamic
for>> Myriad
>> https://reviews.llvm.org/D46453 [sanitizer] Add definitions for Myriad
>> RTEMS platform
>> https://reviews.llvm.org/D46454 [sanitizer] Trivial portion of the port
to>> Myriad RTEMS
>> https://reviews.llvm.org/D46456 [asan] Add support for Myriad RTEMS
memory>> map
>> https://reviews.llvm.org/D46457 [asan] Add a magic shadow value for
shadw
>> gap
>> https://reviews.llvm.org/D46459 [asan] On RTEMS, checks for asan_inited
>> before entering ASan run-time
>> https://reviews.llvm.org/D46461 [asan] Set flags appropriately for
RTEMS
>> https://reviews.llvm.org/D46462 [sanitizer] Allow Fuchsia symbolizer to
be>> reused by Myriad RTEMS
>> https://reviews.llvm.org/D46463 [sanitizer] On RTEMS, avoid recursion
when>> reporting Mmap failure
>> https://reviews.llvm.org/D46465 [asan] Port asan_malloc_linux.cc to
RTEMS
>> https://reviews.llvm.org/D46466 [asan] Add AsanThread::Restart() to
support>> thread restart
>> https://reviews.llvm.org/D46467 [asan] Add argument to allow fake stack
to>> be initialized during thread init
>> https://reviews.llvm.org/D46468 [asan] Add target-specific files for
Myriad>> RTEMS port
>> https://reviews.llvm.org/D46469 [sanitizer] Port fast stack unwinder to
>> sparcv8
>> Thanks,
>> Walter
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Walter Lee via llvm-dev

2018-May-05 01:29 UTC

head link

[llvm-dev] ASan port for Myriad RTEMS

Hi Kostya.  Thanks for the quick feedback.  I will work on addressing your
comments.

In regard to initialization checks, I can eliminate most of them by
initializing the shadow memory very early, but I still need to do something
in two places, __asan_handle_no_return and GetFakeStackFast.  Would it be
ok to have guards for those two places only?

Walter
On Fri, May 4, 2018 at 6:10 PM Kostya Serebryany <kcc at google.com>
wrote:
> Hi Walter,
> I've done a first quick scan.
> Overall looks reasonable, but I'd like to try reducing the number ofnewly introduced platform-specific ifs.
> Vitaly, please also take a look (once my initial comments are addressed).
> One outstanding issue is your problem with initialization vs checking,
which requires you to insert so many ifs.> Is there any chance you can avoid this?
> If you control the OS, surely you can do at least minimal initializationof the shadow at a proper moment...
> --kcc
> On Fri, May 4, 2018 at 2:00 PM Walter Lee via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>> I have ported ASan in LLVM to Myriad RTEMS, and I would like to
>> upstream the port.  Below is the design doc.  Feedback welcome.

https://docs.google.com/document/d/1oxmk0xUojybDaQDAuTEVpHVMi5xQX74cJPyMJbaSaRM
>> The port is expected to work with modified versions of RTEMS and
>> newlib.  I have a git repo with changes to those projects, that I can
>> make available if there is interest.
>> Here is the patch series:
>> https://reviews.llvm.org/D46451 [asan] Add instrumentation support for
>> Myriad
>> https://reviews.llvm.org/D46452 [sanitizer] Don't add
--export-dynamic
for>> Myriad
>> https://reviews.llvm.org/D46453 [sanitizer] Add definitions for Myriad
>> RTEMS platform
>> https://reviews.llvm.org/D46454 [sanitizer] Trivial portion of the port
to>> Myriad RTEMS
>> https://reviews.llvm.org/D46456 [asan] Add support for Myriad RTEMS
memory>> map
>> https://reviews.llvm.org/D46457 [asan] Add a magic shadow value for
shadw
>> gap
>> https://reviews.llvm.org/D46459 [asan] On RTEMS, checks for asan_inited
>> before entering ASan run-time
>> https://reviews.llvm.org/D46461 [asan] Set flags appropriately for
RTEMS
>> https://reviews.llvm.org/D46462 [sanitizer] Allow Fuchsia symbolizer to
be>> reused by Myriad RTEMS
>> https://reviews.llvm.org/D46463 [sanitizer] On RTEMS, avoid recursion
when>> reporting Mmap failure
>> https://reviews.llvm.org/D46465 [asan] Port asan_malloc_linux.cc to
RTEMS
>> https://reviews.llvm.org/D46466 [asan] Add AsanThread::Restart() to
support>> thread restart
>> https://reviews.llvm.org/D46467 [asan] Add argument to allow fake stack
to>> be initialized during thread init
>> https://reviews.llvm.org/D46468 [asan] Add target-specific files for
Myriad>> RTEMS port
>> https://reviews.llvm.org/D46469 [sanitizer] Port fast stack unwinder to
>> sparcv8
>> Thanks,
>> Walter
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Apparently Analagous Threads

Search for more possibly parallel threads

llvm dev - May 2018 - ASan port for Myriad RTEMS

[llvm-dev] ASan port for Myriad RTEMS

[llvm-dev] ASan port for Myriad RTEMS

[llvm-dev] ASan port for Myriad RTEMS

[llvm-dev] ASan port for Myriad RTEMS

[llvm-dev] ASan port for Myriad RTEMS

[llvm-dev] ASan port for Myriad RTEMS

Apparently Analagous Threads