llvm dev - Oct 2020 - [RFC] Implementing the sanitizer runtimes for heterogeneous devices

Hi all,

We are thinking of extending the LLVM sanitizers, starting with asan, to
heterogeneous situations
such as those found in OpenCL and HIP. We have already started thinking about
the way
instrumentation will look like for different address spaces and posted a small
RFC
(http://lists.llvm.org/pipermail/llvm-dev/2020-July/143310.html) about removing
the generated
inttoptr and ptrtoint pair from instrumented IR. We have started to look into
the runtime
implementation, and we will be happy to get feedback and suggestion from the
community so that we
can incorporate them in the design itself. We are looking at implementing a
sanitizer runtime that
would support heterogeneous devices such as AMD GPUs and we present some
directions that we can take
to make that happen:

    1. Add support for compilation of the compiler-rt sanitizer runtime by a
device compiler

       We propose to update the sanitizer parts of compiler-rt so they can
additionally be compiled
        with the HIP compiler and generate device code.  This would be our first
choice. It provides
        obvious advantages over keeping separate sanitizer runtimes for
heterogeneous devices and
        easier maintenance while ensuring feature parity. This can be broadly
summarized in the
        following activities:

        a. Supporting device code generation

            We propose to add suitably protected changes which allow the code to
be compiled for the
            HIP language and generate device code.

        b. Supporting device and host communication

            Sanitizer features like violation reporting requires communication
from device to host.
            We propose to add support for the device code to communicate with
the host for
            reporting. We see this going into a target specific sub-folder.

        c. Adding new interceptors

            We foresee a need for new interceptors, for example, device specific
memory allocators.
            We propose to add these interceptors.

        d. Updating the build system

            We propose to update the build system to build device code when
requested, as well as
            host code.

    2. Implement the device side sanitizer runtime separately

        In this scenario, the device side of the sanitizer runtime would be
implemented in a
        separate folder, possibly in a separate repository.  For example, this
could be added to our
        existing device side runtime. The changes mentioned in 1b and 1c to the
existing host
        runtime would still be required.

Combinations of different approaches listed above are also possible. We will be
happy to hear about
other approaches and ideas to move forward with them as well. Adding direct
support for
heterogeneous device sanitizer runtime in compiler-rt will also be helpful for
other heterogeneous
devices in future.

Many thanks,
Reshabh and Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201029/591f8dbd/attachment.html>

Hi all,

Following approach #2 from the previous RFC, we are preparing some updates to
the address sanitizer
runtime and we would like to get some feedback on how to approach these changes.

In order to isolate the impact of our changes on others, we are thinking to
guard our changes inside a
distinguishable macro, for example, SANITIZER_AMDGPU, and to localize our
changes as much as possible
within new files whose name includes AMDGPU.  We would your feedback on this,
and suggestions
about how to trigger the definition of the macro. The host runtime will always
be targeted to the host
which implies that we will not be able to check the target architecture and set
the macro, like other
architectures. We propose two ways to achieve this:


  1.  Based on LLVM_TARGETS_TO_BUILD value

We can look for AMDGPU in the LLVM_TARGETS_TO_BUILD string and trigger the
AMDGPU

specific macro. We will make sure it does not trigger with the default option
which is the list of

all the supported architectures.



  1.  Adding a new cmake flag

We could require a specific cmake variable be set which when used will trigger
the AMDGPU

specific macro.
We are definitely open to other alternatives.

Many thanks,
Reshabh

From: Sharma, Reshabh Kumar
Sent: Thursday, October 29, 2020 9:34 PM
To: 'llvm-dev at lists.llvm.org' <llvm-dev at lists.llvm.org>
Cc: Sumner, Brian <Brian.Sumner at amd.com>
Subject: [RFC] Implementing the sanitizer runtimes for heterogeneous devices

Hi all,

We are thinking of extending the LLVM sanitizers, starting with asan, to
heterogeneous situations
such as those found in OpenCL and HIP. We have already started thinking about
the way
instrumentation will look like for different address spaces and posted a small
RFC
(http://lists.llvm.org/pipermail/llvm-dev/2020-July/143310.html) about removing
the generated
inttoptr and ptrtoint pair from instrumented IR. We have started to look into
the runtime
implementation, and we will be happy to get feedback and suggestion from the
community so that we
can incorporate them in the design itself. We are looking at implementing a
sanitizer runtime that
would support heterogeneous devices such as AMD GPUs and we present some
directions that we can take
to make that happen:

    1. Add support for compilation of the compiler-rt sanitizer runtime by a
device compiler

       We propose to update the sanitizer parts of compiler-rt so they can
additionally be compiled
        with the HIP compiler and generate device code.  This would be our first
choice. It provides
        obvious advantages over keeping separate sanitizer runtimes for
heterogeneous devices and
        easier maintenance while ensuring feature parity. This can be broadly
summarized in the
        following activities:

        a. Supporting device code generation

            We propose to add suitably protected changes which allow the code to
be compiled for the
            HIP language and generate device code.

        b. Supporting device and host communication

            Sanitizer features like violation reporting requires communication
from device to host.
            We propose to add support for the device code to communicate with
the host for
            reporting. We see this going into a target specific sub-folder.

        c. Adding new interceptors

            We foresee a need for new interceptors, for example, device specific
memory allocators.
            We propose to add these interceptors.

        d. Updating the build system

            We propose to update the build system to build device code when
requested, as well as
            host code.

    2. Implement the device side sanitizer runtime separately

        In this scenario, the device side of the sanitizer runtime would be
implemented in a
        separate folder, possibly in a separate repository.  For example, this
could be added to our
        existing device side runtime. The changes mentioned in 1b and 1c to the
existing host
        runtime would still be required.

Combinations of different approaches listed above are also possible. We will be
happy to hear about
other approaches and ideas to move forward with them as well. Adding direct
support for
heterogeneous device sanitizer runtime in compiler-rt will also be helpful for
other heterogeneous
devices in future.

Many thanks,
Reshabh and Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210510/b2d41f02/attachment-0001.html>