llvm dev - May 2016 - Runtime interception: design problem

Hi everyone,

I am having troubles but this shouldn't be hard to solve for many people
here. I am beginning a runtime feature for the BoundsChecking pass and I
want to replace the libc malloc&free. I followed the design of
AddressSanitizer (Asan) and tried to use the INTERCEPTOR macro from the
interception.h file of compiler-rt library.

Here is the problem. The file I modify (BoundsCheking.cpp) is in
lib/Transforms/Instrumentation/ and I can't include properly interception.h
(which is in projects/compiler-rt/lib/interception/). I looked at the
CMakeLists.txt and how other files included interception.h but they are all
from compiler-rt lib directory.

I assume this is normal as runtime project should be developped under the
correct directory but I don't see how this should be designed in order that
the code optimized by BoundsCheking Pass uses my own malloc&free functions.

I spent quite some time on Asan runtime code and found that runtime could
be initialized with a call to __asan_init() (function defined in
asan_rtl.cc) from the instrumented code directly but I don't know if
it's
the only way to do it or how to reproduce it...

So to resume, I want to make my own malloc and free function to be called
by the code I instrument with BoundsChecking. Any suggestion is welcome =)

Thanks,
Pierre
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160525/99e280af/attachment-0001.html>

Dear Pierre,

Stepping up a level, what is your goal in replacing calls to malloc() 
and free()?  Is it any different than what SAFECode, SoftBound, or ASan do?

Regards,

John Criswell

On 5/25/16 8:05 AM, Pierre Gagelin via llvm-dev wrote:
> Hi everyone,
>
> I am having troubles but this shouldn't be hard to solve for many 
> people here. I am beginning a runtime feature for the BoundsChecking 
> pass and I want to replace the libc malloc&free. I followed the design 
> of AddressSanitizer (Asan) and tried to use the INTERCEPTOR macro from 
> the interception.h file of compiler-rt library.
>
> Here is the problem. The file I modify (BoundsCheking.cpp) is in 
> lib/Transforms/Instrumentation/ and I can't include properly 
> interception.h (which is in projects/compiler-rt/lib/interception/). I 
> looked at the CMakeLists.txt and how other files included 
> interception.h but they are all from compiler-rt lib directory.
>
> I assume this is normal as runtime project should be developped under 
> the correct directory but I don't see how this should be designed in 
> order that the code optimized by BoundsCheking Pass uses my own 
> malloc&free functions.
>
> I spent quite some time on Asan runtime code and found that runtime 
> could be initialized with a call to __asan_init() (function defined in 
> asan_rtl.cc) from the instrumented code directly but I don't know if 
> it's the only way to do it or how to reproduce it...
>
> So to resume, I want to make my own malloc and free function to be 
> called by the code I instrument with BoundsChecking. Any suggestion is 
> welcome =)
>
> Thanks,
> Pierre
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
http://www.cs.rochester.edu/u/criswell

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160525/6d04e000/attachment.html>

Hi John,

On 25 May 2016 at 16:11, John Criswell <jtcriswel at gmail.com> wrote:
> Dear Pierre,
>
> Stepping up a level, what is your goal in replacing calls to malloc() and
> free()?  Is it any different than what SAFECode, SoftBound, or ASan do?
>
That's a good question. I didn't knew about SoftBound until now, so
thank
you for the name =). Anyway here is what I know:
  - ASan is a shadow-based memory protection, which allows accesses to an
addressable field
  - I don't know every aspects of SAFECode. What I looked at was
BaggyBoundsCheck which is an object based memory protection. In this way it
still allows wild accesses inside an allocation
  - I quickly looked at SoftBound and it does protect the memory at a
precise level. However metadata is separately stored which involve higher
overhead

My point is BoundsChecking has been designed (by Nuno Lopes) to be a very
low overhead checking tool. For the moment no runtime have been implemented
and I wanted to do a lightweight one on it (I am just a student so even for
the personal experience I'm interested). The runtime would be necessary to
solve the actual problem of inter-procedural checks.

As said I haven't strong basis on the topic so any suggestion is welcome.
However I know there are several criteria a program can't all match
together like effectiveness, memory use, compatibility...

>
> Regards,
>
> John Criswell
>
Thanks John,
Pierre
> --
> John Criswell
> Assistant Professor
> Department of Computer Science, University of
Rochesterhttp://www.cs.rochester.edu/u/criswell
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160526/e7007681/attachment.html>

> On May 25, 2016, at 6:05 AM, Pierre Gagelin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi everyone,
> 
> I am having troubles but this shouldn't be hard to solve for many
people here. I am beginning a runtime feature for the BoundsChecking pass and I
want to replace the libc malloc&free. I followed the design of
AddressSanitizer (Asan) and tried to use the INTERCEPTOR macro from the
interception.h file of compiler-rt library.
> 
> Here is the problem. The file I modify (BoundsCheking.cpp) is in
lib/Transforms/Instrumentation/ and I can't include properly interception.h
(which is in projects/compiler-rt/lib/interception/).
Something does not make sense to me here: lib/Transforms/... is about stuff that
will transform/generate code, it does not contain code that will be part of the
final binary. So this transform may generates calls to your runtime, but should
not need the runtime to operate.

-- 
Mehdi


> I looked at the CMakeLists.txt and how other files included interception.h
but they are all from compiler-rt lib directory.
> 
> I assume this is normal as runtime project should be developped under the
correct directory but I don't see how this should be designed in order that
the code optimized by BoundsCheking Pass uses my own malloc&free functions.
> 
> I spent quite some time on Asan runtime code and found that runtime could
be initialized with a call to __asan_init() (function defined in asan_rtl.cc)
from the instrumented code directly but I don't know if it's the only
way to do it or how to reproduce it...
> 
> So to resume, I want to make my own malloc and free function to be called
by the code I instrument with BoundsChecking. Any suggestion is welcome =)
> 
> Thanks,
> Pierre
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 26 May 2016 at 16:46, Mehdi Amini <mehdi.amini at apple.com> wrote:
>
> > On May 25, 2016, at 6:05 AM, Pierre Gagelin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hi everyone,
> >
> > I am having troubles but this shouldn't be hard to solve for many
people
> here. I am beginning a runtime feature for the BoundsChecking pass and I
> want to replace the libc malloc&free. I followed the design of
> AddressSanitizer (Asan) and tried to use the INTERCEPTOR macro from the
> interception.h file of compiler-rt library.
> >
> > Here is the problem. The file I modify (BoundsCheking.cpp) is in
> lib/Transforms/Instrumentation/ and I can't include properly
interception.h
> (which is in projects/compiler-rt/lib/interception/).
>
> Something does not make sense to me here: lib/Transforms/... is about
> stuff that will transform/generate code, it does not contain code that will
> be part of the final binary. So this transform may generates calls to your
> runtime, but should not need the runtime to operate.
>
>
Yes I know this does not make sense. That's why I asked for some help to
clear the design (in my head and also in the code).

As I said further in the message, when I analyzed ASan I found the runtime
could be called by instrumented code through the function __asan_init() but
I haven't managed to reproduce the mechanism. Though I haven't spend
much
time on it as I thought maybe another solution existed or someone could
explain it to me easily.

But as you said it could be the regular way to do it I looked a bit deeper
and it appears to be this pattern:
  - the doInitialization of the FunctionPass (which as more access than the
runOnFunction) calls ModuleUtils createSanitizerCtorAndInitFunctions
function to get two pointers (constructor_func, init_func)
  - stores it in the FunctionPass structure
  - runOnFunction may then call init_func via the IRBuilder when needed.
Am I right? I'll try to use it the same way.

> --
> Mehdi
>
Thank you Mehdi!
Pierre
>
>
>
> > I looked at the CMakeLists.txt and how other files included
> interception.h but they are all from compiler-rt lib directory.
> >
> > I assume this is normal as runtime project should be developped under
> the correct directory but I don't see how this should be designed in
order
> that the code optimized by BoundsCheking Pass uses my own malloc&free
> functions.
> >
> > I spent quite some time on Asan runtime code and found that runtime
> could be initialized with a call to __asan_init() (function defined in
> asan_rtl.cc) from the instrumented code directly but I don't know if
it's
> the only way to do it or how to reproduce it...
> >
> > So to resume, I want to make my own malloc and free function to be
> called by the code I instrument with BoundsChecking. Any suggestion is
> welcome =)
> >
> > Thanks,
> > Pierre
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160527/d8a081ae/attachment.html>