llvm dev - Dec 2012 - [LLVMdev] radr://12777299, "potential pthread/eh bug exposed by libsanitizer"

+kledzik at apple.com
The dynamic runtime is using dylib interposition (google for
"__DATA,__interpose).
If I'm understanding correctly (Nick, can you please confirm this?)
this allows to interpose the function regardless of the two-level
namespace.
The support for dynamic runtime in ASan is almost there. But the new
interposition method has revealed some issues with the allocator which
were corked here and there before. Most of those are caused by a
CoreFoundation dependency, which I'm trying to eliminate now.


On Mon, Dec 3, 2012 at 8:50 PM, Rafael Espíndola
<rafael.espindola at gmail.com> wrote:
> On 30 November 2012 13:32, Alexander Potapenko <glider at google.com>
wrote:
>> No, we are not going to use mach_inject. This isn't portable and
may
>> be even harder to set up than mach_override.
>> The new ASan runtime will use the dylib interposition and will in fact
>> require DYLD_INSERT_LIBRARIES to work. However ASan already handles it
>> correctly itself: if the corresponding env var is missing the app is
>> just re-execed.
>> Dylib interposition is supported by Apple and should work on iOS as
>> well as Mac OS. It will also probably simplify hooking the memory
>> allocations in ASan, which is now very tricky.
>
> This is interesting! I had some difficulties with mach_override myself
> in firefox. Don't you have to disable the two-level namespace to be
> able to override the functions you want? What currently blocks using
> DYLD_INSERT_LIBRARIES instead of mach_override?
>
> Cheers,
> Rafael


--
Alexander Potapenko
Software Engineer
Google Moscow

On Tue, Dec 04, 2012 at 09:46:09AM -0800, Alexander Potapenko
wrote:
> +kledzik at apple.com
> The dynamic runtime is using dylib interposition (google for
> "__DATA,__interpose).
> If I'm understanding correctly (Nick, can you please confirm this?)
> this allows to interpose the function regardless of the two-level
> namespace.
> The support for dynamic runtime in ASan is almost there. But the new
> interposition method has revealed some issues with the allocator which
> were corked here and there before. Most of those are caused by a
> CoreFoundation dependency, which I'm trying to eliminate now.
> 
Alexander,
   Are you trying to eliminate the CoreFoundation dependency or the
issues it exposed in the allocator? I am also curious if these 
issues could be related to the observation that libasan with the
mac interpose function support still shows 323 FSF g++ testsuite
failures compared to only 107 on x86_64 Fedora 15 linux? I planned
on trying to find some of those test cases which fail on both
FSF g++ and clang++ with the dyanmic runtime under darwin but
not under linux so I could open an llvm bugzilla on those.
              Jack
> 
> On Mon, Dec 3, 2012 at 8:50 PM, Rafael Espíndola
> <rafael.espindola at gmail.com> wrote:
> > On 30 November 2012 13:32, Alexander Potapenko <glider at
google.com> wrote:
> >> No, we are not going to use mach_inject. This isn't portable
and may
> >> be even harder to set up than mach_override.
> >> The new ASan runtime will use the dylib interposition and will in
fact
> >> require DYLD_INSERT_LIBRARIES to work. However ASan already
handles it
> >> correctly itself: if the corresponding env var is missing the app
is
> >> just re-execed.
> >> Dylib interposition is supported by Apple and should work on iOS
as
> >> well as Mac OS. It will also probably simplify hooking the memory
> >> allocations in ASan, which is now very tricky.
> >
> > This is interesting! I had some difficulties with mach_override myself
> > in firefox. Don't you have to disable the two-level namespace to
be
> > able to override the functions you want? What currently blocks using
> > DYLD_INSERT_LIBRARIES instead of mach_override?
> >
> > Cheers,
> > Rafael
> 
> 
> 
> --
> Alexander Potapenko
> Software Engineer
> Google Moscow

On Tue, Dec 04, 2012 at 09:46:09AM -0800, Alexander Potapenko
wrote:
> +kledzik at apple.com
> The dynamic runtime is using dylib interposition (google for
> "__DATA,__interpose).
> If I'm understanding correctly (Nick, can you please confirm this?)
> this allows to interpose the function regardless of the two-level
> namespace.
> The support for dynamic runtime in ASan is almost there. But the new
> interposition method has revealed some issues with the allocator which
> were corked here and there before. Most of those are caused by a
> CoreFoundation dependency, which I'm trying to eliminate now.
Alexander,
   Maybe Nick was thinking about iOS rather than darwin with regard to
the switchover from mach_override to mac interpose function. I see...

http://clang-developers.42468.n3.nabble.com/PATCH-ASan-Link-with-the-dynamic-ASan-library-on-iOS-td4027668.html

which suggests that that change was made for iOS but not darwin.
         Jack
> 
> 
> On Mon, Dec 3, 2012 at 8:50 PM, Rafael Espíndola
> <rafael.espindola at gmail.com> wrote:
> > On 30 November 2012 13:32, Alexander Potapenko <glider at
google.com> wrote:
> >> No, we are not going to use mach_inject. This isn't portable
and may
> >> be even harder to set up than mach_override.
> >> The new ASan runtime will use the dylib interposition and will in
fact
> >> require DYLD_INSERT_LIBRARIES to work. However ASan already
handles it
> >> correctly itself: if the corresponding env var is missing the app
is
> >> just re-execed.
> >> Dylib interposition is supported by Apple and should work on iOS
as
> >> well as Mac OS. It will also probably simplify hooking the memory
> >> allocations in ASan, which is now very tricky.
> >
> > This is interesting! I had some difficulties with mach_override myself
> > in firefox. Don't you have to disable the two-level namespace to
be
> > able to override the functions you want? What currently blocks using
> > DYLD_INSERT_LIBRARIES instead of mach_override?
> >
> > Cheers,
> > Rafael
> 
> 
> 
> --
> Alexander Potapenko
> Software Engineer
> Google Moscow

Currently the replacement of allocation routines is based on creating
a new malloc zone and a new CFAllocator (because the allocator
replacement is done later than it could be, we must have both). This
makes us depend on CoreFoundation to call CFAllocatorSetDefault.
Because of some bugs in CF which start firing after
CFAllocatorSetDefault, we have to add several hacks to circumvent the
effects of those bugs.
The current hypothesis is that using dylib interposition for memory
allocation routines (instead of playing with malloc zones) will allow
us to remove the CF dependency together with all the hacks. I'm trying
to check if this is true.
The current plan is to have dynamic runtime for both iOS (for which we
don't have any other option) and Mac OS.

On Tue, Dec 4, 2012 at 10:17 AM, Jack Howarth <howarth at
bromo.med.uc.edu> wrote:
> On Tue, Dec 04, 2012 at 09:46:09AM -0800, Alexander Potapenko wrote:
>> +kledzik at apple.com
>> The dynamic runtime is using dylib interposition (google for
>> "__DATA,__interpose).
>> If I'm understanding correctly (Nick, can you please confirm this?)
>> this allows to interpose the function regardless of the two-level
>> namespace.
>> The support for dynamic runtime in ASan is almost there. But the new
>> interposition method has revealed some issues with the allocator which
>> were corked here and there before. Most of those are caused by a
>> CoreFoundation dependency, which I'm trying to eliminate now.
>>
>
> Alexander,
>    Are you trying to eliminate the CoreFoundation dependency or the
> issues it exposed in the allocator? I am also curious if these
> issues could be related to the observation that libasan with the
> mac interpose function support still shows 323 FSF g++ testsuite
> failures compared to only 107 on x86_64 Fedora 15 linux? I planned
> on trying to find some of those test cases which fail on both
> FSF g++ and clang++ with the dyanmic runtime under darwin but
> not under linux so I could open an llvm bugzilla on those.
>               Jack
>
>>
>> On Mon, Dec 3, 2012 at 8:50 PM, Rafael Espíndola
>> <rafael.espindola at gmail.com> wrote:
>> > On 30 November 2012 13:32, Alexander Potapenko <glider at
google.com> wrote:
>> >> No, we are not going to use mach_inject. This isn't
portable and may
>> >> be even harder to set up than mach_override.
>> >> The new ASan runtime will use the dylib interposition and will
in fact
>> >> require DYLD_INSERT_LIBRARIES to work. However ASan already
handles it
>> >> correctly itself: if the corresponding env var is missing the
app is
>> >> just re-execed.
>> >> Dylib interposition is supported by Apple and should work on
iOS as
>> >> well as Mac OS. It will also probably simplify hooking the
memory
>> >> allocations in ASan, which is now very tricky.
>> >
>> > This is interesting! I had some difficulties with mach_override
myself
>> > in firefox. Don't you have to disable the two-level namespace
to be
>> > able to override the functions you want? What currently blocks
using
>> > DYLD_INSERT_LIBRARIES instead of mach_override?
>> >
>> > Cheers,
>> > Rafael
>>
>>
>>
>> --
>> Alexander Potapenko
>> Software Engineer
>> Google Moscow


-- 
Alexander Potapenko
Software Engineer
Google Moscow