llvm dev - Jul 2017 - Using TSAN along with custom llvm IR pass

On Sun, Jul 9, 2017 at 3:50 PM, Nischai Vinesh <nischai.vinesh at
gmail.com> wrote:
> Hello,
>
> I tried that as well but there are no tsan warnings thrown at run time,
even
> though there are data race error!
>
> The steps I followed:
> 1. Generate bitcode using 'clang -emit-llvm' command
> 2. Using the 'opt -load ..' command, I instrumented the bitcode
file with my
> custom pass
> 3. Using the command 'opt -tsan ..', instrumented the output
bitcode file
> from step 2
> 4. Using the llvm backend compiler, 'llc', compiled the program to
native
> assembly
> 5. The output from step 4 is compiled again with 'clang
-fsanitize=thread'
> (linking tsan here) for the final object file.
>
> Executing this final output file works fine, except that it doesn't
throws
> any TSAN warnings!

Hi Nischai,

I see 3 possibilities:
1. Either the code is somehow ends up being non-instrumented, or
2. tsan does not consider what it sees a data race, or
3. the data race is masked by unrelated synchronization.

For 1 you can check that the resulting code in fact contains __tsan_* callbacks.
For 2 and 3 you can rebuilt tsan runtime with TSAN_DEBUG_OUTPUT=2
define, then it will print everything it sees (memory accesses and
synchronization). This output will allow us to figure out why it does
not report a race.

I think the code is not getting instrumented with tsan at all. In the 'step 
3', when I instrument it using 'opt -tsan ...', it gives a warning 
"WARNING: You're attempting to print out a bitcode file." I guess
this
method doesn't instrument the code.

Is there any other way I can instrument the code with tsan?




On Monday, July 10, 2017 at 7:21:28 AM UTC+2, Dmitry Vyukov
wrote:
>
> On Sun, Jul 9, 2017 at 3:50 PM, Nischai Vinesh <nischai... at gmail.com 
> <javascript:>> wrote: 
> > Hello, 
> > 
> > I tried that as well but there are no tsan warnings thrown at run
time,
> even 
> > though there are data race error! 
> > 
> > The steps I followed: 
> > 1. Generate bitcode using 'clang -emit-llvm' command 
> > 2. Using the 'opt -load ..' command, I instrumented the
bitcode file
> with my 
> > custom pass 
> > 3. Using the command 'opt -tsan ..', instrumented the output
bitcode
> file 
> > from step 2 
> > 4. Using the llvm backend compiler, 'llc', compiled the
program to
> native 
> > assembly 
> > 5. The output from step 4 is compiled again with 'clang 
> -fsanitize=thread' 
> > (linking tsan here) for the final object file. 
> > 
> > Executing this final output file works fine, except that it
doesn't
> throws 
> > any TSAN warnings! 
>
>
> Hi Nischai, 
>
> I see 3 possibilities: 
> 1. Either the code is somehow ends up being non-instrumented, or 
> 2. tsan does not consider what it sees a data race, or 
> 3. the data race is masked by unrelated synchronization. 
>
> For 1 you can check that the resulting code in fact contains __tsan_* 
> callbacks. 
> For 2 and 3 you can rebuilt tsan runtime with TSAN_DEBUG_OUTPUT=2 
> define, then it will print everything it sees (memory accesses and 
> synchronization). This output will allow us to figure out why it does 
> not report a race. 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170710/948f690d/attachment.html>

On Mon, Jul 10, 2017 at 1:06 PM, Nischai Vinesh
<nischai.vinesh at gmail.com> wrote:
> I think the code is not getting instrumented with tsan at all. In the
'step
> 3', when I instrument it using 'opt -tsan ...', it gives a
warning "WARNING:
> You're attempting to print out a bitcode file." I guess this
method doesn't
> instrument the code.
>
> Is there any other way I can instrument the code with tsan?

One option is to add your pass to llvm and execute clang with -fsanitize=thread.

> On Monday, July 10, 2017 at 7:21:28 AM UTC+2, Dmitry Vyukov wrote:
>>
>> On Sun, Jul 9, 2017 at 3:50 PM, Nischai Vinesh <nischai... at
gmail.com>
>> wrote:
>> > Hello,
>> >
>> > I tried that as well but there are no tsan warnings thrown at run
time,
>> > even
>> > though there are data race error!
>> >
>> > The steps I followed:
>> > 1. Generate bitcode using 'clang -emit-llvm' command
>> > 2. Using the 'opt -load ..' command, I instrumented the
bitcode file
>> > with my
>> > custom pass
>> > 3. Using the command 'opt -tsan ..', instrumented the
output bitcode
>> > file
>> > from step 2
>> > 4. Using the llvm backend compiler, 'llc', compiled the
program to
>> > native
>> > assembly
>> > 5. The output from step 4 is compiled again with 'clang
>> > -fsanitize=thread'
>> > (linking tsan here) for the final object file.
>> >
>> > Executing this final output file works fine, except that it
doesn't
>> > throws
>> > any TSAN warnings!
>>
>>
>> Hi Nischai,
>>
>> I see 3 possibilities:
>> 1. Either the code is somehow ends up being non-instrumented, or
>> 2. tsan does not consider what it sees a data race, or
>> 3. the data race is masked by unrelated synchronization.
>>
>> For 1 you can check that the resulting code in fact contains __tsan_*
>> callbacks.
>> For 2 and 3 you can rebuilt tsan runtime with TSAN_DEBUG_OUTPUT=2
>> define, then it will print everything it sees (memory accesses and
>> synchronization). This output will allow us to figure out why it does
>> not report a race.
>
> --
> You received this message because you are subscribed to the Google Groups
> "thread-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to thread-sanitizer+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Hi Dmitry,

I managed to get the program instrumented with tsan using 'opt -tsan
..',
but still it is not printing tsan warnings when I run the binary. When I 
instrument the same program with tsan during compilation using 'clang 
-fsanitize=thread ...', without instrumenting my IR pass, it does give out 
tsan warnings!


On Monday, July 10, 2017 at 7:21:28 AM UTC+2, Dmitry Vyukov
wrote:
>
> On Sun, Jul 9, 2017 at 3:50 PM, Nischai Vinesh <nischai... at gmail.com 
> <javascript:>> wrote: 
> > Hello, 
> > 
> > I tried that as well but there are no tsan warnings thrown at run
time,
> even 
> > though there are data race error! 
> > 
> > The steps I followed: 
> > 1. Generate bitcode using 'clang -emit-llvm' command 
> > 2. Using the 'opt -load ..' command, I instrumented the
bitcode file
> with my 
> > custom pass 
> > 3. Using the command 'opt -tsan ..', instrumented the output
bitcode
> file 
> > from step 2 
> > 4. Using the llvm backend compiler, 'llc', compiled the
program to
> native 
> > assembly 
> > 5. The output from step 4 is compiled again with 'clang 
> -fsanitize=thread' 
> > (linking tsan here) for the final object file. 
> > 
> > Executing this final output file works fine, except that it
doesn't
> throws 
> > any TSAN warnings! 
>
>
> Hi Nischai, 
>
> I see 3 possibilities: 
> 1. Either the code is somehow ends up being non-instrumented, or 
> 2. tsan does not consider what it sees a data race, or 
> 3. the data race is masked by unrelated synchronization. 
>
> For 1 you can check that the resulting code in fact contains __tsan_* 
> callbacks. 
> For 2 and 3 you can rebuilt tsan runtime with TSAN_DEBUG_OUTPUT=2 
> define, then it will print everything it sees (memory accesses and 
> synchronization). This output will allow us to figure out why it does 
> not report a race. 
>
So I guess we can rule out the 1st possibility. Can you kindly tell me how 
to rebuild tsan runtime? Sorry, I am new to LLVM and stuff.



 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170711/46e45cc2/attachment.html>

On Tue, Jul 11, 2017 at 3:26 PM, Nischai Vinesh
<nischai.vinesh at gmail.com> wrote:
> Hi Dmitry,
>
> I managed to get the program instrumented with tsan using 'opt -tsan
..',
> but still it is not printing tsan warnings when I run the binary. When I
> instrument the same program with tsan during compilation using 'clang
> -fsanitize=thread ...', without instrumenting my IR pass, it does give
out
> tsan warnings!
>
>
> On Monday, July 10, 2017 at 7:21:28 AM UTC+2, Dmitry Vyukov wrote:
>>
>> On Sun, Jul 9, 2017 at 3:50 PM, Nischai Vinesh <nischai... at
gmail.com>
>> wrote:
>> > Hello,
>> >
>> > I tried that as well but there are no tsan warnings thrown at run
time,
>> > even
>> > though there are data race error!
>> >
>> > The steps I followed:
>> > 1. Generate bitcode using 'clang -emit-llvm' command
>> > 2. Using the 'opt -load ..' command, I instrumented the
bitcode file
>> > with my
>> > custom pass
>> > 3. Using the command 'opt -tsan ..', instrumented the
output bitcode
>> > file
>> > from step 2
>> > 4. Using the llvm backend compiler, 'llc', compiled the
program to
>> > native
>> > assembly
>> > 5. The output from step 4 is compiled again with 'clang
>> > -fsanitize=thread'
>> > (linking tsan here) for the final object file.
>> >
>> > Executing this final output file works fine, except that it
doesn't
>> > throws
>> > any TSAN warnings!
>>
>>
>> Hi Nischai,
>>
>> I see 3 possibilities:
>> 1. Either the code is somehow ends up being non-instrumented, or
>> 2. tsan does not consider what it sees a data race, or
>> 3. the data race is masked by unrelated synchronization.
>>
>> For 1 you can check that the resulting code in fact contains __tsan_*
>> callbacks.
>> For 2 and 3 you can rebuilt tsan runtime with TSAN_DEBUG_OUTPUT=2
>> define, then it will print everything it sees (memory accesses and
>> synchronization). This output will allow us to figure out why it does
>> not report a race.
>
>
> So I guess we can rule out the 1st possibility. Can you kindly tell me how
> to rebuild tsan runtime? Sorry, I am new to LLVM and stuff.
Here is the script we use on build bots:
https://github.com/llvm-mirror/zorg/blob/master/zorg/buildbot/builders/sanitizers/buildbot_standard.sh

You need this option with -DCOMPILER_RT_DEBUG=ON
-DCOMPILER_RT_TSAN_DEBUG_OUTPUT=ON:

echo @@@BUILD_STEP build tsan with stats and debug output@@@
build_tsan "${TSAN_FULL_DEBUG_BUILD_DIR}" "-DCOMPILER_RT_DEBUG=ON
-DCOMPILER_RT_TSAN_DEBUG_OUTPUT=ON -DLLVM_INCLUDE_TESTS=OFF" gcc g++