llvm dev - Sep 2017 - tsan_interface_java.h; any users? reviving https://github.com/google/java-thread-sanitizer?

The interface files you see are for a prototype of Java Tsan that's
internal to Google. It's for use by a JDK that we customized with extensive
JVM patches, and enables running the Tsan engine on Java and native code
simultaneously.

We presented this prototype recently at the JVM Language Summit. Here's the
talk:
https://www.youtube.com/watch?v=UykhZ36W04I

We are starting to explore how to upstream the JVM patches. It's going to
be tricky: The patches are fairly invasive, the setup and expectations are
different in terms of upstream vs. home-grown build system, and the whole
thing will require healthy upstream buy-in. This will all take time, so no
promises of a speedy release, unfortunately.

Since you seem to be ok with bytecode rewriting (we have reasons for not
doing that), I would like to point you to FastTrack, a pure-Java
implementation of the Happens-Before race detection algorithm.
https://github.com/stephenfreund/RoadRunner
(Disclosure) I worked with the tool and the authors in my grad student
days.
I see it's been updated recently too, so feel free to give it a spin.


On Thu, Aug 31, 2017 at 4:00 AM, Dmitry Vyukov <dvyukov at google.com>
wrote:
> On Thu, Aug 31, 2017 at 11:31 AM, Jiri Danek <jdanek at redhat.com>
wrote:
> >> On Aug 30, 2017, at 12:51 AM, Jiri Danek via llvm-dev
> >> <llvm-dev at lists.llvm.org> wrote:
> >> I found https://github.com/google/java-thread-sanitizer, which
seems
> to be
> >> now abandoned. It used to work with ThreadSanitizer v1. Has
anybody ever
> >> tried to make it work with the current ThreadSanitizer?
> > I gave it a try and bending java-thread-sanitizer to my needs seems
> doable
> > so far:
> >
> > 1) making java-thread-sanitizer compile with Gradle and run its
example
> > under Java 8, check (although I have to pass -noverify to JVM)
> > 2) LD_PRELOADing libtsan.so to JVM, accessing it from JNA, check
> > 3) reproducing one tsan test (tests/rtl/tsan_mop.cc) with Java threads
> and
> > made up pointers passed to __tsan_write*(), check
> > 3) disabling Garbage Collection in JVM (just to possibly simplify
things)
> > cannot be done, but it should not matter, because
System.identityHashcode
> > should be unchanging, so check
> >
> > With that, just put that all together and I should have a sanitizer
for
> > java.
> >
> > Only problem so far is that I don't have alternative for
> __tsan_func_entry,
> > so I cannot have reasonable stacktraces in TSan error messages. I
should
> be
> > able to work around it during Java code instrumentation, to save
myself
> > enough info to be able to make sense of the impoverished TSan
messages.
>
>
> > I've been looking at ThreadSanitizer in
> > https://github.com/llvm-mirror/compiler-rt and I noticed some
> Java-related
> > files. Is there a version of ThreadSanitizer build on top of that,
which
> I
> > could use to sanitize my Java programs? I could not find any.
>
>
> +Jaeheon, Alexander, Jeremy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170831/2949013b/attachment.html>

On Fri, Sep 1, 2017 at 1:33 AM, Jaeheon Yi <jaeheon at google.com> wrote:
> The interface files you see are for a prototype of Java Tsan that's
> internal to Google. It's for use by a JDK that we customized with
extensive
> JVM patches, and enables running the Tsan engine on Java and native code
> simultaneously.
>
> We presented this prototype recently at the JVM Language Summit. Here's
> the talk:
> https://www.youtube.com/watch?v=UykhZ36W04I
>
> We are starting to explore how to upstream the JVM patches. It's going
to
> be tricky: The patches are fairly invasive, the setup and expectations are
> different in terms of upstream vs. home-grown build system, and the whole
> thing will require healthy upstream buy-in. This will all take time, so no
> promises of a speedy release, unfortunately.
>
I can imagine. Thanks for the info and the talk. I am not holding my breath.

> Since you seem to be ok with bytecode rewriting (we have reasons for not
> doing that), I would like to point you to FastTrack, a pure-Java
> implementation of the Happens-Before race detection algorithm.
> https://github.com/stephenfreund/RoadRunner
> (Disclosure) I worked with the tool and the authors in my grad student
> days.
> I see it's been updated recently too, so feel free to give it a spin.
>
Thanks! I did. It seems not to understand Thread Pools. JDK docs say
"Memory consistency effects: Actions in a thread prior to submitting a
Runnable object to an Executor happen-before its execution begins, perhaps
in another thread.", but if I am reading the errors from RoadRunner
properly, the tool does not see that. Other than that, it seems to work.
-- 
Jiří Daněk
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170901/c166b2fd/attachment.html>

On Fri, Sep 1, 2017 at 7:14 AM, Jiri Danek <jdanek at redhat.com> wrote:
> On Fri, Sep 1, 2017 at 1:33 AM, Jaeheon Yi <jaeheon at google.com>
wrote:
>
>> The interface files you see are for a prototype of Java Tsan that's
>> internal to Google. It's for use by a JDK that we customized with
extensive
>> JVM patches, and enables running the Tsan engine on Java and native
code
>> simultaneously.
>>
>> We presented this prototype recently at the JVM Language Summit.
Here's
>> the talk:
>> https://www.youtube.com/watch?v=UykhZ36W04I
>>
>> We are starting to explore how to upstream the JVM patches. It's
going to
>> be tricky: The patches are fairly invasive, the setup and expectations
are
>> different in terms of upstream vs. home-grown build system, and the
whole
>> thing will require healthy upstream buy-in. This will all take time, so
no
>> promises of a speedy release, unfortunately.
>>
>
> I can imagine. Thanks for the info and the talk. I am not holding my
> breath.
>
>
>> Since you seem to be ok with bytecode rewriting (we have reasons for
not
>> doing that), I would like to point you to FastTrack, a pure-Java
>> implementation of the Happens-Before race detection algorithm.
>> https://github.com/stephenfreund/RoadRunner
>> (Disclosure) I worked with the tool and the authors in my grad student
>> days.
>> I see it's been updated recently too, so feel free to give it a
spin.
>>
>
> Thanks! I did. It seems not to understand Thread Pools. JDK docs say
> "Memory consistency effects: Actions in a thread prior to submitting a
> Runnable object to an Executor happen-before its execution begins, perhaps
> in another thread.", but if I am reading the errors from RoadRunner
> properly, the tool does not see that. Other than that, it seems to work.
>
Check out InstrumentationFilter. I recall we avoided instrumenting JDK
classes (for $reasons; I don't remember), but maybe you can add just the
Executor class you need.

-- 
> Jiří Daněk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170901/406bf950/attachment.html>