llvm dev - Nov 2012 - [LLVMdev] Bug Report -- Possible optimizer bug with thread_local variables

Hello,

I apologize if this has already been fixed or reported.  I believe there is
a bug in the way the optimizer deals with thread_local variables.  The
attached program, test.c, has a thread-local variable "int Foo" and a
global variable "int *Ptr".  The program takes the following steps:

1) The main thread spawns a new thread and waits
2) The new thread writes Foo = 50 and Ptr = &Foo, then signals the main
thread and waits
3) The main thread prints *Ptr, releases the new thread, and exits

The crux of this example is that the main thread obtains a pointer to the
new thread's TLS via "Ptr".  When I compile with gcc, the program
prints
"50" as expected.  When I compile with LLVM, the program prints
"0".  This
is confirmed in the following three versions of LLVM:
* the 2.9 release
* whatever version of LLVM is driving http://llvm.org/demo/index.cgi
* svn revision 167568 on trunk (this was the most-recent revision as of a
few hours ago)

I've attached the optimized bytecode in test.ll.  This was produced by
http://llvm.org/demo/index.cgi with "LTO" checked.  You can see the
bug in
main(), where LLVM has optimized the load "*Ptr" into the following
instructions:

  %.b = load i1* @Foo.b, align 1   ; main() loads its own @Foo.b, not the
@Foo.b written by run()
  %5 = select i1 %.b, i32 50, i32 0

My guess is that the optimizer does not realize that thread_local addresses
are not constant in the same way that global addresses are constant, since
each thread_local variable actually names N variables, one for each of N
running threads.  Thus, it's not safe to optimize across two accesses of a
thread_local variable unless it can be proven that both accesses will be
performed by the same thread.

In terms of LLVM's design, I've noticed that thread_local variables are
represented in the same way as ordinary variables (via
llvm::GlobalVariable) except that the "isThreadLocal" flag is true. 
This
strikes me as a potential for confusion, because you have this one corner
case -- thread_locals -- in which an "llvm::Constant" is not really a
"constant" in the same way as other constants.  This might be related
to
http://llvm.org/bugs/show_bug.cgi?id=13720, and perhaps a few other bugs.

-Tom

p.s. If anyone is hit by this bug, my current workaround is to declare Ptr
volatile:
    int * volatile Ptr;

Note that if the volatile is moved under the pointer, as in the following:
    volatile int * Ptr;

then the bug reappears, as the load "*Ptr" in main() will be
incorrectly
optimized to:
    %5 = load volatile i32* @Foo, align 4
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121107/ec954dff/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.c
Type: text/x-csrc
Size: 868 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121107/ec954dff/attachment.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.ll
Type: application/octet-stream
Size: 3226 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121107/ec954dff/attachment.obj>

Hi Tom,

On Wed, Nov 7, 2012 at 11:43 PM, Tom Bergan <tbergan at cs.washington.edu>
wrote:
> Hello,
>
> I apologize if this has already been fixed or reported.  I believe there is
> a bug in the way the optimizer deals with thread_local variables.  The
> attached program, test.c, has a thread-local variable "int Foo"
and a global
> variable "int *Ptr".  The program takes the following steps:
>
> 1) The main thread spawns a new thread and waits
> 2) The new thread writes Foo = 50 and Ptr = &Foo, then signals the main
> thread and waits
> 3) The main thread prints *Ptr, releases the new thread, and exits
>
> The crux of this example is that the main thread obtains a pointer to the
> new thread's TLS via "Ptr".  When I compile with gcc, the
program prints
> "50" as expected.  When I compile with LLVM, the program prints
"0".  This
> is confirmed in the following three versions of LLVM:
> * the 2.9 release
> * whatever version of LLVM is driving http://llvm.org/demo/index.cgi
> * svn revision 167568 on trunk (this was the most-recent revision as of a
> few hours ago)
I tried your test.c, compiled with "clang -O3", but couldn't
reproduce
the error you're seeing. (I was using revison 167547).

What flags did you use when you compiled locally?

Thanks,
Hans

On Fri, Nov 9, 2012 at 11:23 AM, Hans Wennborg <hans at chromium.org>
wrote:
> Hi Tom,
>
> On Wed, Nov 7, 2012 at 11:43 PM, Tom Bergan <tbergan at
cs.washington.edu>
> wrote:
> > Hello,
> >
> > I apologize if this has already been fixed or reported.  I believe
there
> is
> > a bug in the way the optimizer deals with thread_local variables.  The
> > attached program, test.c, has a thread-local variable "int
Foo" and a
> global
> > variable "int *Ptr".  The program takes the following steps:
> >
> > 1) The main thread spawns a new thread and waits
> > 2) The new thread writes Foo = 50 and Ptr = &Foo, then signals the
main
> > thread and waits
> > 3) The main thread prints *Ptr, releases the new thread, and exits
> >
> > The crux of this example is that the main thread obtains a pointer to
the
> > new thread's TLS via "Ptr".  When I compile with gcc,
the program prints
> > "50" as expected.  When I compile with LLVM, the program
prints "0".
>  This
> > is confirmed in the following three versions of LLVM:
> > * the 2.9 release
> > * whatever version of LLVM is driving http://llvm.org/demo/index.cgi
> > * svn revision 167568 on trunk (this was the most-recent revision as
of a
> > few hours ago)
>
> I tried your test.c, compiled with "clang -O3", but couldn't
reproduce
> the error you're seeing. (I was using revison 167547).
>
> What flags did you use when you compiled locally?
>
> Thanks,
> Hans
>
Sorry for the incomplete report.  Originally, I actually compiled test.c
using LTO, via "opt -std-compile-opts -std-link-opts".  The bug
manifests
more simply if the variables "Ptr" and "Foo" are declared
static -- in this
case, the bug is demonstrated directly with "clang -O3".

Attached are three files:
* test.c, which is the same as the old test.c, but with "Ptr" and
"Foo"
declared static
* test.0.ll, which was built with "clang -emit-llvm -S -O0 test.c -o
test.0.ll"
* test.3.ll, which was built with "clang -emit-llvm -S -O3 test.c -o
test.3.ll"

To demonstrate the bug (verified with revision 167568):
$ clang -O3 -lpthread test.c -o test
$ ./test   # prints "Foo: 0"

It is also pretty clear that "test.3.ll" is an incorrect optimization
of
"test.0.ll"

Thanks for looking at this!
-Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121109/bf0b8bb1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.c
Type: text/x-csrc
Size: 898 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121109/bf0b8bb1/attachment.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.0.ll
Type: application/octet-stream
Size: 3261 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121109/bf0b8bb1/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.3.ll
Type: application/octet-stream
Size: 3241 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121109/bf0b8bb1/attachment-0001.obj>