llvm dev - Dec 2018 - Disabling LLVM_ATTRIBUTE_ALWAYS_INLINE for development?

Hello all!

I find that using lldb to debug LLVM libraries can be super
frustrating, because a lot of LLVM classes, like the constructor for
StringRef, are marked LLVM_ATTRIBUTE_ALWAYS_INLINE. So when I attempt
to have lldb evaluate an expression that implicitly instantiates a
StringRef, I get 'error: Couldn't lookup symbols:
__ZN4llvm9StringRefC1EPKc'.

As an example, most recently this happened to me when playing around
with llvm::AttributeSet, having attached lldb to an opt built with
-DCMAKE_BUILD_TYPE="Debug":

   (lldb) e $AS.hasAttribute("myattr")
   error: Couldn't lookup symbols:
     __ZN4llvm9StringRefC1EPKc

Despite having built in a "Debug" configuration,
LLVM_ATTRIBUTE_ALWAYS_INLINE makes it very difficult to debug LLVM.

How do you all deal with or work around this problem? Is there a good
way to do so? If not, would anyone object if I sent up a patch to
introduce a CMake variable or something to conditionally disable
LLVM_ATTRIBUTE_ALWAYS_INLINE? I'd like to be able to turn it off, but
I'm not sure if there's some good reason it needs to stay on even for
debug builds?

- Brian Gesiak

Hi,
> On Dec 15, 2018, at 10:32 AM, Brian Gesiak via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hello all!
> 
> I find that using lldb to debug LLVM libraries can be super
> frustrating, because a lot of LLVM classes, like the constructor for
> StringRef, are marked LLVM_ATTRIBUTE_ALWAYS_INLINE. So when I attempt
> to have lldb evaluate an expression that implicitly instantiates a
> StringRef, I get 'error: Couldn't lookup symbols:
> __ZN4llvm9StringRefC1EPKc'.
> 
> As an example, most recently this happened to me when playing around
> with llvm::AttributeSet, having attached lldb to an opt built with
> -DCMAKE_BUILD_TYPE="Debug":
> 
>   (lldb) e $AS.hasAttribute("myattr")
>   error: Couldn't lookup symbols:
>     __ZN4llvm9StringRefC1EPKc
> 
> Despite having built in a "Debug" configuration,
> LLVM_ATTRIBUTE_ALWAYS_INLINE makes it very difficult to debug LLVM.
> 
> How do you all deal with or work around this problem?
I don't think there are any great workarounds. The data formatters in
utils/lldbDataFormatters.py can help a bit.

> Is there a good
> way to do so? If not, would anyone object if I sent up a patch to
> introduce a CMake variable or something to conditionally disable
> LLVM_ATTRIBUTE_ALWAYS_INLINE? I'd like to be able to turn it off, but
> I'm not sure if there's some good reason it needs to stay on even
for
> debug builds?
IIRC the motivation for using always_inline in debug builds was to make binaries
acceptably fast. IMHO this isn't the right tradeoff because it also makes
binaries frustratingly hard to debug.

Some might object that with clang modules, it should be no trouble for a
debugger to evaluate these kinds of expressions. I've CC'd Raphael who
can speak to this much better than I can. My understanding is that modules may
help, but lldb (and presumably gdb as well?) are not going to learn how to parse
templated code out of C++ modules, instantiate them, and JIT-compile them for
you really soon. That seems like an ongoing, longer-term project.

I'd like to see us move away from using always_inline in core ADT headers to
make debugging easier in the near future.

A note about slow Debug binaries. In my experience it's really never
necessary have a separate Debug build tree. It takes up a slot of time/space and
I've never found it useful. It's usually much more convenient to have a
single RelAsserts build tree. If you need to debug code in some .cpp file, you
can just:

$ touch path/to/the/file.cpp
$ export CCC_OVERRIDE_OPTIONS="+-g O0 +-UNDEBUG"
$ ninja opt/clang/etc

The build system will do an incremental recompile of the binary you want to
debug, with the *single file* (or set of files) you care about recompiled with
asserts, at -O0, with debug info. The resulting binary is pretty much just as
fast as a RelAsserts binary and is perfectly debuggable, modulo always_inline
functions ;)

vedant

> 
> - Brian Gesiak
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Sat, Dec 15, 2018 at 12:02 PM Vedant Kumar via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> > On Dec 15, 2018, at 10:32 AM, Brian Gesiak via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hello all!
> >
> > I find that using lldb to debug LLVM libraries can be super
> > frustrating, because a lot of LLVM classes, like the constructor for
> > StringRef, are marked LLVM_ATTRIBUTE_ALWAYS_INLINE. So when I attempt
> > to have lldb evaluate an expression that implicitly instantiates a
> > StringRef, I get 'error: Couldn't lookup symbols:
> > __ZN4llvm9StringRefC1EPKc'.
> >
> > As an example, most recently this happened to me when playing around
> > with llvm::AttributeSet, having attached lldb to an opt built with
> > -DCMAKE_BUILD_TYPE="Debug":
> >
> >   (lldb) e $AS.hasAttribute("myattr")
> >   error: Couldn't lookup symbols:
> >     __ZN4llvm9StringRefC1EPKc
> >
> > Despite having built in a "Debug" configuration,
> > LLVM_ATTRIBUTE_ALWAYS_INLINE makes it very difficult to debug LLVM.
> >
> > How do you all deal with or work around this problem?
>
> I don't think there are any great workarounds. The data formatters in
> utils/lldbDataFormatters.py can help a bit.
>
>
> > Is there a good
> > way to do so? If not, would anyone object if I sent up a patch to
> > introduce a CMake variable or something to conditionally disable
> > LLVM_ATTRIBUTE_ALWAYS_INLINE? I'd like to be able to turn it off,
but
> > I'm not sure if there's some good reason it needs to stay on
even for
> > debug builds?
>
> IIRC the motivation for using always_inline in debug builds was to make
> binaries acceptably fast. IMHO this isn't the right tradeoff because it
> also makes binaries frustratingly hard to debug.
>
> Some might object that with clang modules, it should be no trouble for a
> debugger to evaluate these kinds of expressions. I've CC'd Raphael
who can
> speak to this much better than I can. My understanding is that modules may
> help, but lldb (and presumably gdb as well?) are not going to learn how to
> parse templated code out of C++ modules, instantiate them, and JIT-compile
> them for you really soon. That seems like an ongoing, longer-term project.
>
> I'd like to see us move away from using always_inline in core ADT
headers
> to make debugging easier in the near future.
I just had a random thought.  Could we introduce a flag that would cause
the compiler to emit an out of line definition even for always inline
funcions?  We could enable this flag in debug builds.


>
> A note about slow Debug binaries. In my experience it's really never
> necessary have a separate Debug build tree. It takes up a slot of
> time/space and I've never found it useful. It's usually much more
> convenient to have a single RelAsserts build tree. If you need to debug
> code in some .cpp file, you can just:
>
> $ touch path/to/the/file.cpp
> $ export CCC_OVERRIDE_OPTIONS="+-g O0 +-UNDEBUG"
> $ ninja opt/clang/etc
>
> The build system will do an incremental recompile of the binary you want
> to debug, with the *single file* (or set of files) you care about
> recompiled with asserts, at -O0, with debug info. The resulting binary is
> pretty much just as fast as a RelAsserts binary and is perfectly
> debuggable, modulo always_inline functions ;)
I’m the opposite, I debug so much and across so many translation units that
doing that every single time i wanted to debug something different would be
prohibitively time consuming.  So i find RelWithAsserts unnecessary, and
almost exclusively use debug builds for everything.  The last time I built
release was several months ago.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181215/bc0d3355/attachment.html>

My understanding of `always_inline` is that the attribute is meant to be
used where not inlining a function would cause a (incorrect) change in
semantics of the function (anything that uses certain builtins or inline
assembly getting values of certain common registers like FP/SP/PC). Could
it be worth adding another macro for the attribute that is less ambiguous
in terms of whether forced inlining is a case of optimization, to
distinguish it from uses (I'm not actually sure if there are any) where it
may actually inadvertently change semantics?

On Sat, Dec 15, 2018 at 8:02 PM Vedant Kumar via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> > On Dec 15, 2018, at 10:32 AM, Brian Gesiak via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hello all!
> >
> > I find that using lldb to debug LLVM libraries can be super
> > frustrating, because a lot of LLVM classes, like the constructor for
> > StringRef, are marked LLVM_ATTRIBUTE_ALWAYS_INLINE. So when I attempt
> > to have lldb evaluate an expression that implicitly instantiates a
> > StringRef, I get 'error: Couldn't lookup symbols:
> > __ZN4llvm9StringRefC1EPKc'.
> >
> > As an example, most recently this happened to me when playing around
> > with llvm::AttributeSet, having attached lldb to an opt built with
> > -DCMAKE_BUILD_TYPE="Debug":
> >
> >   (lldb) e $AS.hasAttribute("myattr")
> >   error: Couldn't lookup symbols:
> >     __ZN4llvm9StringRefC1EPKc
> >
> > Despite having built in a "Debug" configuration,
> > LLVM_ATTRIBUTE_ALWAYS_INLINE makes it very difficult to debug LLVM.
> >
> > How do you all deal with or work around this problem?
>
> I don't think there are any great workarounds. The data formatters in
> utils/lldbDataFormatters.py can help a bit.
>
>
> > Is there a good
> > way to do so? If not, would anyone object if I sent up a patch to
> > introduce a CMake variable or something to conditionally disable
> > LLVM_ATTRIBUTE_ALWAYS_INLINE? I'd like to be able to turn it off,
but
> > I'm not sure if there's some good reason it needs to stay on
even for
> > debug builds?
>
> IIRC the motivation for using always_inline in debug builds was to make
> binaries acceptably fast. IMHO this isn't the right tradeoff because it
> also makes binaries frustratingly hard to debug.
>
> Some might object that with clang modules, it should be no trouble for a
> debugger to evaluate these kinds of expressions. I've CC'd Raphael
who can
> speak to this much better than I can. My understanding is that modules may
> help, but lldb (and presumably gdb as well?) are not going to learn how to
> parse templated code out of C++ modules, instantiate them, and JIT-compile
> them for you really soon. That seems like an ongoing, longer-term project.
>
> I'd like to see us move away from using always_inline in core ADT
headers
> to make debugging easier in the near future.
>
> A note about slow Debug binaries. In my experience it's really never
> necessary have a separate Debug build tree. It takes up a slot of
> time/space and I've never found it useful. It's usually much more
> convenient to have a single RelAsserts build tree. If you need to debug
> code in some .cpp file, you can just:
>
> $ touch path/to/the/file.cpp
> $ export CCC_OVERRIDE_OPTIONS="+-g O0 +-UNDEBUG"
> $ ninja opt/clang/etc
>
> The build system will do an incremental recompile of the binary you want
> to debug, with the *single file* (or set of files) you care about
> recompiled with asserts, at -O0, with debug info. The resulting binary is
> pretty much just as fast as a RelAsserts binary and is perfectly
> debuggable, modulo always_inline functions ;)
>
> vedant
>
>
> >
> > - Brian Gesiak
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181215/7c9d5e55/attachment.html>

On Sat, Dec 15, 2018 at 12:02 PM Vedant Kumar via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hi,
>
> > On Dec 15, 2018, at 10:32 AM, Brian Gesiak via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
> >
> > Hello all!
> >
> > I find that using lldb to debug LLVM libraries can be super
> > frustrating, because a lot of LLVM classes, like the constructor for
> > StringRef, are marked LLVM_ATTRIBUTE_ALWAYS_INLINE. So when I attempt
> > to have lldb evaluate an expression that implicitly instantiates a
> > StringRef, I get 'error: Couldn't lookup symbols:
> > __ZN4llvm9StringRefC1EPKc'.
> >
> > As an example, most recently this happened to me when playing around
> > with llvm::AttributeSet, having attached lldb to an opt built with
> > -DCMAKE_BUILD_TYPE="Debug":
> >
> >   (lldb) e $AS.hasAttribute("myattr")
> >   error: Couldn't lookup symbols:
> >     __ZN4llvm9StringRefC1EPKc
> >
> > Despite having built in a "Debug" configuration,
> > LLVM_ATTRIBUTE_ALWAYS_INLINE makes it very difficult to debug LLVM.
> >
> > How do you all deal with or work around this problem?
>
> I don't think there are any great workarounds. The data formatters in
utils/lldbDataFormatters.py can help a bit.
>
>
> > Is there a good
> > way to do so? If not, would anyone object if I sent up a patch to
> > introduce a CMake variable or something to conditionally disable
> > LLVM_ATTRIBUTE_ALWAYS_INLINE? I'd like to be able to turn it off,
but
> > I'm not sure if there's some good reason it needs to stay on
even for
> > debug builds?
>
> IIRC the motivation for using always_inline in debug builds was to make
binaries acceptably fast. IMHO this isn't the right tradeoff because it also
makes binaries frustratingly hard to debug.
>
> Some might object that with clang modules, it should be no trouble for a
debugger to evaluate these kinds of expressions. I've CC'd Raphael who
can speak to this much better than I can. My understanding is that modules may
help, but lldb (and presumably gdb as well?) are not going to learn how to parse
templated code out of C++ modules, instantiate them, and JIT-compile them for
you really soon. That seems like an ongoing, longer-term project.
>
> I'd like to see us move away from using always_inline in core ADT
headers to make debugging easier in the near future.
>
+1 to everything you said. I happened to remember this wasn't always
the case for StringRef, so I did some archeology and found:
[llvm] r247253 - [ADT] Apply a large hammer to StringRef functions:
attribute always_inline.

I'm afraid I missed the commit at the time but there are few remarks I
would like to make:
1) The motivation for this change, at least from what I understand, is
trying to optimize `check-llvm`. As somebody who spends a fair amount
of time tapping fingers while waiting for the tests to run,  I do
believe this is a laudable goal, but I'm uncertain of the results. In
fact, it seems it only saves 1 second (according to the commit
message).
2) I can't speak for anybody else, but I'm not entirely sure a
48-cores box is a common setup (I might be completely wrong on this
one, but I have [and had] access to much more modest hardware). In
other words, I'm not entirely sure the setup that drove the change is
representative of the "typical" llvm developer (for some definition of
"typical").
3) I think the tradeoff here is just not great. I personally devote
much more time debugging llvm (and related subprojects) than running
the testsuite.

YMMV.

Chandler, is there anything that can be done here to make the
experience less frustrating (i.e. revising the decision or reverting)?
The original commit pointed out this was a workaround, but the status
quo is that it makes debugging llvm itself much harder.
 Of course, happy to help.

--
Davide