llvm dev - Jan 2020 - LLVM build performance with LLVM

Yes, exactly.

On Sun, Jan 5, 2020 at 23:43 Mehdi AMINI <joker.eph at gmail.com> wrote:
>
>
> On Fri, Jan 3, 2020 at 8:58 PM Itaru Kitayama via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> I have just tried a Release build of LLVM but
>> with
>> LIBOMPTARGET_ENABLE_DEBUG enabled. The app build became super fast, but
I
>> can not obtain debugging information that I need at runtime.
>>
>
> The description for this flag seems to be: LIBOMPTARGET_ENABLE_DEBUG
> "Allow debug output with the environment variable
LIBOMPTARGET_DEBUG=1"
>
> Is this what you are looking for? Or are you looking into getting debug
> info (Dwarf) for the OpenMP runtime itself?
>
> --
> Mehdi
>
>
>
>>
>> On Sat, Jan 4, 2020 at 5:03 David Blaikie <dblaikie at gmail.com>
wrote:
>>
>>> I'm still confused by that - whether or not the LLVM you built
has debug
>>> info in it shouldn't at all change what goes into the binaries
that LLVM
>>> produces (modulo a few bugs, but nothing that'd produce drastic
performance
>>> swings). You mention in the bug that it's
>>> specifically LIBOMPTARGET_ENABLE_DEBUG that is slowing down your
>>> compilation time? Not the use of a Debug build of LLVM? Have you
tried a
>>> Release build of LLVM but with LIBOMPTARGET_ENABLE_DEBUG enabled?
>>>
>>> On Fri, Jan 3, 2020 at 11:58 AM Itaru Kitayama <itaru.kitayama
at gmail.com>
>>> wrote:
>>>
>>>> At least, to obtain enough information from libomptarget while
running
>>>> my offloading app on GPU capable environment, I have to build
it with an
>>>> LLVM which was built in Debug mode.
>>>>
>>>> On Sat, Jan 4, 2020 at 4:34 David Blaikie <dblaikie at
gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Jan 2, 2020 at 10:55 PM Itaru Kitayama <
>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>
>>>>>> As I am facing the serious super slow down:
>>>>>> https://bugs.llvm.org/show_bug.cgi?id=44407 , I'd
like to
>>>>>> learn more about how it can be minimized. When
debugging an app,
>>>>>> there are times building it with a
>>>>>> Debug build LLVM can not be avoided, but a situation
like 40 times
>>>>>> time increase (40 minutes) is hard
>>>>>> to work with.
>>>>>>
>>>>>
>>>>> Not sure I follow - you shouldn't need a debug build of
clang to debug
>>>>> an application built with clang. You'd use a release
build of clang to
>>>>> build your application with debug info (by passing -g).
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 3, 2020 at 11:14 AM David Blaikie
<dblaikie at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 2, 2020 at 6:09 PM Itaru Kitayama <
>>>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>>>
>>>>>>>> David,
>>>>>>>> Yes, I was indeed trying to build LLVM with a
Debug build. I'll
>>>>>>>> stop doing that from now on.
>>>>>>>>
>>>>>>>
>>>>>>> I mean, you can - it's just going to be super
slow & mostly not what
>>>>>>> you want, unless you're trying to debug that
building process. Usually
>>>>>>> you'd want to isolate one particular test case,
maybe even reduce it,
>>>>>>> before running it with a debug build of LLVM which
will be quite slow
>>>>>>> (because it's not optimized at all).
>>>>>>>
>>>>>>>
>>>>>>>> I am on JSC's JURON machine which has 251
GB of memory on the login
>>>>>>>> node, that's more than sufficient
>>>>>>>> to do a build, I suppose, and the linker is LLD
as LLVM has a CMake
>>>>>>>> variable to select the linker.
>>>>>>>>
>>>>>>>
>>>>>>> Yep, sounds like you're just tripping over the
fact that an
>>>>>>> unoptimized/debug build of LLVM is very slow.
*thumbs up*
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 3, 2020 at 11:02 AM David Blaikie
<dblaikie at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Were you trying to use a Debug build to
build LLVM? Yes, that
>>>>>>>>> would be very slow.
>>>>>>>>>
>>>>>>>>> If you mean you were using a release build
of LLVM to build a
>>>>>>>>> Debug build of LLVM - yeah, that's
generally going to be recommended. Did
>>>>>>>>> this get slower/change significantly in
performance? Many people have
>>>>>>>>> trouble with building Debug builds (no
matter the host compiler) especially
>>>>>>>>> if they're using bfd-ld, since it's
quite slow/uses a lot of memory. There
>>>>>>>>> are a few other issues to do with memory
usage (do you have less than about
>>>>>>>>> a GB of RAM per CPU? Then you'll
probably hit swapping by default & have a
>>>>>>>>> bad time - there are ways around that)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 2, 2020 at 5:42 PM Itaru
Kitayama via llvm-dev <
>>>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> When building LLVM, is it always
recommended to do it with the
>>>>>>>>>> latest official release, currently, it
is 9.0.1? I ask because
>>>>>>>>>> when I tried
>>>>>>>>>> it with a Debug build, it took an
enormous amount of time on
>>>>>>>>>> POWER8.
>>>>>>>>>>
_______________________________________________
>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>>>
>>>>>>>>>
_______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200106/fabb1a62/attachment.html>

Ah, OK, sounds lik LIBOMPTARGET_ENABLE_DEBUG doesn't have anything to do
with your problem, then. Bit of a red herring.

If the LLVM build doesn't support /just/ building the runtime with debug
info (if this is a normal/common scenario, to need to debug into the
runtime library, perhaps it should - but I Have no idea if that's a normal
use case), then you can probably mix-and-match between two builds. Build
the runtime with debug info and drop it into an installation of the
compiler built in release mode?

On Sun, Jan 5, 2020 at 11:39 AM Itaru Kitayama <itaru.kitayama at
gmail.com>
wrote:
> Yes, exactly.
>
> On Sun, Jan 5, 2020 at 23:43 Mehdi AMINI <joker.eph at gmail.com>
wrote:
>
>>
>>
>> On Fri, Jan 3, 2020 at 8:58 PM Itaru Kitayama via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> I have just tried a Release build of LLVM but
>>> with
>>> LIBOMPTARGET_ENABLE_DEBUG enabled. The app build became super fast,
but
>>> I can not obtain debugging information that I need at runtime.
>>>
>>
>> The description for this flag seems to be: LIBOMPTARGET_ENABLE_DEBUG
>> "Allow debug output with the environment variable
LIBOMPTARGET_DEBUG=1"
>>
>> Is this what you are looking for? Or are you looking into getting debug
>> info (Dwarf) for the OpenMP runtime itself?
>>
>> --
>> Mehdi
>>
>>
>>
>>>
>>> On Sat, Jan 4, 2020 at 5:03 David Blaikie <dblaikie at
gmail.com> wrote:
>>>
>>>> I'm still confused by that - whether or not the LLVM you
built has
>>>> debug info in it shouldn't at all change what goes into the
binaries that
>>>> LLVM produces (modulo a few bugs, but nothing that'd
produce drastic
>>>> performance swings). You mention in the bug that it's
>>>> specifically LIBOMPTARGET_ENABLE_DEBUG that is slowing down
your
>>>> compilation time? Not the use of a Debug build of LLVM? Have
you tried a
>>>> Release build of LLVM but with LIBOMPTARGET_ENABLE_DEBUG
enabled?
>>>>
>>>> On Fri, Jan 3, 2020 at 11:58 AM Itaru Kitayama <
>>>> itaru.kitayama at gmail.com> wrote:
>>>>
>>>>> At least, to obtain enough information from libomptarget
while running
>>>>> my offloading app on GPU capable environment, I have to
build it with an
>>>>> LLVM which was built in Debug mode.
>>>>>
>>>>> On Sat, Jan 4, 2020 at 4:34 David Blaikie <dblaikie at
gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 2, 2020 at 10:55 PM Itaru Kitayama <
>>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>>
>>>>>>> As I am facing the serious super slow down:
>>>>>>> https://bugs.llvm.org/show_bug.cgi?id=44407 ,
I'd like to
>>>>>>> learn more about how it can be minimized. When
debugging an app,
>>>>>>> there are times building it with a
>>>>>>> Debug build LLVM can not be avoided, but a
situation like 40 times
>>>>>>> time increase (40 minutes) is hard
>>>>>>> to work with.
>>>>>>>
>>>>>>
>>>>>> Not sure I follow - you shouldn't need a debug
build of clang to
>>>>>> debug an application built with clang. You'd use a
release build of clang
>>>>>> to build your application with debug info (by passing
-g).
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 3, 2020 at 11:14 AM David Blaikie
<dblaikie at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 2, 2020 at 6:09 PM Itaru Kitayama
<
>>>>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> David,
>>>>>>>>> Yes, I was indeed trying to build LLVM with
a Debug build. I'll
>>>>>>>>> stop doing that from now on.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I mean, you can - it's just going to be
super slow & mostly not
>>>>>>>> what you want, unless you're trying to
debug that building process. Usually
>>>>>>>> you'd want to isolate one particular test
case, maybe even reduce it,
>>>>>>>> before running it with a debug build of LLVM
which will be quite slow
>>>>>>>> (because it's not optimized at all).
>>>>>>>>
>>>>>>>>
>>>>>>>>> I am on JSC's JURON machine which has
251 GB of memory on the
>>>>>>>>> login node, that's more than sufficient
>>>>>>>>> to do a build, I suppose, and the linker is
LLD as LLVM has a
>>>>>>>>> CMake variable to select the linker.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yep, sounds like you're just tripping over
the fact that an
>>>>>>>> unoptimized/debug build of LLVM is very slow.
*thumbs up*
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jan 3, 2020 at 11:02 AM David
Blaikie <dblaikie at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Were you trying to use a Debug build to
build LLVM? Yes, that
>>>>>>>>>> would be very slow.
>>>>>>>>>>
>>>>>>>>>> If you mean you were using a release
build of LLVM to build a
>>>>>>>>>> Debug build of LLVM - yeah, that's
generally going to be recommended. Did
>>>>>>>>>> this get slower/change significantly in
performance? Many people have
>>>>>>>>>> trouble with building Debug builds (no
matter the host compiler) especially
>>>>>>>>>> if they're using bfd-ld, since
it's quite slow/uses a lot of memory. There
>>>>>>>>>> are a few other issues to do with
memory usage (do you have less than about
>>>>>>>>>> a GB of RAM per CPU? Then you'll
probably hit swapping by default & have a
>>>>>>>>>> bad time - there are ways around that)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 2, 2020 at 5:42 PM Itaru
Kitayama via llvm-dev <
>>>>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> When building LLVM, is it always
recommended to do it with the
>>>>>>>>>>> latest official release, currently,
it is 9.0.1? I ask because
>>>>>>>>>>> when I tried
>>>>>>>>>>> it with a Debug build, it took an
enormous amount of time on
>>>>>>>>>>> POWER8.
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>>>>
>>>>>>>>>>
_______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200105/591c77da/attachment.html>

I guess I like what you described, but that's also beyond my toolchain
build skills.

All,
Is it thought as acceptable for a user seeing 40 minutes of app build time
if he is using
a "Debug" Clang? While, with a "Release" Clang, it just
finishes within a
minute.


On Mon, Jan 6, 2020 at 5:15 AM David Blaikie <dblaikie at gmail.com>
wrote:
> Ah, OK, sounds lik LIBOMPTARGET_ENABLE_DEBUG doesn't have anything to
do
> with your problem, then. Bit of a red herring.
>
> If the LLVM build doesn't support /just/ building the runtime with
debug
> info (if this is a normal/common scenario, to need to debug into the
> runtime library, perhaps it should - but I Have no idea if that's a
normal
> use case), then you can probably mix-and-match between two builds. Build
> the runtime with debug info and drop it into an installation of the
> compiler built in release mode?
>
> On Sun, Jan 5, 2020 at 11:39 AM Itaru Kitayama <itaru.kitayama at
gmail.com>
> wrote:
>
>> Yes, exactly.
>>
>> On Sun, Jan 5, 2020 at 23:43 Mehdi AMINI <joker.eph at gmail.com>
wrote:
>>
>>>
>>>
>>> On Fri, Jan 3, 2020 at 8:58 PM Itaru Kitayama via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> I have just tried a Release build of LLVM but
>>>> with
>>>> LIBOMPTARGET_ENABLE_DEBUG enabled. The app build became super
fast, but
>>>> I can not obtain debugging information that I need at runtime.
>>>>
>>>
>>> The description for this flag seems to be:
LIBOMPTARGET_ENABLE_DEBUG
>>> "Allow debug output with the environment variable
LIBOMPTARGET_DEBUG=1"
>>>
>>> Is this what you are looking for? Or are you looking into getting
debug
>>> info (Dwarf) for the OpenMP runtime itself?
>>>
>>> --
>>> Mehdi
>>>
>>>
>>>
>>>>
>>>> On Sat, Jan 4, 2020 at 5:03 David Blaikie <dblaikie at
gmail.com> wrote:
>>>>
>>>>> I'm still confused by that - whether or not the LLVM
you built has
>>>>> debug info in it shouldn't at all change what goes into
the binaries that
>>>>> LLVM produces (modulo a few bugs, but nothing that'd
produce drastic
>>>>> performance swings). You mention in the bug that it's
>>>>> specifically LIBOMPTARGET_ENABLE_DEBUG that is slowing down
your
>>>>> compilation time? Not the use of a Debug build of LLVM?
Have you tried a
>>>>> Release build of LLVM but with LIBOMPTARGET_ENABLE_DEBUG
enabled?
>>>>>
>>>>> On Fri, Jan 3, 2020 at 11:58 AM Itaru Kitayama <
>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>
>>>>>> At least, to obtain enough information from
libomptarget while
>>>>>> running my offloading app on GPU capable environment, I
have to build it
>>>>>> with an LLVM which was built in Debug mode.
>>>>>>
>>>>>> On Sat, Jan 4, 2020 at 4:34 David Blaikie <dblaikie
at gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 2, 2020 at 10:55 PM Itaru Kitayama <
>>>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>>>
>>>>>>>> As I am facing the serious super slow down:
>>>>>>>> https://bugs.llvm.org/show_bug.cgi?id=44407 ,
I'd like to
>>>>>>>> learn more about how it can be minimized. When
debugging an app,
>>>>>>>> there are times building it with a
>>>>>>>> Debug build LLVM can not be avoided, but a
situation like 40 times
>>>>>>>> time increase (40 minutes) is hard
>>>>>>>> to work with.
>>>>>>>>
>>>>>>>
>>>>>>> Not sure I follow - you shouldn't need a debug
build of clang to
>>>>>>> debug an application built with clang. You'd
use a release build of clang
>>>>>>> to build your application with debug info (by
passing -g).
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 3, 2020 at 11:14 AM David Blaikie
<dblaikie at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 2, 2020 at 6:09 PM Itaru
Kitayama <
>>>>>>>>> itaru.kitayama at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> David,
>>>>>>>>>> Yes, I was indeed trying to build LLVM
with a Debug build. I'll
>>>>>>>>>> stop doing that from now on.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I mean, you can - it's just going to be
super slow & mostly not
>>>>>>>>> what you want, unless you're trying to
debug that building process. Usually
>>>>>>>>> you'd want to isolate one particular
test case, maybe even reduce it,
>>>>>>>>> before running it with a debug build of
LLVM which will be quite slow
>>>>>>>>> (because it's not optimized at all).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I am on JSC's JURON machine which
has 251 GB of memory on the
>>>>>>>>>> login node, that's more than
sufficient
>>>>>>>>>> to do a build, I suppose, and the
linker is LLD as LLVM has a
>>>>>>>>>> CMake variable to select the linker.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yep, sounds like you're just tripping
over the fact that an
>>>>>>>>> unoptimized/debug build of LLVM is very
slow. *thumbs up*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 3, 2020 at 11:02 AM David
Blaikie <dblaikie at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Were you trying to use a Debug
build to build LLVM? Yes, that
>>>>>>>>>>> would be very slow.
>>>>>>>>>>>
>>>>>>>>>>> If you mean you were using a
release build of LLVM to build a
>>>>>>>>>>> Debug build of LLVM - yeah,
that's generally going to be recommended. Did
>>>>>>>>>>> this get slower/change
significantly in performance? Many people have
>>>>>>>>>>> trouble with building Debug builds
(no matter the host compiler) especially
>>>>>>>>>>> if they're using bfd-ld, since
it's quite slow/uses a lot of memory. There
>>>>>>>>>>> are a few other issues to do with
memory usage (do you have less than about
>>>>>>>>>>> a GB of RAM per CPU? Then
you'll probably hit swapping by default & have a
>>>>>>>>>>> bad time - there are ways around
that)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 2, 2020 at 5:42 PM
Itaru Kitayama via llvm-dev <
>>>>>>>>>>> llvm-dev at lists.llvm.org>
wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> When building LLVM, is it
always recommended to do it with the
>>>>>>>>>>>> latest official release,
currently, it is 9.0.1? I ask because
>>>>>>>>>>>> when I tried
>>>>>>>>>>>> it with a Debug build, it took
an enormous amount of time on
>>>>>>>>>>>> POWER8.
>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>>>>>
>>>>>>>>>>>
_______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200106/82e70dc0/attachment.html>