llvm dev - May 2015 - [LLVMdev] libiomp, not libgomp as default library linked with -fopenmp

Jack,

Could you, please, submit a bug report? -- including steps to
reproduce (where you got imageMagick sources, how exactly you compiled
them, etc)

Andrey


On Fri, May 1, 2015 at 3:56 PM, Jack Howarth
<howarth.mailing.lists at gmail.com> wrote:
>
>
> On Fri, May 1, 2015 at 4:45 AM, Andrey Bokhanko <andreybokhanko at
gmail.com>
> wrote:
>>
>> Chandler,
>>
>> Thanks for the reply -- I always included you in libiomp supporters
camp;
>> it is good to see I wasn't mistaken! ;-)
>>
>> On Fri, May 1, 2015 at 12:51 AM, Chandler Carruth <chandlerc at
google.com>
>> wrote:
>>>
>>> Is there no way to support libgomp here as well? I don't say
this to hold
>>> up changing the defaults in any way, just curious. =]
>>
>>
>> No, sorry. libgomp doesn't support Intel API and clang generates
Intel API
>> calls only -- as simple as that. Someday someone may implement
generation of
>> GNU API calls as well, but this is a separate big task that, IMHO,
doesn't
>> serve any real purpose -- and potentially introduces nasty GPL-related
legal
>> issues.
>>
>> There is an option to choose what library clang links
>> (-fopenmp={libiomp|libgomp}), though.
>>
>>>
>>> I totally agree, I think things are way better now. I generally
support
>>> the direction. I think there are a few things I'd suggest we do
as part of
>>> the process, but I think these are really small and just about
"how" we
>>> switch.
>>>
>>> 1) I completely agree with the comments some others have made about
us
>>> needing to make it clear that this isn't some Intel-only thing,
its the LLVM
>>> OpenMP runtime. Some suggestions that I think would make sense to
help here:
>>> - I agree with finding some non-Intel folks to add as explicit code
>>> owners. I don't know who has been sufficiently involved, but if
Hal makes
>>> sense, awesome.
>>
>>
>> This really belongs to a separate thread
>> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-April/085037.html);
see my
>> answer there in a couple of minutes.
>>
>>>
>>> - Clearly updating the readme and such would be appropriate.
>>> - I suspect we should change the name of the installed library.
'libiomp'
>>> is pretty clearly the Intel library. We could continue in the grand
>>> tradition of LLVM naming conventions and use 'libllomp'? Of
course, we
>>> should install symlinks under the name 'libiomp' if needed
for existing
>>> users to not be broken.
>>> - Any other changes?
>>
>>
>> Adding openmp-dev list (in retrospect, should have been done at the
very
>> start...), Jim Cownie and Andrey Churbanov.
>>
>>>
>>> 2) I think we need to update the instructions for checking out LLVM
and
>>> all the tools to include checking out the openmp project. I'm
planning to
>>> try it out in a bit.
>>
>>
>> Cool! Thank you!
>>
>>>
>>> 3) It would be nice to get at least one boring benchmark into the
>>> test-suite that uses OpenMP just so there's more coverage that
the basic
>>> stuff all works. In particular, if we could get the benchmark that
Phoronix
>>> and others keep pointing at, that'd be nice.
>>
>>
>>> Speaking of which, have you checked the performance of some of the
basic
>>> benchmarks using OpenMP with the two runtimes? Or looked at Clang
vs GCC
>>> there? I'd be interested to see the numbers.
>>
>>
>> This is very tricky for me -- I'm employed by a CPU vendor (Intel),
and we
>> have very strict rules and long processes for publishing benchmark
results.
>> I simply can't run a benchmark and say: "hey! clang has this
number and gcc
>> has that number".
>>
>> The only thing I can share is that we do tested SPEC OMP2012
>> (https://www.spec.org/omp2012/), which is the industry standard for OMP
>> benchmarks, on a non-server class Darwin machine, and the results are
quite
>> good and comparable with other compilers.
>>
>> Speaking on Phoronix, two benchmarks where clang always lose due to
lack
>> of OpenMP are "John the Ripper"
>>
(http://www.phoronix.com/scan.php?page=article&item=clang-gcc-broadwell&num=3)
>> and ImageMagick -- though latter is not included in most recent
"clang vs
>> gcc" comparison.
>
>
> Unfortunately, the current llvm 3.7svn and available OPENMP patches
don't
> allow for imageMagick-6.8.2 to compile. The build fails on the compiler
> error...
>
> magick/attribute.c:331:12: error: loop iteration variable in the associated
> loop of 'omp parallel for' directive may not be threadprivate or
thread
> local, predetermined as
>       private
>       for (i=0; i < (ssize_t) image->colors; i++)
>            ^
> magick/attribute.c:325:9: note: loop iteration variable is predetermined as
> linear
>         i;
>         ^
> magick/attribute.c:978:12: error: loop iteration variable in the associated
> loop of 'omp parallel for' directive may not be threadprivate or
thread
> local, predetermined as
>       private
>       for (i=0; i < (ssize_t) image->colors; i++)
>            ^
> magick/attribute.c:972:9: note: loop iteration variable is predetermined as
> linear
>         i;
>         ^
> 2 errors generated.
>
> I get the same error when I try to build the latest ImageMagick 6.9.1-2
> sources with clang 3.7svn using -fopenmp=libiomp5 -Xclang
-fopenmp=libiomp5.
>
>>
>> Is there a generous soul (not employed by a CPU vendor :-)) willing to
run
>> "John the Ripper" with "clang -fopenmp=libiomp5 -Xclang
-fopenmp=libiomp5
>> -lm -O3" and compare results with "clang -O3"?
>>
>> Also, Jack Howarth did testing with some other benchmarks, and it is
nice
>> to see that clang + libiomp compare quite well (to say it mildly ;-))
with
>> gcc + libgomp!
>>
>> Andrey
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>

On Sat, May 2, 2015 at 4:27 PM, Andrey Bokhanko <andreybokhanko at
gmail.com>
wrote:
> Jack,
>
> Could you, please, submit a bug report? -- including steps to
> reproduce (where you got imageMagick sources, how exactly you compiled
> them, etc)
>
>
Andrey,
       I opened...

https://llvm.org/bugs/show_bug.cgi?id=23387

This issue appears many times during the imagemagick build where the
register type being used in declaring loop iteration variables. The
preprocessed source attached to the bug report can be used to reproduce the
compiler error with the command...

# clang-3.7 -DHAVE_CONFIG_H -DMAGICKCORE_HDRI_ENABLE=1
-DMAGICKCORE_QUANTUM_DEPTH=16 -O3 -g -fopenmp=libiomp5 -Xclang
-fopenmp=libiomp5 -Wall -fexceptions -D_FORTIFY_SOURCE=0 -D_THREAD_SAFE
-DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -c attribute.i

where 'clang-3.7' is whatever you call your local clang 3.7svn compiler.
        Hopefully that should suffice to debug this (unless you are missing
some more entries in the openmp headers and the preprocessed source needs
to be regenerated).
                   Jack
ps This issue can be hacked around for now by removing usage the
'register'
type on the offending local iteration variable declarations which exposed a
second blocker in the imagemagick build...

https://llvm.org/bugs/show_bug.cgi?id=23392

due to a compiler ICE. The problem can be reproduced with the preprocessed
source attached to that report and the command...

clang-3.7 -DHAVE_CONFIG_H -DMAGICKCORE_HDRI_ENABLE=1
-DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp=libiomp5 -O2 -g -Xclang
-fopenmp=libiomp5 -L/sw/opt/llvm-3.7.0/lib -Wall -march=penryn -fexceptions
-D_FORTIFY_SOURCE=0 -D_THREAD_SAFE -pthread -DMAGICKCORE_HDRI_ENABLE=1
-DMAGICKCORE_QUANTUM_DEPTH=16 -c feature.i -fno-common -DPIC

The ICE smells like a similar issue with static support since the
compilation warnings include...

magick/feature.c:317:60: warning: extra tokens at the end of '#pragma omp
parallel for' are ignored [-Wextra-tokens] #pragma omp parallel for
schedule(static,4) shared(status)
magick_threads(edge_image,edge_image,edge_image->rows,1)
magick/feature.c:429:60: warning: extra tokens at the end of '#pragma omp
parallel for' are ignored [-Wextra-tokens] #pragma omp parallel for
schedule(static,4) shared(status)
magick_threads(edge_image,edge_image,edge_image->rows,1)


Andrey,
>        I opened...
>
> https://llvm.org/bugs/show_bug.cgi?id=23387
>
> for this issue (which appears many in the imagemagick build where the
> register type being used
> in declaring loop iteration variables. I also uploaded the preprocessed
> source so the issue can be
> reproduced with a command...
>
> # clang-3.7 -DHAVE_CONFIG_H -DMAGICKCORE_HDRI_ENABLE=1
> -DMAGICKCORE_QUANTUM_DEPTH=16 -O3 -g -fopenmp=libiomp5 -Xclang
> -fopenmp=libiomp5 -Wall -fexceptions -D_FORTIFY_SOURCE=0 -D_THREAD_SAFE
> -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16 -c attribute.i
>
> where 'clang-3.7' is whatever you call your local clang 3.7svn
compiler.
>         Hopefully that should suffice to debug this (unless you are
> missing some more entries in the openmp headers).
>                    Jack
> ps This issue can be hacked around for now by removing usage the
> 'register' type on the offending local iteration variable
declarations.
> This exposed a second blocker in the imagemagick build...
>
> https://llvm.org/bugs/show_bug.cgi?id=23392
>
> due to a compiler ICE. The problem can be reproduced with the preprocessed
> source attached to that report and the command...
>
> # clang-3.7 -DHAVE_CONFIG_H -DMAGICKCORE_HDRI_ENABLE=1
> -DMAGICKCORE_QUANTUM_DEPTH=16 -fopenmp=libiomp5 -O2 -g -Xclang
> -fopenmp=libiomp5 -L/sw/opt/llvm-3.7.0/lib -Wall -march=penryn -fexceptions
> -D_FORTIFY_SOURCE=0 -D_THREAD_SAFE -pthread -DMAGICKCORE_HDRI_ENABLE=1
> -DMAGICKCORE_QUANTUM_DEPTH=16 -c feature.i -fno-common -DPICAndrey
>
>
> On Fri, May 1, 2015 at 3:56 PM, Jack Howarth
> <howarth.mailing.lists at gmail.com> wrote:
> >
> >
> > On Fri, May 1, 2015 at 4:45 AM, Andrey Bokhanko <
> andreybokhanko at gmail.com>
> > wrote:
> >>
> >> Chandler,
> >>
> >> Thanks for the reply -- I always included you in libiomp
supporters
> camp;
> >> it is good to see I wasn't mistaken! ;-)
> >>
> >> On Fri, May 1, 2015 at 12:51 AM, Chandler Carruth <chandlerc at
google.com
> >
> >> wrote:
> >>>
> >>> Is there no way to support libgomp here as well? I don't
say this to
> hold
> >>> up changing the defaults in any way, just curious. =]
> >>
> >>
> >> No, sorry. libgomp doesn't support Intel API and clang
generates Intel
> API
> >> calls only -- as simple as that. Someday someone may implement
> generation of
> >> GNU API calls as well, but this is a separate big task that, IMHO,
> doesn't
> >> serve any real purpose -- and potentially introduces nasty
GPL-related
> legal
> >> issues.
> >>
> >> There is an option to choose what library clang links
> >> (-fopenmp={libiomp|libgomp}), though.
> >>
> >>>
> >>> I totally agree, I think things are way better now. I
generally support
> >>> the direction. I think there are a few things I'd suggest
we do as
> part of
> >>> the process, but I think these are really small and just about
"how" we
> >>> switch.
> >>>
> >>> 1) I completely agree with the comments some others have made
about us
> >>> needing to make it clear that this isn't some Intel-only
thing, its
> the LLVM
> >>> OpenMP runtime. Some suggestions that I think would make sense
to help
> here:
> >>> - I agree with finding some non-Intel folks to add as explicit
code
> >>> owners. I don't know who has been sufficiently involved,
but if Hal
> makes
> >>> sense, awesome.
> >>
> >>
> >> This really belongs to a separate thread
> >>
(http://lists.cs.uiuc.edu/pipermail/llvmdev/2015-April/085037.html);
> see my
> >> answer there in a couple of minutes.
> >>
> >>>
> >>> - Clearly updating the readme and such would be appropriate.
> >>> - I suspect we should change the name of the installed
library.
> 'libiomp'
> >>> is pretty clearly the Intel library. We could continue in the
grand
> >>> tradition of LLVM naming conventions and use
'libllomp'? Of course, we
> >>> should install symlinks under the name 'libiomp' if
needed for existing
> >>> users to not be broken.
> >>> - Any other changes?
> >>
> >>
> >> Adding openmp-dev list (in retrospect, should have been done at
the very
> >> start...), Jim Cownie and Andrey Churbanov.
> >>
> >>>
> >>> 2) I think we need to update the instructions for checking out
LLVM and
> >>> all the tools to include checking out the openmp project.
I'm planning
> to
> >>> try it out in a bit.
> >>
> >>
> >> Cool! Thank you!
> >>
> >>>
> >>> 3) It would be nice to get at least one boring benchmark into
the
> >>> test-suite that uses OpenMP just so there's more coverage
that the
> basic
> >>> stuff all works. In particular, if we could get the benchmark
that
> Phoronix
> >>> and others keep pointing at, that'd be nice.
> >>
> >>
> >>> Speaking of which, have you checked the performance of some of
the
> basic
> >>> benchmarks using OpenMP with the two runtimes? Or looked at
Clang vs
> GCC
> >>> there? I'd be interested to see the numbers.
> >>
> >>
> >> This is very tricky for me -- I'm employed by a CPU vendor
(Intel), and
> we
> >> have very strict rules and long processes for publishing benchmark
> results.
> >> I simply can't run a benchmark and say: "hey! clang has
this number and
> gcc
> >> has that number".
> >>
> >> The only thing I can share is that we do tested SPEC OMP2012
> >> (https://www.spec.org/omp2012/), which is the industry standard
for OMP
> >> benchmarks, on a non-server class Darwin machine, and the results
are
> quite
> >> good and comparable with other compilers.
> >>
> >> Speaking on Phoronix, two benchmarks where clang always lose due
to lack
> >> of OpenMP are "John the Ripper"
> >> (
>
http://www.phoronix.com/scan.php?page=article&item=clang-gcc-broadwell&num=3
> )
> >> and ImageMagick -- though latter is not included in most recent
"clang
> vs
> >> gcc" comparison.
> >
> >
> > Unfortunately, the current llvm 3.7svn and available OPENMP patches
don't
> > allow for imageMagick-6.8.2 to compile. The build fails on the
compiler
> > error...
> >
> > magick/attribute.c:331:12: error: loop iteration variable in the
> associated
> > loop of 'omp parallel for' directive may not be threadprivate
or thread
> > local, predetermined as
> >       private
> >       for (i=0; i < (ssize_t) image->colors; i++)
> >            ^
> > magick/attribute.c:325:9: note: loop iteration variable is
predetermined
> as
> > linear
> >         i;
> >         ^
> > magick/attribute.c:978:12: error: loop iteration variable in the
> associated
> > loop of 'omp parallel for' directive may not be threadprivate
or thread
> > local, predetermined as
> >       private
> >       for (i=0; i < (ssize_t) image->colors; i++)
> >            ^
> > magick/attribute.c:972:9: note: loop iteration variable is
predetermined
> as
> > linear
> >         i;
> >         ^
> > 2 errors generated.
> >
> > I get the same error when I try to build the latest ImageMagick
6.9.1-2
> > sources with clang 3.7svn using -fopenmp=libiomp5 -Xclang
> -fopenmp=libiomp5.
> >
> >>
> >> Is there a generous soul (not employed by a CPU vendor :-))
willing to
> run
> >> "John the Ripper" with "clang -fopenmp=libiomp5
-Xclang
> -fopenmp=libiomp5
> >> -lm -O3" and compare results with "clang -O3"?
> >>
> >> Also, Jack Howarth did testing with some other benchmarks, and it
is
> nice
> >> to see that clang + libiomp compare quite well (to say it mildly
;-))
> with
> >> gcc + libgomp!
> >>
> >> Andrey
> >>
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150502/c9701051/attachment.html>

On a positive note, current llvm 3.7svn with the two outstanding OPENMP
patches applied builds the openmp support in gromacs 5.0.4 and the
resulting build fully passes the gromacs regression test suite. Tested on
x86_64-apple-darwin14.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150502/59b58a23/attachment.html>

A couple more data points. Current llvm 3.7svn with the two outstanding
OPENMP patches can build the openmp support in gdl 0.9.5 (which completely
passes its test suite) and apbs 1.4.1's limited openmp support.

On Sat, May 2, 2015 at 11:11 PM, Jack Howarth <
howarth.mailing.lists at gmail.com> wrote:
>     On a positive note, current llvm 3.7svn with the two outstanding
> OPENMP patches applied builds the openmp support in gromacs 5.0.4 and the
> resulting build fully passes the gromacs regression test suite. Tested on
> x86_64-apple-darwin14.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150503/b6dd7bb8/attachment.html>