llvm dev - Aug 2016 - Questions on LLVM vectorization diagnostics

Hi Dangeti, Ramakrishna, Adam, and Gerolf,
>Yes this is an area that needs further improvement.  We have some immediate
plans to make these more useful.  See the recent llvm-dev threads [1], [2].
It takes a lot of dedicated effort to make vectorization report easier to
understand by ordinary programmers
(i.e., those who are not compiler writers). Having done that for ICC ourselves,
we truly believe it was a good
investment of resource. There are areas where both expert and non-expert of
vectorizer development
can equally contribute. That includes getting the source code location right and
variable names (and memory
references) printed at the source level representation. If anyone has data on
how good LLVM is on these
areas, we'd appreciate a pointer to such information. Otherwise, we'll
study that when our development
effort hit that area, report back, and contribute for improvement. 
>> In our analysis we never seen llvm trying to vectorize outer loops. Is
this well known? Is outer loop vectorization implemented in LLVM
>>as in GCC? (http://dl.acm.org/citation.cfm?id=1454119
<http://dl.acm.org/citation.cfm?id=1454119>) If not, is someone working on
it?
>
>I heard various people mention this but I am not sure whether actual work is
already taking place.
We are currently working on introducing a next generation vectorizer design to
LLVM, aiming to support OpenMP4.5 SIMD
(i.e., including outer loop vectorization). I hope to be able to send in an RFC
on the high level design document to LLVM-DEV
next month. We are currently working on an RFC for "vectorizer's
output" (IR, not diagnostic), to be discussed before the next
gen design. As part of this next gen work, we'll also be working on
improving diagnostics. Stay tuned.
>actual work is already taking place.
Yes, our hands are dirty with actual coding work to ensure that the high level
design makes sense. :)

Thanks,
Hideki Saito (hideki dot saito at intel dot com)
Technical Lead of Vectorizer Development
Intel Compiler and Languages

-----------------------------------------------------------------------------
Message: 3
Date: Thu, 23 Jun 2016 10:45:28 -0700
From: Adam Nemet via llvm-dev <llvm-dev at lists.llvm.org>
To: Dangeti Tharun kumar <cs15mtech11002 at iith.ac.in>
Cc: llvm-dev at lists.llvm.org, Santanu Das <cs15mtech11018 at iith.ac.in>
Subject: Re: [llvm-dev] Questions on LLVM vectorization diagnostics
Message-ID: <B6F42D93-F676-4CB1-8413-A37A07490A55 at apple.com>
Content-Type: text/plain; charset="utf-8"

Hi Dangeti,
> On Jun 23, 2016, at 8:20 AM, Dangeti Tharun kumar via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
> 
> Dear LLVM Community,
> 
> I am D Tharun Kumar, masters student at Indian Institute of Technology
Hyderabad, working in a team to improve current vectorizer in LLVM. As an
initial study, we are studying various benchmarks to analyze and compare
vectorizing capabilities of LLVM, GCC and ICC. We found that vectorization
remarks given by LLVM are vague and brief, comparatively GCC and ICC are giving
detailed diagnostics.
Yes this is an area that needs further improvement.  We have some immediate
plans to make these more useful.  See the recent llvm-dev threads [1],
[2].
> I am interested to know why the LLVM diagnostics are brief and not
intuitive (making them less helpful)?
I think it’s just lack of work or weakness in the analyses to provide more
detailed information.  It would be good to file bugs for specific cases where we
fall behind.
> In our analysis we never seen llvm trying to vectorize outer loops. Is this
well known? Is outer loop vectorization implemented in LLVM as in GCC?
(http://dl.acm.org/citation.cfm?id=1454119
<http://dl.acm.org/citation.cfm?id=1454119>) If not, is someone working on
it?
I heard various people mention this but I am not sure whether actual work is
already taking place.
> On the TSVC benchmark suite, out of a total of 151 loops, LLVM, GCC and ICC
vectorized 70, 82 and 112 loops respectively. Is the cause for lag of LLVM the
inability of LLVM’s vectorizer, or are there any (enabling) optimization passes
running before GCC’s vectorizer that are helping GCC perform better?
I don’t know about the GCC but I’ve seen ICC perform loop transformation more
aggressively that can increase the coverage for loop vectorization.  ICC
performs Loop Distribution/Fusion/Interchange, etc by default at their highest
optimization level.  We have some of these passes (distribution, interchange)
but not on by default yet.

Arguably, there is also some difference between focus areas for these compilers.
I think that ICC has a more HPC focus than LLVM or GCC.  We have Polly which is
geared toward more the HPC use cases.
> Loop peeling to enhance vectorization is present in GCC and ICC, but, the
LLVM remarks don’t say anything about alignment. Does LLVM has this
functionality and the vectorizer doesn’t remark about it, or it doesn’t it have
the functionality at all?
We don’t have it.
> Finally, we appreciate suggestions and directions for improving the
vectorization framework of LLVM.
This is a pretty active area.  Probably reading up on recent llvm-dev discussion
in this area would be helpful to you.
> I would also like to know if anyone worked or is working on improving
vectorization remarks.
Yes we are.  If you’re interested working on this area it would be good to
coordinate.

Adam
> 
> Regards,
> 
> Dangeti Tharun kumar
> M.TECH Computer Science
> IIT Hyderabad
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[1] http://thread.gmane.org/gmane.comp.compilers.llvm.devel/98334
[2] http://thread.gmane.org/gmane.comp.compilers.llvm.devel/99126

Has there been a follow up? I’m very interested in specific examples underlying
the key design decisions. Specifically I expect that you have examples that have
a x% speed-up with ICC vs clang because of XYZ in your design. Similar if you
have examples for better diagnostics if probably makes sense to share them.

Thanks
Gerolf
> On Jun 24, 2016, at 12:45 AM, Saito, Hideki via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> 
> Hi Dangeti, Ramakrishna, Adam, and Gerolf,
> 
>> Yes this is an area that needs further improvement.  We have some
immediate plans to make these more useful.  See the recent llvm-dev threads [1],
[2].
> 
> It takes a lot of dedicated effort to make vectorization report easier to
understand by ordinary programmers
> (i.e., those who are not compiler writers). Having done that for ICC
ourselves, we truly believe it was a good
> investment of resource. There are areas where both expert and non-expert of
vectorizer development
> can equally contribute. That includes getting the source code location
right and variable names (and memory
> references) printed at the source level representation. If anyone has data
on how good LLVM is on these
> areas, we'd appreciate a pointer to such information. Otherwise,
we'll study that when our development
> effort hit that area, report back, and contribute for improvement. 
> 
>>> In our analysis we never seen llvm trying to vectorize outer loops.
Is this well known? Is outer loop vectorization implemented in LLVM
>>> as in GCC? (http://dl.acm.org/citation.cfm?id=1454119
<http://dl.acm.org/citation.cfm?id=1454119>) If not, is someone working on
it?
>> 
>> I heard various people mention this but I am not sure whether actual
work is already taking place.
> 
> We are currently working on introducing a next generation vectorizer design
to LLVM, aiming to support OpenMP4.5 SIMD
> (i.e., including outer loop vectorization). I hope to be able to send in an
RFC on the high level design document to LLVM-DEV
> next month. We are currently working on an RFC for "vectorizer's
output" (IR, not diagnostic), to be discussed before the next
> gen design. As part of this next gen work, we'll also be working on
improving diagnostics. Stay tuned.
> 
>> actual work is already taking place.
> 
> Yes, our hands are dirty with actual coding work to ensure that the high
level design makes sense. :)
> 
> Thanks,
> Hideki Saito (hideki dot saito at intel dot com)
> Technical Lead of Vectorizer Development
> Intel Compiler and Languages
> 
>
-----------------------------------------------------------------------------
> Message: 3
> Date: Thu, 23 Jun 2016 10:45:28 -0700
> From: Adam Nemet via llvm-dev <llvm-dev at lists.llvm.org>
> To: Dangeti Tharun kumar <cs15mtech11002 at iith.ac.in>
> Cc: llvm-dev at lists.llvm.org, Santanu Das <cs15mtech11018 at
iith.ac.in>
> Subject: Re: [llvm-dev] Questions on LLVM vectorization diagnostics
> Message-ID: <B6F42D93-F676-4CB1-8413-A37A07490A55 at apple.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hi Dangeti,
> 
>> On Jun 23, 2016, at 8:20 AM, Dangeti Tharun kumar via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>> 
>> Dear LLVM Community,
>> 
>> I am D Tharun Kumar, masters student at Indian Institute of Technology
Hyderabad, working in a team to improve current vectorizer in LLVM. As an
initial study, we are studying various benchmarks to analyze and compare
vectorizing capabilities of LLVM, GCC and ICC. We found that vectorization
remarks given by LLVM are vague and brief, comparatively GCC and ICC are giving
detailed diagnostics.
> 
> Yes this is an area that needs further improvement.  We have some immediate
plans to make these more useful.  See the recent llvm-dev threads [1], [2].
>> I am interested to know why the LLVM diagnostics are brief and not
intuitive (making them less helpful)?
> I think it’s just lack of work or weakness in the analyses to provide more
detailed information.  It would be good to file bugs for specific cases where we
fall behind.
>> In our analysis we never seen llvm trying to vectorize outer loops. Is
this well known? Is outer loop vectorization implemented in LLVM as in GCC?
(http://dl.acm.org/citation.cfm?id=1454119
<http://dl.acm.org/citation.cfm?id=1454119>) If not, is someone working on
it?
> 
> I heard various people mention this but I am not sure whether actual work
is already taking place.
> 
>> On the TSVC benchmark suite, out of a total of 151 loops, LLVM, GCC and
ICC vectorized 70, 82 and 112 loops respectively. Is the cause for lag of LLVM
the inability of LLVM’s vectorizer, or are there any (enabling) optimization
passes running before GCC’s vectorizer that are helping GCC perform better?
> 
> I don’t know about the GCC but I’ve seen ICC perform loop transformation
more aggressively that can increase the coverage for loop vectorization.  ICC
performs Loop Distribution/Fusion/Interchange, etc by default at their highest
optimization level.  We have some of these passes (distribution, interchange)
but not on by default yet.
> 
> Arguably, there is also some difference between focus areas for these
compilers.  I think that ICC has a more HPC focus than LLVM or GCC.  We have
Polly which is geared toward more the HPC use cases.
> 
>> Loop peeling to enhance vectorization is present in GCC and ICC, but,
the LLVM remarks don’t say anything about alignment. Does LLVM has this
functionality and the vectorizer doesn’t remark about it, or it doesn’t it have
the functionality at all?
> We don’t have it.
> 
>> Finally, we appreciate suggestions and directions for improving the
vectorization framework of LLVM.
> 
> This is a pretty active area.  Probably reading up on recent llvm-dev
discussion in this area would be helpful to you.
> 
>> I would also like to know if anyone worked or is working on improving
vectorization remarks.
> 
> Yes we are.  If you’re interested working on this area it would be good to
coordinate.
> 
> Adam
> 
>> 
>> Regards,
>> 
>> Dangeti Tharun kumar
>> M.TECH Computer Science
>> IIT Hyderabad
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> 
> [1] http://thread.gmane.org/gmane.comp.compilers.llvm.devel/98334
> [2] http://thread.gmane.org/gmane.comp.compilers.llvm.devel/99126
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi, Gerolf.

We've been a bit quiet for some time. After listening to feedback on the
project internally and externally,
we decided to take a more generally accepted community development model ----
building up through
a collection of small incremental changes ---- than trying to make a big step
forward. That change of course
took a bit of time, but we are getting close to the first NFC patch on which we
hope to incrementally build
up new functionalities.

Within a few weeks, we plan to send in the first of the series of RFCs, which is
soon to be followed by
the NFC patch for review as the first step. We are also making a submission for
a talk about this project,
plus a submission for a BoF about vector masking, at 2016 LLVM DEV Meeting. I
hope our submissions
will be accepted. Looking forward to have great discussions on the mailing list,
patch review process,
and in person.
>I’m very interested in specific examples underlying the key design
decisions.
Since the two paragraphs above aren't too useful in answering your
questions, let me talk about
one particular example: auto-vectorization of outer loops.

I do not know whether any readers here have noticed: ICC auto-vectorizer works
inner to outer.
If the inner loop is auto-vectorized, outer loop is no longer a vectorization
candidate. Currently,
it does not have an ability to compare the benefit of vectorizing outer loop and
the benefit of
vectorizing inner loop(s) ---- people in academia, here's a paper
opportunity. :)
Often times, outer loop vectorization requires a massaging of inner loop control
flow --- and ICC
Vectorizer does such massaging on its underlying IR level ---- just like many of
you who have
implemented OpenMP SIMD, OpenCL and other explicit vector programming model.
This is
okay when you know which loop to vectorize ahead of time. Not so nice if you are
trying to
decide between inner loop vectorization or outer loop vectorization. As such,
one of the key
design consideration was being able to "pseudo-massage inner loop control
flow" w/o modifying
the underlying IR, until the cost model decides where to vectorize.

This, by itself, is a rather ambitious project, many people advised us to go
many incremental small steps,
and we listened. That has led to the small NFC patch mentioned above.

I hope this revelation is interesting enough for many of you to stay tuned for
our further development.
I probably spoke too much about ICC vectorizer internal. One of the future RFCs
(it'll certainly take some
time to get to that point through many incremental steps) will be talking about
the inner versus outer
auto-vectorization. We hope to get there sooner than later.

---------------------------
Now, I have one question. Suppose we'd like to split the vectorization
decision as an Analysis pass and vectorization
transformation as a Transformation pass. Is it acceptable if an Analysis pass
creates new Instructions and new  BasicBlocks,
keep them unreachable from the underlying IR of the Function/Loop, and pass
those to the Transformation pass as
part of Analysis's internal data? We've been operating under the
assumption that such Analysis pass behavior is unacceptable.
Please let us know if this is a generally acceptable way for an Analysis pass to
work ---- this might make our development
move quicker. Why we'd want to do this? As mentioned above, we need to
"pseudo-massage inner loop control flow"
before deciding where/whether to vectorize. Hope someone can give us a clear
answer.

Thanks,
Hideki Saito
Intel Compilers and Languages

-----Original Message-----
From: ghoflehner at apple.com [mailto:ghoflehner at apple.com] 
Sent: Wednesday, August 24, 2016 5:38 PM
To: Saito, Hideki <hideki.saito at intel.com>
Cc: llvm-dev at lists.llvm.org; Dangeti Tharun kumar <cs15mtech11002 at
iith.ac.in>; Santanu Das <cs15mtech11018 at iith.ac.in>
Subject: Re: [llvm-dev] Questions on LLVM vectorization diagnostics

Has there been a follow up? I’m very interested in specific examples underlying
the key design decisions. Specifically I expect that you have examples that have
a x% speed-up with ICC vs clang because of XYZ in your design. Similar if you
have examples for better diagnostics if probably makes sense to share them.

Thanks
Gerolf
> On Jun 24, 2016, at 12:45 AM, Saito, Hideki via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> 
> Hi Dangeti, Ramakrishna, Adam, and Gerolf,
> 
>> Yes this is an area that needs further improvement.  We have some
immediate plans to make these more useful.  See the recent llvm-dev threads [1],
[2].
> 
> It takes a lot of dedicated effort to make vectorization report easier 
> to understand by ordinary programmers (i.e., those who are not 
> compiler writers). Having done that for ICC ourselves, we truly 
> believe it was a good investment of resource. There are areas where 
> both expert and non-expert of vectorizer development can equally 
> contribute. That includes getting the source code location right and 
> variable names (and memory
> references) printed at the source level representation. If anyone has 
> data on how good LLVM is on these areas, we'd appreciate a pointer to 
> such information. Otherwise, we'll study that when our development
effort hit that area, report back, and contribute for improvement.
> 
>>> In our analysis we never seen llvm trying to vectorize outer loops.
>>> Is this well known? Is outer loop vectorization implemented in LLVM
as in GCC? (http://dl.acm.org/citation.cfm?id=1454119
<http://dl.acm.org/citation.cfm?id=1454119>) If not, is someone working on
it?
>> 
>> I heard various people mention this but I am not sure whether actual
work is already taking place.
> 
> We are currently working on introducing a next generation vectorizer 
> design to LLVM, aiming to support OpenMP4.5 SIMD (i.e., including 
> outer loop vectorization). I hope to be able to send in an RFC on the 
> high level design document to LLVM-DEV next month. We are currently working
on an RFC for "vectorizer's output" (IR, not diagnostic), to be
discussed before the next gen design. As part of this next gen work, we'll
also be working on improving diagnostics. Stay tuned.
> 
>> actual work is already taking place.
> 
> Yes, our hands are dirty with actual coding work to ensure that the 
> high level design makes sense. :)
> 
> Thanks,
> Hideki Saito (hideki dot saito at intel dot com) Technical Lead of 
> Vectorizer Development Intel Compiler and Languages
> 
> ----------------------------------------------------------------------
> -------
> Message: 3
> Date: Thu, 23 Jun 2016 10:45:28 -0700
> From: Adam Nemet via llvm-dev <llvm-dev at lists.llvm.org>
> To: Dangeti Tharun kumar <cs15mtech11002 at iith.ac.in>
> Cc: llvm-dev at lists.llvm.org, Santanu Das <cs15mtech11018 at
iith.ac.in>
> Subject: Re: [llvm-dev] Questions on LLVM vectorization diagnostics
> Message-ID: <B6F42D93-F676-4CB1-8413-A37A07490A55 at apple.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hi Dangeti,
> 
>> On Jun 23, 2016, at 8:20 AM, Dangeti Tharun kumar via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>> 
>> Dear LLVM Community,
>> 
>> I am D Tharun Kumar, masters student at Indian Institute of Technology
Hyderabad, working in a team to improve current vectorizer in LLVM. As an
initial study, we are studying various benchmarks to analyze and compare
vectorizing capabilities of LLVM, GCC and ICC. We found that vectorization
remarks given by LLVM are vague and brief, comparatively GCC and ICC are giving
detailed diagnostics.
> 
> Yes this is an area that needs further improvement.  We have some immediate
plans to make these more useful.  See the recent llvm-dev threads [1], [2].
>> I am interested to know why the LLVM diagnostics are brief and not
intuitive (making them less helpful)?
> I think it’s just lack of work or weakness in the analyses to provide more
detailed information.  It would be good to file bugs for specific cases where we
fall behind.
>> In our analysis we never seen llvm trying to vectorize outer loops. Is
this well known? Is outer loop vectorization implemented in LLVM as in GCC?
(http://dl.acm.org/citation.cfm?id=1454119
<http://dl.acm.org/citation.cfm?id=1454119>) If not, is someone working on
it?
> 
> I heard various people mention this but I am not sure whether actual work
is already taking place.
> 
>> On the TSVC benchmark suite, out of a total of 151 loops, LLVM, GCC and
ICC vectorized 70, 82 and 112 loops respectively. Is the cause for lag of LLVM
the inability of LLVM’s vectorizer, or are there any (enabling) optimization
passes running before GCC’s vectorizer that are helping GCC perform better?
> 
> I don’t know about the GCC but I’ve seen ICC perform loop transformation
more aggressively that can increase the coverage for loop vectorization.  ICC
performs Loop Distribution/Fusion/Interchange, etc by default at their highest
optimization level.  We have some of these passes (distribution, interchange)
but not on by default yet.
> 
> Arguably, there is also some difference between focus areas for these
compilers.  I think that ICC has a more HPC focus than LLVM or GCC.  We have
Polly which is geared toward more the HPC use cases.
> 
>> Loop peeling to enhance vectorization is present in GCC and ICC, but,
the LLVM remarks don’t say anything about alignment. Does LLVM has this
functionality and the vectorizer doesn’t remark about it, or it doesn’t it have
the functionality at all?
> We don’t have it.
> 
>> Finally, we appreciate suggestions and directions for improving the
vectorization framework of LLVM.
> 
> This is a pretty active area.  Probably reading up on recent llvm-dev
discussion in this area would be helpful to you.
> 
>> I would also like to know if anyone worked or is working on improving
vectorization remarks.
> 
> Yes we are.  If you’re interested working on this area it would be good to
coordinate.
> 
> Adam
> 
>> 
>> Regards,
>> 
>> Dangeti Tharun kumar
>> M.TECH Computer Science
>> IIT Hyderabad
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> 
> [1] http://thread.gmane.org/gmane.comp.compilers.llvm.devel/98334
> [2] http://thread.gmane.org/gmane.comp.compilers.llvm.devel/99126
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev