Craig Topper via llvm-dev
2018-Jan-06 03:00 UTC
[llvm-dev] Relationship between clang, opt and llc
I don't think "clang -help" prints options about optimizations. Clang itself doesn't have direct support for fine grained optimization control. Just the flag for levels -O0/-O1/-O2/-O3. This is intended to be simple and sufficient interface for most users who just want to compile their code. So I don't think there's a way to pass just -dse to clang. opt on the other hand is more of a utility for developers of llvm that provides fine grained control of optimizations for testing purposes. ~Craig On Fri, Jan 5, 2018 at 5:41 PM, toddy wang <wenwangtoddy at gmail.com> wrote:> Craig, thanks a lot! > > I'm actually confused by clang optimization flags. > > If I run clang -help, it will show many optimizations (denoted as set A) > and non-optimization options (denoted as set B). > If I run llvm-as < /dev/null | opt -O0/1/2/3 -disable-output > -debug-pass=Arguments, it also shows many optimization flags (denote as set > C). > > There are many options in set C while not in set A, and also options in > set A but not in set C. > > The general question is: what is the relationship between set A and set > C, at the same optimization level O0/O1/O2/O3? > Another question is: how to specify an option in set C as a clang command > line option, if it is not in A? > > For example, -dse is in set C but not in set A, how can I specify it as a > clang option? Or simply I cannot do that. > > > > > > > > On Fri, Jan 5, 2018 at 7:55 PM, Craig Topper <craig.topper at gmail.com> > wrote: > >> O0 didn't start applying optnone until r304127 in May 2017 which is after >> the 4.0 family was branched. So only 5.0, 6.0, and trunk have that >> behavior. Commit message copied below >> >> Author: Mehdi Amini <joker.eph at gmail.com> >> >> Date: Mon May 29 05:38:20 2017 +0000 >> >> >> IRGen: Add optnone attribute on function during O0 >> >> >> >> Amongst other, this will help LTO to correctly handle/honor files >> >> compiled with O0, helping debugging failures. >> >> It also seems in line with how we handle other options, like how >> >> -fnoinline adds the appropriate attribute as well. >> >> >> >> Differential Revision: https://reviews.llvm.org/D28404 >> >> >> >> ~Craig >> >> On Fri, Jan 5, 2018 at 4:49 PM, toddy wang <wenwangtoddy at gmail.com> >> wrote: >> >>> @Zhaopei, thanks for the clarification. >>> >>> @Craig and @Michael, for clang 4.0.1, -Xclang -disable-O0-optnone gives >>> the following error message. From which version -disable-O0-optnone gets >>> supported? >>> >>> [twang15 at c89 temp]$ clang++ -O0 -Xclang -disable-O0-optnone -Xclang >>> -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc >>> error: unknown argument: '-disable-O0-optnone' >>> >>> [twang15 at c89 temp]$ clang++ --version >>> clang version 4.0.1 (tags/RELEASE_401/final) >>> Target: x86_64-unknown-linux-gnu >>> >>> On Fri, Jan 5, 2018 at 4:45 PM, Craig Topper <craig.topper at gmail.com> >>> wrote: >>> >>>> If you pass -O0 to clang, most functions will be tagged with an optnone >>>> function attribute that will prevent opt and llc even if you pass -O3 to >>>> opt and llc. This is the mostly likely cause for the slow down in 2. >>>> >>>> You can disable the optnone function attribute behavior by passing >>>> "-Xclang -disable-O0-optnone" to clang >>>> >>>> ~Craig >>>> >>>> On Fri, Jan 5, 2018 at 1:19 PM, toddy wang via llvm-dev < >>>> llvm-dev at lists.llvm.org> wrote: >>>> >>>>> I tried the following on LULESH1.0 serial version ( >>>>> https://codesign.llnl.gov/lulesh/LULESH.cc) >>>>> >>>>> 1. clang++ -O3 LULESH.cc; ./a.out 20 >>>>> Runtime: 9.487353 second >>>>> >>>>> 2. clang++ -O0 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>> clang++ b.o -o b.out; ./b.out 20 >>>>> Runtime: 24.15 seconds >>>>> >>>>> 3. clang++ -O3 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>> clang++ b.o -o b.out; ./b.out 20 >>>>> Runtime: 9.53 seconds >>>>> >>>>> 1 and 3 have almost the same performance, while 2 is significantly >>>>> worse, while I expect 1, 2 ,3 should have trivial difference. >>>>> >>>>> Is this a wrong expectation? >>>>> >>>>> @Peizhao, what did you try in your last post? >>>>> >>>>> On Tue, Apr 11, 2017 at 12:15 PM, Peizhao Ou via llvm-dev < >>>>> llvm-dev at lists.llvm.org> wrote: >>>>> >>>>>> It's really nice of you pointing out the -Xclang option, it makes >>>>>> things much easier. I really appreciate your help! >>>>>> >>>>>> Best, >>>>>> Peizhao >>>>>> >>>>>> On Mon, Apr 10, 2017 at 10:12 PM, Mehdi Amini <mehdi.amini at apple.com> >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> On Apr 10, 2017, at 5:21 PM, Craig Topper via llvm-dev < >>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>> >>>>>>> clang -O0 does not disable all optimization passes modify the IR.; >>>>>>> In fact it causes most functions to get tagged with noinline to prevent >>>>>>> inlinining >>>>>>> >>>>>>> >>>>>>> It also disable lifetime instrinsics emission and TBAA, etc. >>>>>>> >>>>>>> >>>>>>> >>>>>>> What you really need to do is >>>>>>> >>>>>>> clang -O3 -c emit-llvm -o source.bc -v >>>>>>> >>>>>>> Find the -cc1 command line from that output. Execute that command >>>>>>> with --disable-llvm-passes. leave the -O3 and everything else. >>>>>>> >>>>>>> >>>>>>> That’s a bit complicated: CC1 options can be passed through with >>>>>>> -Xclang, for example here just adding to the regular clang invocation ` >>>>>>> -Xclang -disable-llvm-passes` >>>>>>> >>>>>>> Best, >>>>>>> >>>>>>> — >>>>>>> Mehdi >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> You should be able to feed the output from that command to opt/llc >>>>>>> and get consistent results. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> ~Craig >>>>>>> >>>>>>> On Mon, Apr 10, 2017 at 4:57 PM, Peizhao Ou via llvm-dev < >>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>> >>>>>>>> Hi folks, >>>>>>>> >>>>>>>> I am wondering about the relationship clang, opt and llc. I >>>>>>>> understand that this has been asked, e.g., http://stackoverflow.com >>>>>>>> /questions/40350990/relationship-between-clang-opt-llc-and-l >>>>>>>> lvm-linker. Sorry for posting a similar question again, but I >>>>>>>> still have something that hasn't been resolved yet. >>>>>>>> >>>>>>>> More specifically I am wondering about the following two approaches >>>>>>>> compiling optimized executable: >>>>>>>> >>>>>>>> 1. clang -O3 -c source.c -o source.o >>>>>>>> ... >>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>> >>>>>>>> 2. clang -O0 -c -emit-llvm -o source.bc >>>>>>>> opt -O3 source.bc -o source.bc >>>>>>>> llc -O3 -filetype=obj source.bc -o source.o >>>>>>>> ... >>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>> >>>>>>>> I took a look at the source code of the clang tool and the opt >>>>>>>> tool, they both seem to use the PassManagerBuilder::populateModulePassManager() >>>>>>>> and PassManagerBuilder::populateFunctionPassManager() functions to >>>>>>>> add passes to their optimization pipeline; and for the backend, the clang >>>>>>>> and llc both use the addPassesToEmitFile() function to generate object code. >>>>>>>> >>>>>>>> So presumably the above two approaches to generating optimized >>>>>>>> executable file should do the same thing. However, I am seeing that the >>>>>>>> second approach is around 2% slower than the first approach (which is the >>>>>>>> way developers usually use) pretty consistently. >>>>>>>> >>>>>>>> Can anyone point me to the reasons why this happens? Or even >>>>>>>> correct my wrong understanding of the relationship between these two >>>>>>>> approaches? >>>>>>>> >>>>>>>> PS: I used the -debug-pass=Structure option to print out the >>>>>>>> passes, they seem the same except that the first approach has an extra pass >>>>>>>> called "-add-discriminator", but I don't think that's the reason. >>>>>>>> >>>>>>>> Peizhao >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> llvm-dev at lists.llvm.org >>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> LLVM Developers mailing list >>>>>>> llvm-dev at lists.llvm.org >>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> LLVM Developers mailing list >>>>>> llvm-dev at lists.llvm.org >>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> LLVM Developers mailing list >>>>> llvm-dev at lists.llvm.org >>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>> >>>>> >>>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180105/b9f6e317/attachment-0001.html>
toddy wang via llvm-dev
2018-Jan-06 07:30 UTC
[llvm-dev] Relationship between clang, opt and llc
What I am trying is to compile a program with different sets of optimization flags. If there is no fine-grained control over clang optimization flags, it would be impossible to achieve what I intend. Although there is fine-grained control via opt, for a large-scale projects, clang-opt-llc pipeline may not be a drop-in solution. On Fri, Jan 5, 2018 at 10:00 PM, Craig Topper <craig.topper at gmail.com> wrote:> I don't think "clang -help" prints options about optimizations. Clang > itself doesn't have direct support for fine grained optimization control. > Just the flag for levels -O0/-O1/-O2/-O3. This is intended to be simple and > sufficient interface for most users who just want to compile their code. So > I don't think there's a way to pass just -dse to clang. > > opt on the other hand is more of a utility for developers of llvm that > provides fine grained control of optimizations for testing purposes. > > > > ~Craig > > On Fri, Jan 5, 2018 at 5:41 PM, toddy wang <wenwangtoddy at gmail.com> wrote: > >> Craig, thanks a lot! >> >> I'm actually confused by clang optimization flags. >> >> If I run clang -help, it will show many optimizations (denoted as set A) >> and non-optimization options (denoted as set B). >> If I run llvm-as < /dev/null | opt -O0/1/2/3 -disable-output >> -debug-pass=Arguments, it also shows many optimization flags (denote as set >> C). >> >> There are many options in set C while not in set A, and also options in >> set A but not in set C. >> >> The general question is: what is the relationship between set A and set >> C, at the same optimization level O0/O1/O2/O3? >> Another question is: how to specify an option in set C as a clang command >> line option, if it is not in A? >> >> For example, -dse is in set C but not in set A, how can I specify it as a >> clang option? Or simply I cannot do that. >> >> >> >> >> >> >> >> On Fri, Jan 5, 2018 at 7:55 PM, Craig Topper <craig.topper at gmail.com> >> wrote: >> >>> O0 didn't start applying optnone until r304127 in May 2017 which is >>> after the 4.0 family was branched. So only 5.0, 6.0, and trunk have that >>> behavior. Commit message copied below >>> >>> Author: Mehdi Amini <joker.eph at gmail.com> >>> >>> Date: Mon May 29 05:38:20 2017 +0000 >>> >>> >>> IRGen: Add optnone attribute on function during O0 >>> >>> >>> >>> Amongst other, this will help LTO to correctly handle/honor files >>> >>> compiled with O0, helping debugging failures. >>> >>> It also seems in line with how we handle other options, like how >>> >>> -fnoinline adds the appropriate attribute as well. >>> >>> >>> >>> Differential Revision: https://reviews.llvm.org/D28404 >>> >>> >>> >>> ~Craig >>> >>> On Fri, Jan 5, 2018 at 4:49 PM, toddy wang <wenwangtoddy at gmail.com> >>> wrote: >>> >>>> @Zhaopei, thanks for the clarification. >>>> >>>> @Craig and @Michael, for clang 4.0.1, -Xclang -disable-O0-optnone >>>> gives the following error message. From which version -disable-O0-optnone >>>> gets supported? >>>> >>>> [twang15 at c89 temp]$ clang++ -O0 -Xclang -disable-O0-optnone -Xclang >>>> -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc >>>> error: unknown argument: '-disable-O0-optnone' >>>> >>>> [twang15 at c89 temp]$ clang++ --version >>>> clang version 4.0.1 (tags/RELEASE_401/final) >>>> Target: x86_64-unknown-linux-gnu >>>> >>>> On Fri, Jan 5, 2018 at 4:45 PM, Craig Topper <craig.topper at gmail.com> >>>> wrote: >>>> >>>>> If you pass -O0 to clang, most functions will be tagged with an >>>>> optnone function attribute that will prevent opt and llc even if you pass >>>>> -O3 to opt and llc. This is the mostly likely cause for the slow down in 2. >>>>> >>>>> You can disable the optnone function attribute behavior by passing >>>>> "-Xclang -disable-O0-optnone" to clang >>>>> >>>>> ~Craig >>>>> >>>>> On Fri, Jan 5, 2018 at 1:19 PM, toddy wang via llvm-dev < >>>>> llvm-dev at lists.llvm.org> wrote: >>>>> >>>>>> I tried the following on LULESH1.0 serial version ( >>>>>> https://codesign.llnl.gov/lulesh/LULESH.cc) >>>>>> >>>>>> 1. clang++ -O3 LULESH.cc; ./a.out 20 >>>>>> Runtime: 9.487353 second >>>>>> >>>>>> 2. clang++ -O0 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>>> clang++ b.o -o b.out; ./b.out 20 >>>>>> Runtime: 24.15 seconds >>>>>> >>>>>> 3. clang++ -O3 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>>> clang++ b.o -o b.out; ./b.out 20 >>>>>> Runtime: 9.53 seconds >>>>>> >>>>>> 1 and 3 have almost the same performance, while 2 is significantly >>>>>> worse, while I expect 1, 2 ,3 should have trivial difference. >>>>>> >>>>>> Is this a wrong expectation? >>>>>> >>>>>> @Peizhao, what did you try in your last post? >>>>>> >>>>>> On Tue, Apr 11, 2017 at 12:15 PM, Peizhao Ou via llvm-dev < >>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>> >>>>>>> It's really nice of you pointing out the -Xclang option, it makes >>>>>>> things much easier. I really appreciate your help! >>>>>>> >>>>>>> Best, >>>>>>> Peizhao >>>>>>> >>>>>>> On Mon, Apr 10, 2017 at 10:12 PM, Mehdi Amini <mehdi.amini at apple.com >>>>>>> > wrote: >>>>>>> >>>>>>>> >>>>>>>> On Apr 10, 2017, at 5:21 PM, Craig Topper via llvm-dev < >>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>> >>>>>>>> clang -O0 does not disable all optimization passes modify the IR.; >>>>>>>> In fact it causes most functions to get tagged with noinline to prevent >>>>>>>> inlinining >>>>>>>> >>>>>>>> >>>>>>>> It also disable lifetime instrinsics emission and TBAA, etc. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> What you really need to do is >>>>>>>> >>>>>>>> clang -O3 -c emit-llvm -o source.bc -v >>>>>>>> >>>>>>>> Find the -cc1 command line from that output. Execute that command >>>>>>>> with --disable-llvm-passes. leave the -O3 and everything else. >>>>>>>> >>>>>>>> >>>>>>>> That’s a bit complicated: CC1 options can be passed through with >>>>>>>> -Xclang, for example here just adding to the regular clang invocation ` >>>>>>>> -Xclang -disable-llvm-passes` >>>>>>>> >>>>>>>> Best, >>>>>>>> >>>>>>>> — >>>>>>>> Mehdi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> You should be able to feed the output from that command to opt/llc >>>>>>>> and get consistent results. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ~Craig >>>>>>>> >>>>>>>> On Mon, Apr 10, 2017 at 4:57 PM, Peizhao Ou via llvm-dev < >>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>> >>>>>>>>> Hi folks, >>>>>>>>> >>>>>>>>> I am wondering about the relationship clang, opt and llc. I >>>>>>>>> understand that this has been asked, e.g., >>>>>>>>> http://stackoverflow.com/questions/40350990/relationsh >>>>>>>>> ip-between-clang-opt-llc-and-llvm-linker. Sorry for posting a >>>>>>>>> similar question again, but I still have something that hasn't been >>>>>>>>> resolved yet. >>>>>>>>> >>>>>>>>> More specifically I am wondering about the following two >>>>>>>>> approaches compiling optimized executable: >>>>>>>>> >>>>>>>>> 1. clang -O3 -c source.c -o source.o >>>>>>>>> ... >>>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>>> >>>>>>>>> 2. clang -O0 -c -emit-llvm -o source.bc >>>>>>>>> opt -O3 source.bc -o source.bc >>>>>>>>> llc -O3 -filetype=obj source.bc -o source.o >>>>>>>>> ... >>>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>>> >>>>>>>>> I took a look at the source code of the clang tool and the opt >>>>>>>>> tool, they both seem to use the PassManagerBuilder::populateModulePassManager() >>>>>>>>> and PassManagerBuilder::populateFunctionPassManager() functions >>>>>>>>> to add passes to their optimization pipeline; and for the backend, the >>>>>>>>> clang and llc both use the addPassesToEmitFile() function to generate >>>>>>>>> object code. >>>>>>>>> >>>>>>>>> So presumably the above two approaches to generating optimized >>>>>>>>> executable file should do the same thing. However, I am seeing that the >>>>>>>>> second approach is around 2% slower than the first approach (which is the >>>>>>>>> way developers usually use) pretty consistently. >>>>>>>>> >>>>>>>>> Can anyone point me to the reasons why this happens? Or even >>>>>>>>> correct my wrong understanding of the relationship between these two >>>>>>>>> approaches? >>>>>>>>> >>>>>>>>> PS: I used the -debug-pass=Structure option to print out the >>>>>>>>> passes, they seem the same except that the first approach has an extra pass >>>>>>>>> called "-add-discriminator", but I don't think that's the reason. >>>>>>>>> >>>>>>>>> Peizhao >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> LLVM Developers mailing list >>>>>>>>> llvm-dev at lists.llvm.org >>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> llvm-dev at lists.llvm.org >>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> LLVM Developers mailing list >>>>>>> llvm-dev at lists.llvm.org >>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> LLVM Developers mailing list >>>>>> llvm-dev at lists.llvm.org >>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>> >>>>>> >>>>> >>>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180106/c5bb18df/attachment-0001.html>
toddy wang via llvm-dev
2018-Jan-06 20:25 UTC
[llvm-dev] Relationship between clang, opt and llc
@Craig and @Michael After installing clang-5.0 (download from http://releases.llvm.org, does not have Flang build's slowdown mention above), 1. clang++ -O0 -Xclang -disable-O0-optnone -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; clang++ b.o -o b.out; ./b.out 20 runtime: 2.354069e+01 2. clang++ -O1 -Xclang -disable-O0-optnone -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; clang++ b.o -o b.out; ./b.out 20 runtime: 9.046271e+00 3. clang++ -O3 LULESH.cc runtime: 9.118835e+00 4. clang++ -O2 -Xclang -disable-O0-optnone -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; clang++ b.o -o b.out; ./b.out 20 runtime: 9.091278e+00 5. clang++ -O3 -Xclang -disable-O0-optnone -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; clang++ b.o -o b.out; ./b.out 20 runtime: 9.096919e+00 Apparently, clang++ -O0 -Xclang -disable-O0-optnone does not work as expected. The conclusion seems to be -Xclang -disable-O0-optnone works when clang optimization level is O1/O2/O3, not O0. Any comments? On Sat, Jan 6, 2018 at 2:30 AM, toddy wang <wenwangtoddy at gmail.com> wrote:> What I am trying is to compile a program with different sets of > optimization flags. > If there is no fine-grained control over clang optimization flags, it > would be impossible to achieve what I intend. > > Although there is fine-grained control via opt, for a large-scale > projects, clang-opt-llc pipeline may not be a drop-in solution. > > On Fri, Jan 5, 2018 at 10:00 PM, Craig Topper <craig.topper at gmail.com> > wrote: > >> I don't think "clang -help" prints options about optimizations. Clang >> itself doesn't have direct support for fine grained optimization control. >> Just the flag for levels -O0/-O1/-O2/-O3. This is intended to be simple and >> sufficient interface for most users who just want to compile their code. So >> I don't think there's a way to pass just -dse to clang. >> >> opt on the other hand is more of a utility for developers of llvm that >> provides fine grained control of optimizations for testing purposes. >> >> >> >> ~Craig >> >> On Fri, Jan 5, 2018 at 5:41 PM, toddy wang <wenwangtoddy at gmail.com> >> wrote: >> >>> Craig, thanks a lot! >>> >>> I'm actually confused by clang optimization flags. >>> >>> If I run clang -help, it will show many optimizations (denoted as set A) >>> and non-optimization options (denoted as set B). >>> If I run llvm-as < /dev/null | opt -O0/1/2/3 -disable-output >>> -debug-pass=Arguments, it also shows many optimization flags (denote as set >>> C). >>> >>> There are many options in set C while not in set A, and also options in >>> set A but not in set C. >>> >>> The general question is: what is the relationship between set A and set >>> C, at the same optimization level O0/O1/O2/O3? >>> Another question is: how to specify an option in set C as a clang >>> command line option, if it is not in A? >>> >>> For example, -dse is in set C but not in set A, how can I specify it as >>> a clang option? Or simply I cannot do that. >>> >>> >>> >>> >>> >>> >>> >>> On Fri, Jan 5, 2018 at 7:55 PM, Craig Topper <craig.topper at gmail.com> >>> wrote: >>> >>>> O0 didn't start applying optnone until r304127 in May 2017 which is >>>> after the 4.0 family was branched. So only 5.0, 6.0, and trunk have that >>>> behavior. Commit message copied below >>>> >>>> Author: Mehdi Amini <joker.eph at gmail.com> >>>> >>>> Date: Mon May 29 05:38:20 2017 +0000 >>>> >>>> >>>> IRGen: Add optnone attribute on function during O0 >>>> >>>> >>>> >>>> Amongst other, this will help LTO to correctly handle/honor files >>>> >>>> compiled with O0, helping debugging failures. >>>> >>>> It also seems in line with how we handle other options, like how >>>> >>>> -fnoinline adds the appropriate attribute as well. >>>> >>>> >>>> >>>> Differential Revision: https://reviews.llvm.org/D28404 >>>> >>>> >>>> >>>> ~Craig >>>> >>>> On Fri, Jan 5, 2018 at 4:49 PM, toddy wang <wenwangtoddy at gmail.com> >>>> wrote: >>>> >>>>> @Zhaopei, thanks for the clarification. >>>>> >>>>> @Craig and @Michael, for clang 4.0.1, -Xclang -disable-O0-optnone >>>>> gives the following error message. From which version -disable-O0-optnone >>>>> gets supported? >>>>> >>>>> [twang15 at c89 temp]$ clang++ -O0 -Xclang -disable-O0-optnone -Xclang >>>>> -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc >>>>> error: unknown argument: '-disable-O0-optnone' >>>>> >>>>> [twang15 at c89 temp]$ clang++ --version >>>>> clang version 4.0.1 (tags/RELEASE_401/final) >>>>> Target: x86_64-unknown-linux-gnu >>>>> >>>>> On Fri, Jan 5, 2018 at 4:45 PM, Craig Topper <craig.topper at gmail.com> >>>>> wrote: >>>>> >>>>>> If you pass -O0 to clang, most functions will be tagged with an >>>>>> optnone function attribute that will prevent opt and llc even if you pass >>>>>> -O3 to opt and llc. This is the mostly likely cause for the slow down in 2. >>>>>> >>>>>> You can disable the optnone function attribute behavior by passing >>>>>> "-Xclang -disable-O0-optnone" to clang >>>>>> >>>>>> ~Craig >>>>>> >>>>>> On Fri, Jan 5, 2018 at 1:19 PM, toddy wang via llvm-dev < >>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>> >>>>>>> I tried the following on LULESH1.0 serial version ( >>>>>>> https://codesign.llnl.gov/lulesh/LULESH.cc) >>>>>>> >>>>>>> 1. clang++ -O3 LULESH.cc; ./a.out 20 >>>>>>> Runtime: 9.487353 second >>>>>>> >>>>>>> 2. clang++ -O0 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>>>> clang++ b.o -o b.out; ./b.out 20 >>>>>>> Runtime: 24.15 seconds >>>>>>> >>>>>>> 3. clang++ -O3 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>>>> clang++ b.o -o b.out; ./b.out 20 >>>>>>> Runtime: 9.53 seconds >>>>>>> >>>>>>> 1 and 3 have almost the same performance, while 2 is significantly >>>>>>> worse, while I expect 1, 2 ,3 should have trivial difference. >>>>>>> >>>>>>> Is this a wrong expectation? >>>>>>> >>>>>>> @Peizhao, what did you try in your last post? >>>>>>> >>>>>>> On Tue, Apr 11, 2017 at 12:15 PM, Peizhao Ou via llvm-dev < >>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>> >>>>>>>> It's really nice of you pointing out the -Xclang option, it makes >>>>>>>> things much easier. I really appreciate your help! >>>>>>>> >>>>>>>> Best, >>>>>>>> Peizhao >>>>>>>> >>>>>>>> On Mon, Apr 10, 2017 at 10:12 PM, Mehdi Amini < >>>>>>>> mehdi.amini at apple.com> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> On Apr 10, 2017, at 5:21 PM, Craig Topper via llvm-dev < >>>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>>> >>>>>>>>> clang -O0 does not disable all optimization passes modify the IR.; >>>>>>>>> In fact it causes most functions to get tagged with noinline to prevent >>>>>>>>> inlinining >>>>>>>>> >>>>>>>>> >>>>>>>>> It also disable lifetime instrinsics emission and TBAA, etc. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> What you really need to do is >>>>>>>>> >>>>>>>>> clang -O3 -c emit-llvm -o source.bc -v >>>>>>>>> >>>>>>>>> Find the -cc1 command line from that output. Execute that command >>>>>>>>> with --disable-llvm-passes. leave the -O3 and everything else. >>>>>>>>> >>>>>>>>> >>>>>>>>> That’s a bit complicated: CC1 options can be passed through with >>>>>>>>> -Xclang, for example here just adding to the regular clang invocation ` >>>>>>>>> -Xclang -disable-llvm-passes` >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> >>>>>>>>> — >>>>>>>>> Mehdi >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> You should be able to feed the output from that command to opt/llc >>>>>>>>> and get consistent results. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> ~Craig >>>>>>>>> >>>>>>>>> On Mon, Apr 10, 2017 at 4:57 PM, Peizhao Ou via llvm-dev < >>>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>>> >>>>>>>>>> Hi folks, >>>>>>>>>> >>>>>>>>>> I am wondering about the relationship clang, opt and llc. I >>>>>>>>>> understand that this has been asked, e.g., >>>>>>>>>> http://stackoverflow.com/questions/40350990/relationsh >>>>>>>>>> ip-between-clang-opt-llc-and-llvm-linker. Sorry for posting a >>>>>>>>>> similar question again, but I still have something that hasn't been >>>>>>>>>> resolved yet. >>>>>>>>>> >>>>>>>>>> More specifically I am wondering about the following two >>>>>>>>>> approaches compiling optimized executable: >>>>>>>>>> >>>>>>>>>> 1. clang -O3 -c source.c -o source.o >>>>>>>>>> ... >>>>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>>>> >>>>>>>>>> 2. clang -O0 -c -emit-llvm -o source.bc >>>>>>>>>> opt -O3 source.bc -o source.bc >>>>>>>>>> llc -O3 -filetype=obj source.bc -o source.o >>>>>>>>>> ... >>>>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>>>> >>>>>>>>>> I took a look at the source code of the clang tool and the opt >>>>>>>>>> tool, they both seem to use the PassManagerBuilder::populateModulePassManager() >>>>>>>>>> and PassManagerBuilder::populateFunctionPassManager() functions >>>>>>>>>> to add passes to their optimization pipeline; and for the backend, the >>>>>>>>>> clang and llc both use the addPassesToEmitFile() function to generate >>>>>>>>>> object code. >>>>>>>>>> >>>>>>>>>> So presumably the above two approaches to generating optimized >>>>>>>>>> executable file should do the same thing. However, I am seeing that the >>>>>>>>>> second approach is around 2% slower than the first approach (which is the >>>>>>>>>> way developers usually use) pretty consistently. >>>>>>>>>> >>>>>>>>>> Can anyone point me to the reasons why this happens? Or even >>>>>>>>>> correct my wrong understanding of the relationship between these two >>>>>>>>>> approaches? >>>>>>>>>> >>>>>>>>>> PS: I used the -debug-pass=Structure option to print out the >>>>>>>>>> passes, they seem the same except that the first approach has an extra pass >>>>>>>>>> called "-add-discriminator", but I don't think that's the reason. >>>>>>>>>> >>>>>>>>>> Peizhao >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> LLVM Developers mailing list >>>>>>>>>> llvm-dev at lists.llvm.org >>>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> LLVM Developers mailing list >>>>>>>>> llvm-dev at lists.llvm.org >>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> llvm-dev at lists.llvm.org >>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> LLVM Developers mailing list >>>>>>> llvm-dev at lists.llvm.org >>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180106/fe06a957/attachment.html>
Sean Silva via llvm-dev
2018-Jan-07 00:37 UTC
[llvm-dev] Relationship between clang, opt and llc
On Jan 5, 2018 11:30 PM, "toddy wang via llvm-dev" <llvm-dev at lists.llvm.org> wrote: What I am trying is to compile a program with different sets of optimization flags. If there is no fine-grained control over clang optimization flags, it would be impossible to achieve what I intend. LLD has -lto-newpm-passes (and the corresponding -lto-newpm-aa-pipeline) which allows you to pass a custom pass pipeline with full control. At one point I was using a similar modification to clang (see https://reviews.llvm.org/D21954) that never landed. -- Sean Silva Although there is fine-grained control via opt, for a large-scale projects, clang-opt-llc pipeline may not be a drop-in solution. On Fri, Jan 5, 2018 at 10:00 PM, Craig Topper <craig.topper at gmail.com> wrote:> I don't think "clang -help" prints options about optimizations. Clang > itself doesn't have direct support for fine grained optimization control. > Just the flag for levels -O0/-O1/-O2/-O3. This is intended to be simple and > sufficient interface for most users who just want to compile their code. So > I don't think there's a way to pass just -dse to clang. > > opt on the other hand is more of a utility for developers of llvm that > provides fine grained control of optimizations for testing purposes. > > > > ~Craig > > On Fri, Jan 5, 2018 at 5:41 PM, toddy wang <wenwangtoddy at gmail.com> wrote: > >> Craig, thanks a lot! >> >> I'm actually confused by clang optimization flags. >> >> If I run clang -help, it will show many optimizations (denoted as set A) >> and non-optimization options (denoted as set B). >> If I run llvm-as < /dev/null | opt -O0/1/2/3 -disable-output >> -debug-pass=Arguments, it also shows many optimization flags (denote as set >> C). >> >> There are many options in set C while not in set A, and also options in >> set A but not in set C. >> >> The general question is: what is the relationship between set A and set >> C, at the same optimization level O0/O1/O2/O3? >> Another question is: how to specify an option in set C as a clang command >> line option, if it is not in A? >> >> For example, -dse is in set C but not in set A, how can I specify it as a >> clang option? Or simply I cannot do that. >> >> >> >> >> >> >> >> On Fri, Jan 5, 2018 at 7:55 PM, Craig Topper <craig.topper at gmail.com> >> wrote: >> >>> O0 didn't start applying optnone until r304127 in May 2017 which is >>> after the 4.0 family was branched. So only 5.0, 6.0, and trunk have that >>> behavior. Commit message copied below >>> >>> Author: Mehdi Amini <joker.eph at gmail.com> >>> >>> Date: Mon May 29 05:38:20 2017 +0000 >>> >>> >>> IRGen: Add optnone attribute on function during O0 >>> >>> >>> >>> Amongst other, this will help LTO to correctly handle/honor files >>> >>> compiled with O0, helping debugging failures. >>> >>> It also seems in line with how we handle other options, like how >>> >>> -fnoinline adds the appropriate attribute as well. >>> >>> >>> >>> Differential Revision: https://reviews.llvm.org/D28404 >>> >>> >>> >>> ~Craig >>> >>> On Fri, Jan 5, 2018 at 4:49 PM, toddy wang <wenwangtoddy at gmail.com> >>> wrote: >>> >>>> @Zhaopei, thanks for the clarification. >>>> >>>> @Craig and @Michael, for clang 4.0.1, -Xclang -disable-O0-optnone >>>> gives the following error message. From which version -disable-O0-optnone >>>> gets supported? >>>> >>>> [twang15 at c89 temp]$ clang++ -O0 -Xclang -disable-O0-optnone -Xclang >>>> -disable-llvm-passes -c -emit-llvm -o a.bc LULESH.cc >>>> error: unknown argument: '-disable-O0-optnone' >>>> >>>> [twang15 at c89 temp]$ clang++ --version >>>> clang version 4.0.1 (tags/RELEASE_401/final) >>>> Target: x86_64-unknown-linux-gnu >>>> >>>> On Fri, Jan 5, 2018 at 4:45 PM, Craig Topper <craig.topper at gmail.com> >>>> wrote: >>>> >>>>> If you pass -O0 to clang, most functions will be tagged with an >>>>> optnone function attribute that will prevent opt and llc even if you pass >>>>> -O3 to opt and llc. This is the mostly likely cause for the slow down in 2. >>>>> >>>>> You can disable the optnone function attribute behavior by passing >>>>> "-Xclang -disable-O0-optnone" to clang >>>>> >>>>> ~Craig >>>>> >>>>> On Fri, Jan 5, 2018 at 1:19 PM, toddy wang via llvm-dev < >>>>> llvm-dev at lists.llvm.org> wrote: >>>>> >>>>>> I tried the following on LULESH1.0 serial version ( >>>>>> https://codesign.llnl.gov/lulesh/LULESH.cc) >>>>>> >>>>>> 1. clang++ -O3 LULESH.cc; ./a.out 20 >>>>>> Runtime: 9.487353 second >>>>>> >>>>>> 2. clang++ -O0 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>>> clang++ b.o -o b.out; ./b.out 20 >>>>>> Runtime: 24.15 seconds >>>>>> >>>>>> 3. clang++ -O3 -Xclang -disable-llvm-passes -c -emit-llvm -o a.bc >>>>>> LULESH.cc; opt -O3 a.bc -o b.bc; llc -O3 -filetype=obj b.bc -o b.o ; >>>>>> clang++ b.o -o b.out; ./b.out 20 >>>>>> Runtime: 9.53 seconds >>>>>> >>>>>> 1 and 3 have almost the same performance, while 2 is significantly >>>>>> worse, while I expect 1, 2 ,3 should have trivial difference. >>>>>> >>>>>> Is this a wrong expectation? >>>>>> >>>>>> @Peizhao, what did you try in your last post? >>>>>> >>>>>> On Tue, Apr 11, 2017 at 12:15 PM, Peizhao Ou via llvm-dev < >>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>> >>>>>>> It's really nice of you pointing out the -Xclang option, it makes >>>>>>> things much easier. I really appreciate your help! >>>>>>> >>>>>>> Best, >>>>>>> Peizhao >>>>>>> >>>>>>> On Mon, Apr 10, 2017 at 10:12 PM, Mehdi Amini <mehdi.amini at apple.com >>>>>>> > wrote: >>>>>>> >>>>>>>> >>>>>>>> On Apr 10, 2017, at 5:21 PM, Craig Topper via llvm-dev < >>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>> >>>>>>>> clang -O0 does not disable all optimization passes modify the IR.; >>>>>>>> In fact it causes most functions to get tagged with noinline to prevent >>>>>>>> inlinining >>>>>>>> >>>>>>>> >>>>>>>> It also disable lifetime instrinsics emission and TBAA, etc. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> What you really need to do is >>>>>>>> >>>>>>>> clang -O3 -c emit-llvm -o source.bc -v >>>>>>>> >>>>>>>> Find the -cc1 command line from that output. Execute that command >>>>>>>> with --disable-llvm-passes. leave the -O3 and everything else. >>>>>>>> >>>>>>>> >>>>>>>> That’s a bit complicated: CC1 options can be passed through with >>>>>>>> -Xclang, for example here just adding to the regular clang invocation ` >>>>>>>> -Xclang -disable-llvm-passes` >>>>>>>> >>>>>>>> Best, >>>>>>>> >>>>>>>> — >>>>>>>> Mehdi >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> You should be able to feed the output from that command to opt/llc >>>>>>>> and get consistent results. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ~Craig >>>>>>>> >>>>>>>> On Mon, Apr 10, 2017 at 4:57 PM, Peizhao Ou via llvm-dev < >>>>>>>> llvm-dev at lists.llvm.org> wrote: >>>>>>>> >>>>>>>>> Hi folks, >>>>>>>>> >>>>>>>>> I am wondering about the relationship clang, opt and llc. I >>>>>>>>> understand that this has been asked, e.g., >>>>>>>>> http://stackoverflow.com/questions/40350990/relationsh >>>>>>>>> ip-between-clang-opt-llc-and-llvm-linker. Sorry for posting a >>>>>>>>> similar question again, but I still have something that hasn't been >>>>>>>>> resolved yet. >>>>>>>>> >>>>>>>>> More specifically I am wondering about the following two >>>>>>>>> approaches compiling optimized executable: >>>>>>>>> >>>>>>>>> 1. clang -O3 -c source.c -o source.o >>>>>>>>> ... >>>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>>> >>>>>>>>> 2. clang -O0 -c -emit-llvm -o source.bc >>>>>>>>> opt -O3 source.bc -o source.bc >>>>>>>>> llc -O3 -filetype=obj source.bc -o source.o >>>>>>>>> ... >>>>>>>>> clang a.o b.o c.o ... -o executable >>>>>>>>> >>>>>>>>> I took a look at the source code of the clang tool and the opt >>>>>>>>> tool, they both seem to use the PassManagerBuilder::populateModulePassManager() >>>>>>>>> and PassManagerBuilder::populateFunctionPassManager() functions >>>>>>>>> to add passes to their optimization pipeline; and for the backend, the >>>>>>>>> clang and llc both use the addPassesToEmitFile() function to generate >>>>>>>>> object code. >>>>>>>>> >>>>>>>>> So presumably the above two approaches to generating optimized >>>>>>>>> executable file should do the same thing. However, I am seeing that the >>>>>>>>> second approach is around 2% slower than the first approach (which is the >>>>>>>>> way developers usually use) pretty consistently. >>>>>>>>> >>>>>>>>> Can anyone point me to the reasons why this happens? Or even >>>>>>>>> correct my wrong understanding of the relationship between these two >>>>>>>>> approaches? >>>>>>>>> >>>>>>>>> PS: I used the -debug-pass=Structure option to print out the >>>>>>>>> passes, they seem the same except that the first approach has an extra pass >>>>>>>>> called "-add-discriminator", but I don't think that's the reason. >>>>>>>>> >>>>>>>>> Peizhao >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> LLVM Developers mailing list >>>>>>>>> llvm-dev at lists.llvm.org >>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> llvm-dev at lists.llvm.org >>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> LLVM Developers mailing list >>>>>>> llvm-dev at lists.llvm.org >>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> LLVM Developers mailing list >>>>>> llvm-dev at lists.llvm.org >>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>>> >>>>>> >>>>> >>>> >>> >> >_______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180106/deda8419/attachment.html>