Yes running LLVM performance test suite with LTO and Thin LTO enabled. Thanks, Siddharth On Fri, May 11, 2018 at 7:32 PM, Teresa Johnson <tejohnson at google.com> wrote:> I'm not completely sure what you are asking. Are you looking for > performance benchmarks to use for LTO and ThinLTO testing? Or are you > asking how to build/run with LTO and ThinLTO? Are you asking how to run > LLVM's performance test-suite with LTO and ThinLTO? > > Teresa > > On Fri, May 11, 2018 at 5:21 AM Siddharth Shankar Swain < > h2015096 at pilani.bits-pilani.ac.in> wrote: > >> Hi, >> Thanks for the info, If i only want to run performance test on benchmarks >> for LTO and Thin LTO enabled target, Can u suggest ways to do it ? I want >> to do it at my end. >> Thanks, >> Siddharth >> >> On Wed, May 2, 2018 at 7:03 PM, Teresa Johnson <tejohnson at google.com> >> wrote: >> >>> I've only measured performance on x86. There are some old results for >>> SPEC cpu2006 in the blog post here: >>> http://blog.llvm.org/2016/06/thinlto-scalable-and- >>> incremental-lto.html >>> The benefit is benchmark dependent, e.g. small benchmarks often don't >>> improve much as they don't require whole program optimizations. >>> >>> We've improved the ThinLTO optimizations since then, but I don't have >>> any current numbers for SPEC handy. >>> >>> Note that the performance is higher if you combine with PGO as the two >>> techniques are complementary. I would say 5-10% improvement is pretty >>> common (when measured on top of PGO), for codes that can benefit from whole >>> program / cross-module optimizations. >>> >>> Teresa >>> >>> On Tue, May 1, 2018 at 12:12 PM, Siddharth Shankar Swain via llvm-dev < >>> llvm-dev at lists.llvm.org> wrote: >>> >>>> Hi all, >>>> We have enabled Thin LTO and LTO for a specific target architecture. >>>> What can be the possible scopes of improvement depending on the target >>>> after we enable the basic LTO and thin LTO.? >>>> Thanks, >>>> Siddharth >>>> >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>>> >>> >>> >>> -- >>> Teresa Johnson | Software Engineer | tejohnson at google.com | >>> 408-460-2413 >>> >> >> > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com | > 408-460-2413 >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180511/0e5fe75e/attachment.html>
Hopefully someone else on the dev list who has experience with LNT and the LLVM test-suite will chime in. I've never run it myself. Adding Mehdi since he ran it with LTO/ThinLTO. I found some documentation for the test-suite: https://llvm.org/docs/TestingGuide.html#test-suite-overview http://llvm.org/docs/lnt/quickstart.html http://llvm.org/docs/lnt/tests.html#llvm-cmake-test-suite But I can't find any documentation describing how to pass custom options to lnt runtests, so hopefully Mehdi or someone else can help. Teresa On Fri, May 11, 2018 at 7:59 AM Siddharth Shankar Swain < h2015096 at pilani.bits-pilani.ac.in> wrote:> Yes running LLVM performance test suite with LTO and Thin LTO enabled. > Thanks, > Siddharth > > On Fri, May 11, 2018 at 7:32 PM, Teresa Johnson <tejohnson at google.com> > wrote: > >> I'm not completely sure what you are asking. Are you looking for >> performance benchmarks to use for LTO and ThinLTO testing? Or are you >> asking how to build/run with LTO and ThinLTO? Are you asking how to run >> LLVM's performance test-suite with LTO and ThinLTO? >> >> Teresa >> >> On Fri, May 11, 2018 at 5:21 AM Siddharth Shankar Swain < >> h2015096 at pilani.bits-pilani.ac.in> wrote: >> >>> Hi, >>> Thanks for the info, If i only want to run performance test on >>> benchmarks for LTO and Thin LTO enabled target, Can u suggest ways to do it >>> ? I want to do it at my end. >>> Thanks, >>> Siddharth >>> >>> On Wed, May 2, 2018 at 7:03 PM, Teresa Johnson <tejohnson at google.com> >>> wrote: >>> >>>> I've only measured performance on x86. There are some old results for >>>> SPEC cpu2006 in the blog post here: >>>> >>>> http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html >>>> The benefit is benchmark dependent, e.g. small benchmarks often don't >>>> improve much as they don't require whole program optimizations. >>>> >>>> We've improved the ThinLTO optimizations since then, but I don't have >>>> any current numbers for SPEC handy. >>>> >>>> Note that the performance is higher if you combine with PGO as the two >>>> techniques are complementary. I would say 5-10% improvement is pretty >>>> common (when measured on top of PGO), for codes that can benefit from whole >>>> program / cross-module optimizations. >>>> >>>> Teresa >>>> >>>> On Tue, May 1, 2018 at 12:12 PM, Siddharth Shankar Swain via llvm-dev < >>>> llvm-dev at lists.llvm.org> wrote: >>>> >>>>> Hi all, >>>>> We have enabled Thin LTO and LTO for a specific target architecture. >>>>> What can be the possible scopes of improvement depending on the target >>>>> after we enable the basic LTO and thin LTO.? >>>>> Thanks, >>>>> Siddharth >>>>> >>>>> >>>>> _______________________________________________ >>>>> LLVM Developers mailing list >>>>> llvm-dev at lists.llvm.org >>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>> >>>>> >>>> >>>> >>>> -- >>>> Teresa Johnson | Software Engineer | tejohnson at google.com | >>>> 408-460-2413 >>>> >>> >>> >> >> -- >> Teresa Johnson | Software Engineer | tejohnson at google.com | >> 408-460-2413 >> > >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180511/c3af62ee/attachment.html>
Hi Siddharth The easiest way to do that is to run from LNT. You can refer to the quick start guide Teresa linked in her previous reply. You just need to pass in the proper --cflags to build test suite with the configuration you want. There are more advanced options like using a cmake cache file, which works as well. You can also search for LNT jobs on the bots to see how they are configured to run, for example: http://lab.llvm.org:8011/builders/clang-cmake-armv7-lnt <http://lab.llvm.org:8011/builders/clang-cmake-armv7-lnt> http://green.lab.llvm.org/green/job/lnt-test-suite-x86_64-O3-flto/2718/consoleFull <http://green.lab.llvm.org/green/job/lnt-test-suite-x86_64-O3-flto/2718/consoleFull> Steven> On May 11, 2018, at 8:39 AM, Teresa Johnson via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Hopefully someone else on the dev list who has experience with LNT and the LLVM test-suite will chime in. I've never run it myself. Adding Mehdi since he ran it with LTO/ThinLTO. > > I found some documentation for the test-suite: > https://llvm.org/docs/TestingGuide.html#test-suite-overview <https://llvm.org/docs/TestingGuide.html#test-suite-overview> > http://llvm.org/docs/lnt/quickstart.html <http://llvm.org/docs/lnt/quickstart.html> > http://llvm.org/docs/lnt/tests.html#llvm-cmake-test-suite <http://llvm.org/docs/lnt/tests.html#llvm-cmake-test-suite> > > But I can't find any documentation describing how to pass custom options to lnt runtests, so hopefully Mehdi or someone else can help. > > Teresa > > On Fri, May 11, 2018 at 7:59 AM Siddharth Shankar Swain <h2015096 at pilani.bits-pilani.ac.in <mailto:h2015096 at pilani.bits-pilani.ac.in>> wrote: > Yes running LLVM performance test suite with LTO and Thin LTO enabled. > Thanks, > Siddharth > > On Fri, May 11, 2018 at 7:32 PM, Teresa Johnson <tejohnson at google.com <mailto:tejohnson at google.com>> wrote: > I'm not completely sure what you are asking. Are you looking for performance benchmarks to use for LTO and ThinLTO testing? Or are you asking how to build/run with LTO and ThinLTO? Are you asking how to run LLVM's performance test-suite with LTO and ThinLTO? > > Teresa > > On Fri, May 11, 2018 at 5:21 AM Siddharth Shankar Swain <h2015096 at pilani.bits-pilani.ac.in <mailto:h2015096 at pilani.bits-pilani.ac.in>> wrote: > Hi, > Thanks for the info, If i only want to run performance test on benchmarks for LTO and Thin LTO enabled target, Can u suggest ways to do it ? I want to do it at my end. > Thanks, > Siddharth > > On Wed, May 2, 2018 at 7:03 PM, Teresa Johnson <tejohnson at google.com <mailto:tejohnson at google.com>> wrote: > I've only measured performance on x86. There are some old results for SPEC cpu2006 in the blog post here: > http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html <http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html> > The benefit is benchmark dependent, e.g. small benchmarks often don't improve much as they don't require whole program optimizations. > > We've improved the ThinLTO optimizations since then, but I don't have any current numbers for SPEC handy. > > Note that the performance is higher if you combine with PGO as the two techniques are complementary. I would say 5-10% improvement is pretty common (when measured on top of PGO), for codes that can benefit from whole program / cross-module optimizations. > > Teresa > > On Tue, May 1, 2018 at 12:12 PM, Siddharth Shankar Swain via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > Hi all, > We have enabled Thin LTO and LTO for a specific target architecture. What can be the possible scopes of improvement depending on the target after we enable the basic LTO and thin LTO.? > Thanks, > Siddharth > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > > > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com <mailto:tejohnson at google.com> | 408-460-2413 > > > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com <mailto:tejohnson at google.com> | 408-460-2413 > > > > -- > Teresa Johnson | Software Engineer | tejohnson at google.com <mailto:tejohnson at google.com> | 408-460-2413 <tel:408-460-2413>_______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180511/aacad9e4/attachment.html>