On Tue, Feb 18, 2014 at 10:15 PM, Bob Wilson <bob.wilson at apple.com> wrote:> Our instrumentation code is basically done now, so you can try it out and > compare the performance. Just build with -finstr-profile-generate. >Is this in trunk? clang-3.5: error: unknown argument: '-finstr-profile-generate' --kcc> You may want to hack the compiler-rt code in lib/profile/PGOProfiling.c to > disable writing the output, since the current implementation is pretty > naive. > > If it turns out as you suggest, and the different kinds of instrumentation > serve different purposes, then I think it would make sense to do both. > > On Feb 18, 2014, at 3:13 AM, Kostya Serebryany <kcc at google.com> wrote: > > Regarding performance, I've made a simple coverage with counters and > compared it with AsanCoverage. > > AsanCoverage produces code like this: > mov 0xe86cce(%rip),%al > test %al,%al > je 48b4a0 # to call __sanitizer_cov > ... > callq 4715b0 <__sanitizer_cov> > > A simple counter-based thing (which just increments counters and does > nothing else useful) produces this: > incq 0xe719c6(%rip) > > The performance is more or less the same, although the issue with false > sharing still remains > (http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-October/066116.html) > > Do you have any more details about the planned clang coverage? > > Thanks, > > --kcc > > > On Tue, Feb 18, 2014 at 1:00 PM, Kostya Serebryany <kcc at google.com> wrote: > >> >> >> >> On Tue, Feb 18, 2014 at 12:20 AM, Bob Wilson <bob.wilson at apple.com>wrote: >> >>> >>> On Feb 17, 2014, at 5:13 AM, Kostya Serebryany <kcc at google.com> wrote: >>> > Then my question: will there be any objection if I disentangle >>> AsanCoverage from ASan and make it a separate LLVM phase with the proper >>> clang driver support? >>> > Or it will be an unwelcome competition with the planned clang coverage? >>> >>> I don’t view it as a competition, but assuming that we both succeed in >>> our plans, LLVM would then end up with two very similar solutions for code >>> coverage. Does it really make sense to have two? >> >> >> It depends. If the two will indeed have the same functionality -- then no. >> My understanding about your plans is that the upcoming coverage will >> provide "counters" (== how many times a bb/edge was executed). >> AsanCoverage produces booleans (== 1, iff a function/bb was executed), >> which is less information, but faster. >> How much faster -- I can't tell w/o your performance numbers. >> For our early users the performance is critical and booleans are >> sufficient. >> >> If we end up needing both variants, we may want to keep them similar from >> user perspective, e.g. have flag combinations like these: >> -coverage=per-edge,counter >> -coverage=per-function,counter >> -coverage=per-block,counter >> -coverage=per-function,boolean >> -coverage=per-block,boolean >> >> --kcc >> > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140219/b9ec7a41/attachment.html>
Ah, that's -fprofile-instr-generate On Wed, Feb 19, 2014 at 2:03 PM, Kostya Serebryany <kcc at google.com> wrote:> > > > On Tue, Feb 18, 2014 at 10:15 PM, Bob Wilson <bob.wilson at apple.com> wrote: > >> Our instrumentation code is basically done now, so you can try it out and >> compare the performance. Just build with -finstr-profile-generate. >> > > Is this in trunk? > clang-3.5: error: unknown argument: '-finstr-profile-generate' > > --kcc > > >> You may want to hack the compiler-rt code in lib/profile/PGOProfiling.c >> to disable writing the output, since the current implementation is pretty >> naive. >> >> If it turns out as you suggest, and the different kinds of >> instrumentation serve different purposes, then I think it would make sense >> to do both. >> >> On Feb 18, 2014, at 3:13 AM, Kostya Serebryany <kcc at google.com> wrote: >> >> Regarding performance, I've made a simple coverage with counters and >> compared it with AsanCoverage. >> >> AsanCoverage produces code like this: >> mov 0xe86cce(%rip),%al >> test %al,%al >> je 48b4a0 # to call __sanitizer_cov >> ... >> callq 4715b0 <__sanitizer_cov> >> >> A simple counter-based thing (which just increments counters and does >> nothing else useful) produces this: >> incq 0xe719c6(%rip) >> >> The performance is more or less the same, although the issue with false >> sharing still remains >> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-October/066116.html) >> >> Do you have any more details about the planned clang coverage? >> >> Thanks, >> >> --kcc >> >> >> On Tue, Feb 18, 2014 at 1:00 PM, Kostya Serebryany <kcc at google.com>wrote: >> >>> >>> >>> >>> On Tue, Feb 18, 2014 at 12:20 AM, Bob Wilson <bob.wilson at apple.com>wrote: >>> >>>> >>>> On Feb 17, 2014, at 5:13 AM, Kostya Serebryany <kcc at google.com> wrote: >>>> > Then my question: will there be any objection if I disentangle >>>> AsanCoverage from ASan and make it a separate LLVM phase with the proper >>>> clang driver support? >>>> > Or it will be an unwelcome competition with the planned clang >>>> coverage? >>>> >>>> I don’t view it as a competition, but assuming that we both succeed in >>>> our plans, LLVM would then end up with two very similar solutions for code >>>> coverage. Does it really make sense to have two? >>> >>> >>> It depends. If the two will indeed have the same functionality -- then >>> no. >>> My understanding about your plans is that the upcoming coverage will >>> provide "counters" (== how many times a bb/edge was executed). >>> AsanCoverage produces booleans (== 1, iff a function/bb was executed), >>> which is less information, but faster. >>> How much faster -- I can't tell w/o your performance numbers. >>> For our early users the performance is critical and booleans are >>> sufficient. >>> >>> If we end up needing both variants, we may want to keep them similar >>> from user perspective, e.g. have flag combinations like these: >>> -coverage=per-edge,counter >>> -coverage=per-function,counter >>> -coverage=per-block,counter >>> -coverage=per-function,boolean >>> -coverage=per-block,boolean >>> >>> --kcc >>> >> >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140219/1a616c76/attachment.html>
I've built chromium with " -fprofile-instr-generate -fsanitize=address" -- the performance looks good! The file format from r198638 is indeed rudimentary. Do you already know how the real output format will look like? Just to summarize what I think is important: - minimal size on disk, minimal amount of files - minimal i/o while writing to disk, no lockf or some such - separate process produces separate file(s) - fast merging of outputs from different runs of the same binary - only the coverage output files and the binary (+DSOs) are required to "symbolize" the coverage data (map the data to file names & line numbers) Ideally, symbolizing the cov data will use the debug info in the binary (i.e. llvm-symbolizer, addr2-line or atos), this is what we've done in AsanCov, but I don't see a clean way to make it work with counters... --kcc On Wed, Feb 19, 2014 at 2:08 PM, Kostya Serebryany <kcc at google.com> wrote:> Ah, that's -fprofile-instr-generate > > > On Wed, Feb 19, 2014 at 2:03 PM, Kostya Serebryany <kcc at google.com> wrote: > >> >> >> >> On Tue, Feb 18, 2014 at 10:15 PM, Bob Wilson <bob.wilson at apple.com>wrote: >> >>> Our instrumentation code is basically done now, so you can try it out >>> and compare the performance. Just build with -finstr-profile-generate. >>> >> >> Is this in trunk? >> clang-3.5: error: unknown argument: '-finstr-profile-generate' >> >> --kcc >> >> >>> You may want to hack the compiler-rt code in lib/profile/PGOProfiling.c >>> to disable writing the output, since the current implementation is pretty >>> naive. >>> >>> If it turns out as you suggest, and the different kinds of >>> instrumentation serve different purposes, then I think it would make sense >>> to do both. >>> >>> On Feb 18, 2014, at 3:13 AM, Kostya Serebryany <kcc at google.com> wrote: >>> >>> Regarding performance, I've made a simple coverage with counters and >>> compared it with AsanCoverage. >>> >>> AsanCoverage produces code like this: >>> mov 0xe86cce(%rip),%al >>> test %al,%al >>> je 48b4a0 # to call __sanitizer_cov >>> ... >>> callq 4715b0 <__sanitizer_cov> >>> >>> A simple counter-based thing (which just increments counters and does >>> nothing else useful) produces this: >>> incq 0xe719c6(%rip) >>> >>> The performance is more or less the same, although the issue with false >>> sharing still remains >>> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-October/066116.html) >>> >>> Do you have any more details about the planned clang coverage? >>> >>> Thanks, >>> >>> --kcc >>> >>> >>> On Tue, Feb 18, 2014 at 1:00 PM, Kostya Serebryany <kcc at google.com>wrote: >>> >>>> >>>> >>>> >>>> On Tue, Feb 18, 2014 at 12:20 AM, Bob Wilson <bob.wilson at apple.com>wrote: >>>> >>>>> >>>>> On Feb 17, 2014, at 5:13 AM, Kostya Serebryany <kcc at google.com> wrote: >>>>> > Then my question: will there be any objection if I disentangle >>>>> AsanCoverage from ASan and make it a separate LLVM phase with the proper >>>>> clang driver support? >>>>> > Or it will be an unwelcome competition with the planned clang >>>>> coverage? >>>>> >>>>> I don’t view it as a competition, but assuming that we both succeed in >>>>> our plans, LLVM would then end up with two very similar solutions for code >>>>> coverage. Does it really make sense to have two? >>>> >>>> >>>> It depends. If the two will indeed have the same functionality -- then >>>> no. >>>> My understanding about your plans is that the upcoming coverage will >>>> provide "counters" (== how many times a bb/edge was executed). >>>> AsanCoverage produces booleans (== 1, iff a function/bb was executed), >>>> which is less information, but faster. >>>> How much faster -- I can't tell w/o your performance numbers. >>>> For our early users the performance is critical and booleans are >>>> sufficient. >>>> >>>> If we end up needing both variants, we may want to keep them similar >>>> from user perspective, e.g. have flag combinations like these: >>>> -coverage=per-edge,counter >>>> -coverage=per-function,counter >>>> -coverage=per-block,counter >>>> -coverage=per-function,boolean >>>> -coverage=per-block,boolean >>>> >>>> --kcc >>>> >>> >>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140219/6d6544d8/attachment.html>
Seemingly Similar Threads
- [LLVMdev] asan coverage
- [LLVMdev] Capabilities of Clang's PGO (e.g. improving code density)
- [LLVMdev] Code Owner for PGO and Branch Weights
- [LLVMdev] Capabilities of Clang's PGO (e.g. improving code density)
- [LLVMdev] Capabilities of Clang's PGO (e.g. improving code density)