Regarding performance, I've made a simple coverage with counters and compared it with AsanCoverage. AsanCoverage produces code like this: mov 0xe86cce(%rip),%al test %al,%al je 48b4a0 # to call __sanitizer_cov ... callq 4715b0 <__sanitizer_cov> A simple counter-based thing (which just increments counters and does nothing else useful) produces this: incq 0xe719c6(%rip) The performance is more or less the same, although the issue with false sharing still remains (http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-October/066116.html) Do you have any more details about the planned clang coverage? Thanks, --kcc On Tue, Feb 18, 2014 at 1:00 PM, Kostya Serebryany <kcc at google.com> wrote:> > > > On Tue, Feb 18, 2014 at 12:20 AM, Bob Wilson <bob.wilson at apple.com> wrote: > >> >> On Feb 17, 2014, at 5:13 AM, Kostya Serebryany <kcc at google.com> wrote: >> > Then my question: will there be any objection if I disentangle >> AsanCoverage from ASan and make it a separate LLVM phase with the proper >> clang driver support? >> > Or it will be an unwelcome competition with the planned clang coverage? >> >> I don’t view it as a competition, but assuming that we both succeed in >> our plans, LLVM would then end up with two very similar solutions for code >> coverage. Does it really make sense to have two? > > > It depends. If the two will indeed have the same functionality -- then no. > My understanding about your plans is that the upcoming coverage will > provide "counters" (== how many times a bb/edge was executed). > AsanCoverage produces booleans (== 1, iff a function/bb was executed), > which is less information, but faster. > How much faster -- I can't tell w/o your performance numbers. > For our early users the performance is critical and booleans are > sufficient. > > If we end up needing both variants, we may want to keep them similar from > user perspective, e.g. have flag combinations like these: > -coverage=per-edge,counter > -coverage=per-function,counter > -coverage=per-block,counter > -coverage=per-function,boolean > -coverage=per-block,boolean > > --kcc >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140218/1d36dc48/attachment.html>
Our instrumentation code is basically done now, so you can try it out and compare the performance. Just build with -finstr-profile-generate. You may want to hack the compiler-rt code in lib/profile/PGOProfiling.c to disable writing the output, since the current implementation is pretty naive. If it turns out as you suggest, and the different kinds of instrumentation serve different purposes, then I think it would make sense to do both. On Feb 18, 2014, at 3:13 AM, Kostya Serebryany <kcc at google.com> wrote:> Regarding performance, I've made a simple coverage with counters and compared it with AsanCoverage. > > AsanCoverage produces code like this: > mov 0xe86cce(%rip),%al > test %al,%al > je 48b4a0 # to call __sanitizer_cov > ... > callq 4715b0 <__sanitizer_cov> > > A simple counter-based thing (which just increments counters and does nothing else useful) produces this: > incq 0xe719c6(%rip) > > The performance is more or less the same, although the issue with false sharing still remains > (http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-October/066116.html) > > Do you have any more details about the planned clang coverage? > > Thanks, > > --kcc > > > On Tue, Feb 18, 2014 at 1:00 PM, Kostya Serebryany <kcc at google.com> wrote: > > > > On Tue, Feb 18, 2014 at 12:20 AM, Bob Wilson <bob.wilson at apple.com> wrote: > > On Feb 17, 2014, at 5:13 AM, Kostya Serebryany <kcc at google.com> wrote: > > Then my question: will there be any objection if I disentangle AsanCoverage from ASan and make it a separate LLVM phase with the proper clang driver support? > > Or it will be an unwelcome competition with the planned clang coverage? > > I don’t view it as a competition, but assuming that we both succeed in our plans, LLVM would then end up with two very similar solutions for code coverage. Does it really make sense to have two? > > It depends. If the two will indeed have the same functionality -- then no. > My understanding about your plans is that the upcoming coverage will provide "counters" (== how many times a bb/edge was executed). > AsanCoverage produces booleans (== 1, iff a function/bb was executed), which is less information, but faster. > How much faster -- I can't tell w/o your performance numbers. > For our early users the performance is critical and booleans are sufficient. > > If we end up needing both variants, we may want to keep them similar from user perspective, e.g. have flag combinations like these: > -coverage=per-edge,counter > -coverage=per-function,counter > -coverage=per-block,counter > -coverage=per-function,boolean > -coverage=per-block,boolean > > --kcc >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140218/11aae340/attachment.html>
On Tue, Feb 18, 2014 at 10:15 PM, Bob Wilson <bob.wilson at apple.com> wrote:> Our instrumentation code is basically done now, so you can try it out and > compare the performance. Just build with -finstr-profile-generate. >Is this in trunk? clang-3.5: error: unknown argument: '-finstr-profile-generate' --kcc> You may want to hack the compiler-rt code in lib/profile/PGOProfiling.c to > disable writing the output, since the current implementation is pretty > naive. > > If it turns out as you suggest, and the different kinds of instrumentation > serve different purposes, then I think it would make sense to do both. > > On Feb 18, 2014, at 3:13 AM, Kostya Serebryany <kcc at google.com> wrote: > > Regarding performance, I've made a simple coverage with counters and > compared it with AsanCoverage. > > AsanCoverage produces code like this: > mov 0xe86cce(%rip),%al > test %al,%al > je 48b4a0 # to call __sanitizer_cov > ... > callq 4715b0 <__sanitizer_cov> > > A simple counter-based thing (which just increments counters and does > nothing else useful) produces this: > incq 0xe719c6(%rip) > > The performance is more or less the same, although the issue with false > sharing still remains > (http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-October/066116.html) > > Do you have any more details about the planned clang coverage? > > Thanks, > > --kcc > > > On Tue, Feb 18, 2014 at 1:00 PM, Kostya Serebryany <kcc at google.com> wrote: > >> >> >> >> On Tue, Feb 18, 2014 at 12:20 AM, Bob Wilson <bob.wilson at apple.com>wrote: >> >>> >>> On Feb 17, 2014, at 5:13 AM, Kostya Serebryany <kcc at google.com> wrote: >>> > Then my question: will there be any objection if I disentangle >>> AsanCoverage from ASan and make it a separate LLVM phase with the proper >>> clang driver support? >>> > Or it will be an unwelcome competition with the planned clang coverage? >>> >>> I don’t view it as a competition, but assuming that we both succeed in >>> our plans, LLVM would then end up with two very similar solutions for code >>> coverage. Does it really make sense to have two? >> >> >> It depends. If the two will indeed have the same functionality -- then no. >> My understanding about your plans is that the upcoming coverage will >> provide "counters" (== how many times a bb/edge was executed). >> AsanCoverage produces booleans (== 1, iff a function/bb was executed), >> which is less information, but faster. >> How much faster -- I can't tell w/o your performance numbers. >> For our early users the performance is critical and booleans are >> sufficient. >> >> If we end up needing both variants, we may want to keep them similar from >> user perspective, e.g. have flag combinations like these: >> -coverage=per-edge,counter >> -coverage=per-function,counter >> -coverage=per-block,counter >> -coverage=per-function,boolean >> -coverage=per-block,boolean >> >> --kcc >> > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140219/b9ec7a41/attachment.html>