On Tue, Aug 11, 2015 at 7:25 PM, Kostya Serebryany <kcc at google.com> wrote:> ... >> So if I'm seeing tens of thousands of distinct test files, that >> represents tens of thousands of distinct edges? >> > > In the extreme case -- yes. > However usually a single file covers more than one unique edge. > Also, if you are running the fuzzer in parallel (-jobs=N) some edges can > be discovered many times. > ... >> With -fsanitize-coverage=indirect-calls it will also track indir call > edges (uniq pairs of caller-callee). > >>Ok, I think the parallel jobs and unique caller/callee pairs must be where it got amped up a bit. I'm using "bb,indirect-calls,8bit-counters".> save_minimized_corpus 0 If 1, the minimized corpus is >>> saved into the first input directory >>> ------------- >>> >>>> >> Ohh, ok. I think I misunderstood this to trying to minimize the size of >> the test case while still reproducing a crash. Similar to how afl-tmin >> works, I was thinking. I'll give this a try. >> >> Should I only use this option periodically or can I run it this way all >> the time? Do we end up spending more execution time minimizing the >> corpus? Will it delete redundant test cases, including ones that were >> there before this test run started? >> > > You should only use this option if you want to store the minimized corpus > somewhere, >Ok, so I can do it periodically to prune the test corpus. That's great.> or if the initial stage (between "#0 READ" and "#1331 INITED") > takes too long. > Otherwise you should not bother since libFuzzer minimizes the corpus in > memory on every run. > (minimization is done with a trivial greedy algorithm, not even close to > really minimal solution, but good enough). > The output looks like this: > > #0 READ cov 0 bits 0 units 1331 exec/s 0 > ... > #1024 pulse cov 8043 bits 13474 units 1331 exec/s 256 > #1331 INITED cov 8050 bits 13689 units 594 exec/s 221 > #2048 pulse cov 8050 bits 13689 units 594 exec/s 341 > > This means that the corpus on disk had 1331 units, they were read, > shuffled, executed, and those that added coverage were chosen. > >>Hah! this means I've been misreading this line all along. My eyes zoomed in on "N exec/s" and I assumed that was the throughput (and I just ignored the "suffix" entry). So it's "cov X" / "bits Y" / "units Z" / "exec/s W" ? I guess the PCRE2 use case in the docs explains some of this -- I should've read this more closely. ... Aha, of course.> I run non-async-signal-safe code in the signal handler, bummer. > Let me try to fix this (no promises for a quick fix, I'll be out for a > while). > >>Ok, no problem. If you don't get around to it I can probably come up w/a fix. -- -Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150811/e83738bf/attachment.html>
On Tue, Aug 11, 2015 at 5:51 PM, Brian Cain <brian.cain at gmail.com> wrote:> On Tue, Aug 11, 2015 at 7:25 PM, Kostya Serebryany <kcc at google.com> wrote: > >> ... >>> So if I'm seeing tens of thousands of distinct test files, that >>> represents tens of thousands of distinct edges? >>> >> >> In the extreme case -- yes. >> However usually a single file covers more than one unique edge. >> Also, if you are running the fuzzer in parallel (-jobs=N) some edges can >> be discovered many times. >> ... >> > > >> With -fsanitize-coverage=indirect-calls it will also track indir call >> edges (uniq pairs of caller-callee). >> >>> > Ok, I think the parallel jobs and unique caller/callee pairs must be where > it got amped up a bit. I'm using "bb,indirect-calls,8bit-counters". >With 8bit-counters you may get up to 8 test cases for every edge.> > >> save_minimized_corpus 0 If 1, the minimized corpus is >>>> saved into the first input directory >>>> ------------- >>>> >>>>> >>> Ohh, ok. I think I misunderstood this to trying to minimize the size of >>> the test case while still reproducing a crash. Similar to how afl-tmin >>> works, I was thinking. I'll give this a try. >>> >>> Should I only use this option periodically or can I run it this way all >>> the time? Do we end up spending more execution time minimizing the >>> corpus? Will it delete redundant test cases, including ones that were >>> there before this test run started? >>> >> >> You should only use this option if you want to store the minimized corpus >> somewhere, >> > > Ok, so I can do it periodically to prune the test corpus. That's great. > > >> or if the initial stage (between "#0 READ" and "#1331 INITED") >> takes too long. >> Otherwise you should not bother since libFuzzer minimizes the corpus in >> memory on every run. >> (minimization is done with a trivial greedy algorithm, not even close to >> really minimal solution, but good enough). >> The output looks like this: >> >> #0 READ cov 0 bits 0 units 1331 exec/s 0 >> ... >> #1024 pulse cov 8043 bits 13474 units 1331 exec/s 256 >> #1331 INITED cov 8050 bits 13689 units 594 exec/s 221 >> #2048 pulse cov 8050 bits 13689 units 594 exec/s 341 >> >> This means that the corpus on disk had 1331 units, they were read, >> shuffled, executed, and those that added coverage were chosen. >> >>> > > Hah! this means I've been misreading this line all along. My eyes zoomed > in on "N exec/s" and I assumed that was the throughput (and I just ignored > the "suffix" entry). So it's "cov X" / "bits Y" / "units Z" / "exec/s W" > ? I guess the PCRE2 use case in the docs explains some of this -- I > should've read this more closely. >Yep. Hm... Maybe I should put ":" there, like #2048 pulse cov: 8050 bits: 13689 units: 594 exec/s: 341 ??> > ... > Aha, of course. > >> I run non-async-signal-safe code in the signal handler, bummer. >> Let me try to fix this (no promises for a quick fix, I'll be out for a >> while). >> >>> > Ok, no problem. If you don't get around to it I can probably come up w/a > fix. > > > -- > -Brian >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150811/56c44ba1/attachment.html>
Kostya Serebryany via llvm-dev <llvm-dev at lists.llvm.org> writes:> On Tue, Aug 11, 2015 at 5:51 PM, Brian Cain <brian.cain at gmail.com> wrote: >> On Tue, Aug 11, 2015 at 7:25 PM, Kostya Serebryany <kcc at google.com> wrote: >>> #0 READ cov 0 bits 0 units 1331 exec/s 0 >>> ... >>> #1024 pulse cov 8043 bits 13474 units 1331 exec/s 256 >>> #1331 INITED cov 8050 bits 13689 units 594 exec/s 221 >>> #2048 pulse cov 8050 bits 13689 units 594 exec/s 341 >>> >>> This means that the corpus on disk had 1331 units, they were read, >>> shuffled, executed, and those that added coverage were chosen. >> >> Hah! this means I've been misreading this line all along. My eyes zoomed >> in on "N exec/s" and I assumed that was the throughput (and I just ignored >> the "suffix" entry). So it's "cov X" / "bits Y" / "units Z" / "exec/s W" >> ? I guess the PCRE2 use case in the docs explains some of this -- I >> should've read this more closely. >> > > Yep. > Hm... Maybe I should put ":" there, like > #2048 pulse cov: 8050 bits: 13689 units: 594 exec/s: 341 > ??Please do. The ":" makes this much clearer, IMHO. It might even be worth it to throw some "," in there like #2048 pulse cov: 8050, bits: 13689, units: 594, exec/s: 341 I find the exec/s in particular looks like a unit, which makes me really want to read "594 exec/s" in the current output. Though, then I can't figure out what the 341 means, which usually clues me in.