On Tue, Apr 19, 2016 at 1:18 PM, Filipe Cabecinhas <filcab at gmail.com> wrote:> Thanks for proposing this. It seems like it might be an interesting > tool for us too. But this proposal seems a bit hand-wavy, and I think > it's missing some crucial info before we start heading this way. >> At least for the tools you are currently starting to implement, it > would be nice to have some estimates and plans on what is going to > happen. >I would actually like to see a small RFC about each tool and what the> plan (overhead/slowdown, pseudo-code for instrumentation, UX, ...) is > before starting to commit. >I don't expect the plan to be very detailed, nor for everything to be> pinned down, of course. This seems to be a bit at a research stage, > and I'm totally ok with it. But would rather it not be as opaque as it > is now. The way the tools will report problems/data/etc, is important, > for example. >The main response here is that these tools are in an exploratory stage and that what we are proposing is not a fixed list of tools but a framework within which we can build prototypes and refine them. We do not know which ideas will work out and which ones will be abandoned. In some cases, answers will not all be known until a prototype is in place to see how it behaves on large applications: we might need to add additional shadow metadata or additional instrumentation to produce the right amount of actionable output. What we want to avoid is having to build prototypes out-of-tree until they are proven and polished and only then trying to commit large patches upstream. Do you have any estimates on memory overhead (both memory usage> (+shadow), and code size) you expect? As well as estimated about the > possible slowdown? >The shadow memory for tools we are considering include 64:1, 4:1, and 1:1. This is all within the ranges of existing sanitizers. Each tool will have inlined instrumentation to update shadow memory on either every memory access or in some cases just focusing on the heap; I do not have a number for code size expansion. The slowdown ranges were in the original email: 2x-5x.> We are proposing the name EfficiencySanitizer, or "esan" for short, to > refer > > to this suite of dynamic instrumentation tools for improving program > > efficiency. As we have a number of different tools that share quite a > bit > > of their implementation we plan to consider them sub-tools under the > > EfficiencySanitizer umbrella, rather than adding a whole bunch of > separate > > instrumentation and runtime library components. > > How much code is expected to be shared? Most? Similar to what the > sanitizers already share? >More than what the existing sanitizers share.> Do we expect the shadow memory mapping to be (mostly) the same among > all esan's tools? >Most will use a single mapping and single algorithm, but the scaledown will vary as mentioned.> Do you already have an idea of a few tools + the type of code that > would be shared (for example: "read/write instrumentation is mostly > the same among these tools", or "generating reports for these tools is > mostly the same", or something similar)? >Instrumentation will be very similar (minor differences in shadow scaling and shadow metadata format), identical intrinsic handling, identical slowpath entries, identical libc interception, identical shadow management. Reports are not fully explored.> While these tools are not addressing correctness issues like other > > sanitizers, they will be sharing a lot of the existing sanitizer runtime > > library support code. Furthermore, users are already familiar with the > > sanitizer brand, and it seems better to extend that concept rather than > add > > some new term. > > Not the email to bike-shed the name, but I don't like "Efficiency" > that much here :-) >There were name discussions prior to the RFC and the conclusion was that there is just not going to be a single first-place choice for everyone. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160419/2035c019/attachment.html>
Alexander Potapenko via llvm-dev
2016-Apr-20 12:19 UTC
[llvm-dev] RFC: EfficiencySanitizer
On Tue, Apr 19, 2016 at 9:17 PM, Derek Bruening via llvm-dev <llvm-dev at lists.llvm.org> wrote:> On Tue, Apr 19, 2016 at 1:18 PM, Filipe Cabecinhas <filcab at gmail.com> wrote: > The shadow memory for tools we are considering include 64:1, 4:1, and 1:1. > This is all within the ranges of existing sanitizers. Each tool will have > inlined instrumentation to update shadow memory on either every memory > access or in some cases just focusing on the heap; I do not have a number > for code size expansion. The slowdown ranges were in the original email: > 2x-5x.Given that you're going to store metadata for 4-64 cache lines of user memory in a single shadow memory cache line, do you anticipate additional slowdown related to false sharing of shadow memory in multithreaded applications? Any plans to deal with that? -- Alexander Potapenko Software Engineer Google Germany GmbH Erika-Mann-Straße, 33 80636 München Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg
> > > Given that you're going to store metadata for 4-64 cache lines of user > memory in a single shadow memory cache line, do you anticipate > additional slowdown related to false sharing of shadow memory in > multithreaded applications? Any plans to deal with that? >The optimization of check-before-write would avoid most writes and so alleviate the false sharing problem.> > > -- > Alexander Potapenko > Software Engineer > > Google Germany GmbH > Erika-Mann-Straße, 33 > 80636 München > > Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle > Registergericht und -nummer: Hamburg, HRB 86891 > Sitz der Gesellschaft: Hamburg > > -- > You received this message because you are subscribed to the Google Groups > "efficiency-sanitizer" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to efficiency-sanitizer+unsubscribe at google.com. > To post to this group, send email to efficiency-sanitizer at google.com. > To view this discussion on the web visit > https://groups.google.com/a/google.com/d/msgid/efficiency-sanitizer/CAG_fn%3DUbdJhC439heRJ%3DtbVM8f5FKzBPTNkCxBQr0m9ObQV-fw%40mail.gmail.com > . >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160420/e7affc26/attachment.html>