llvm dev - Nov 2013 - [LLVMdev] Any objections to my importing GoogleMock to go with GoogleTest in LLVM?

On Tue, Nov 12, 2013 at 7:24 PM, Sean Silva <silvas at purdue.edu> wrote:
> Could you maybe give an example or two to whet our testing appetite?

It would honestly be simpler for me to write the tests after pulling it in
and point at them. The GoogleMock project has some good examples as well.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/e97774d0/attachment.html>

On 11/14/13 06:06 PM, Chandler Carruth wrote:
>
> On Tue, Nov 12, 2013 at 7:24 PM, Sean Silva <silvas at purdue.edu 
> <mailto:silvas at purdue.edu>> wrote:
>
>     Could you maybe give an example or two to whet our testing appetite?
>
>
> It would honestly be simpler for me to write the tests after pulling 
> it in and point at them. The GoogleMock project has some good examples 
> as well.
+1 for showing an example of something you can accomplish that wouldn't 
otherwise be (easily) possible.

Why not post a patch on reviews?

On Thu, Nov 14, 2013 at 6:06 AM, Chandler Carruth <chandlerc at
google.com>wrote:
>
> On Tue, Nov 12, 2013 at 7:24 PM, Sean Silva <silvas at purdue.edu>
wrote:
>
>> Could you maybe give an example or two to whet our testing appetite?
>
>
> It would honestly be simpler for me to write the tests after pulling it in
> and point at them. The GoogleMock project has some good examples as well.
>
Doesn't the entire operational behavior of the passmanager entirely boil
down to the ordering of particular method invocations on passes (and that
particular pieces of IR are passed between them)? It seems like it would be
possible to print designated strings when particular things happen, and
then use FileCheck to check the order (and there's even CHECK-DAG if you
want to be bulletproof in the presence of parallelism).

Or more usefully, I think it would be good to define an
"introspection" C++
interface for the pass manager, with a callback for each of various
interesting points in the operation of the pass manager. Not only would
that would be independently useful, but a trivial "logging" one would
allow
you to FileCheck every nook and cranny worth looking at.
A number of other use cases that come to mind:
- There are a couple really hacky (but useful) -debug-pass= options to opt
that would be nice to "properly implement", as well as being
accessible in
a clean way from C++.
- opt's -print-before-all, -print-after-all, etc.
- It would allow decoupling pass timing from the passmanager itself
(currently time-passes is implemented !!!with a global variable!!! btw)
- non-intrusively collect detailed information/statistics about the
optimization pipeline. For example, which pass is responsible for
eliminating the most functions? Which pass is responsible for eliminating
*this* instruction in my function?

Also, btw, a canonical machine-readable (e.g. YAML) way to represent the
set of passes to be run would also be good and would go hand-in-hand as
part of the logging functionality needed for testing.
That would end the madness of describing a set of passes as a sequence of
command line options to opt, rather than having a proper canonical
representation built into the PassManager.
It would have been really useful for part of the work I was doing this
Summer, which involved analyzing different pass combinations for LTO, and I
ended up essentially having text files with a bunch of opt commandline
options and doing "opt `cat foopasses.txt | sed '/^#/d' | tr
'\n' ' '`
bar.bc", and communicating with colleagues involved essentially pasting opt
commandline options to each other.
A flexible format would also open the door to later having parameterized
passes (something else I could have used this Summer for various things),
not to mention the ability to eliminate all the nasty cl::opt !!!global!!!
configurables that the various passes have. The interface for such options
would be essentially the same as cl::opt options currently, except instead
of `cl::opt<bool> EnableFoo("enable-foo")`, the pass is given
essentially a
map of strings to values (rather than having them be ad-hoc parsed through
global state); the pass then just does `this->EnableFoo
getOption<bool>(Opts, "enable-foo")` (`this` added for clarity
that it is a
member). A simple concrete example where this would be useful is running
two inlining passes in the same pass pipeline with different inlining
thresholds.


-- Sean Silva
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131115/449c5218/attachment.html>

On Fri, Nov 15, 2013 at 2:12 AM, Sean Silva <silvas at purdue.edu> wrote:
> - non-intrusively collect detailed information/statistics about the
> optimization pipeline. For example, which pass is responsible for
> eliminating the most functions? Which pass is responsible for eliminating
> *this* instruction in my function?
>
>
In general, improved observability into the optimization pipeline can have
possibly huge knock-on effects for improving it.

Not entirely kidding:

A sort of "pass-pseudo-dtrace" could be implemented in a nicely
independent
way by defining "probes" at the various locations exposed by the
"introspection" API (no actual dynamic instrumentation; it just
executes
stuff in those "introspection" callbacks). Silly strawman oneliner:
    opt -pass-pseudo-dtrace='pass:*::before { self->shl_before
curmodule["shl"].count() } pass:*::after
/curmodule["shl"].count() <self->shl_before -  5/ {
print(curpassname, "managed to eliminate more than
5 shl's!") }' foo.bc
My experience with dtrace is that such trivially available observability
and powerful ad-hoc queries (and the deeper investigations they lead to and
facilitate) are the single most effective way to identify
previously-unknown things that could be improved (or that are broken) in a
system.
Even purely as a learning tool it might be worth having that.

How else would you answer the question "which passes eliminate shl's in
a
typical compilation"? (and that's just one of an open-ended class of
queries that such a system would facilitate).

AFAIK the only way to currently answer that question is to drop down to
C++, write a pass that gathers the statistics, hack up opt to insert these
instrumentation passes between passes in the desired pipeline (or do some
horrible brittle commandline transformation), rebuild things (possibly
rebuilding most of LLVM if you don't already have an incremental build
hot): end result is that it would take at least an order of magnitude more
time (or even 2). If you have to shave a yak to get an answer, you ask
fewer questions.

(Note: while you may be able to name some passes that eliminate shl's off
the top of your head, it's precisely the ones that you *can't* think of
off
the top of your head that have the potential to expose unexpected
properties of the system). Adding &&'s and ||'s quickly moves
you into a
realm where even gurus can at best guess: maybe a step up in complexity is:
partition the set of passes into the 9 categories defined by
"{add,remove,not change} load's && {add,remove,not change}
shl's", are
there any categories that are unpopulated?. And conjunctions and
disjunctions barely scratch the surface of what such a "dtrace-like"
system
can easily access (e.g. how does the average integer size in the module
change throughout the optimization pipeline? generate data that can be used
for plotting a line-graph. Which passes have the largest impact on average
integer size?).

-- Sean Silva
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131115/ed01b580/attachment.html>