llvm dev - Jun 2016 - XRay: Demo on x86_64/Linux almost done; some questions.

Hi,

The following three patches, when applied cleanly to LLVM+Clang+compiler-rt
will now allow for building and running binaries with a very simple version
of XRay included:

http://reviews.llvm.org/D21612 (compiler-rt)
http://reviews.llvm.org/D20352 (clang)
http://reviews.llvm.org/D19904 (llvm)

To use this on x86_64-unknown-linux-gnu you'd need to set the flags
'-fxray-instrument' and if you'd like to instrument all functions,
set
'-fxray-instruction-threshold=1'. Apply those flags to all your object
files, and link statically for best effect.

Now, some questions before I continue:

- What is the preferred way of controlling the behaviour of the runtime
library for something like XRay? Do the sanitizers check/use environment
variables or commandline-flags to control behaviour at runtime/init?
- We would like to be able to trigger the patching/unpatching routines at
runtime in a portable manner. In Linux and other UNIX-like environments
signals (SIGUSR1 and SIGUSR2) might be good candidates for signalling the
tracing infrastructure to start/stop at user-controlled times (useful for
long-running servers). Is there a preference for this, or are there
alternatives in this space that might make better sense?
- Currently we are using 'printf' for the logging, but could use a
simple
thread-local in-memory log that flushes to disk when full. Any other
preferred ways of doing this?
- Documentation for how to use/run XRay may need to live in a central
location, but since the changes to the LLVM pieces are currently in three
different places, are there suggestions for where the docs should live?

Cheers

PS. An example log of how to use XRay with clang+llvm+compiler-rt with the
above patches:

[16-06-28 17:21:02] dberris at dberris: ~/xray/llvm-build% cat test.cc
#include <cstdio>
#include <cassert>

[[clang::xray_always_instrument]] void foo() { std::printf("Hello,
XRay!\n"); }

[[clang::xray_never_instrument]] void bar() { std::printf("Not
instrumented\n"); }

extern void baz();  // defined in other.cc

int main(int argc, char* argv[]) {
  printf("main has started.\n");
  bar();
  foo();
  baz();
}
[16-06-28 17:30:13] dberris at dberris: ~/xray/llvm-build% cat other.cc
#include <cstdio>

[[clang::xray_always_instrument]] void baz() {
  std::printf("Welcome!\n");
}

[16-06-28 17:30:16] dberris at dberris: ~/xray/llvm-build% ./bin/clang -c
test.cc -x c++ -std=c++11 -fxray-instrument
[16-06-28 17:30:34] dberris at dberris: ~/xray/llvm-build% ./bin/clang -c
other.cc -x c++ -std=c++11 -fxray-instrument
[16-06-28 17:30:46] dberris at dberris: ~/xray/llvm-build% ./bin/clang -o
test.bin test.o other.o -fxray-instrument
[16-06-28 17:30:55] dberris at dberris: ~/xray/llvm-build% ./test.bin
__xray_instr_map at 0x400f5b..0x400fdb
400cf0  E       *       @function(400cf0)
400d1c  X       *       @function(400cf0)
400da0  E       *       @function(400da0)
400dcc  X       *       @function(400da0)
main has started.
Not instrumented
4133: [9699491610577808] E1
Hello, XRay!
4133: [9699491610592074] X1
4133: [9699491610599840] E2
Welcome!
4133: [9699491610611254] X2
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160628/a78ad57b/attachment.html>

On Tue, Jun 28, 2016 at 12:31 AM, Dean Michael Berris via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> - What is the preferred way of controlling the behaviour of the runtime
> library for something like XRay? Do the sanitizers check/use environment
> variables or commandline-flags to control behaviour at runtime/init?
>
The sanitizers each use a different, single environment variable:
https://github.com/google/sanitizers/wiki/SanitizerCommonFlags
IMO this works pretty well, and it supports multiple flags.

> - We would like to be able to trigger the patching/unpatching routines at
> runtime in a portable manner. In Linux and other UNIX-like environments
> signals (SIGUSR1 and SIGUSR2) might be good candidates for signalling the
> tracing infrastructure to start/stop at user-controlled times (useful for
> long-running servers). Is there a preference for this, or are there
> alternatives in this space that might make better sense?
>
No idea, maybe others have a better idea.

> - Currently we are using 'printf' for the logging, but could use a
simple
> thread-local in-memory log that flushes to disk when full. Any other
> preferred ways of doing this?
>
Should XRay use libc? Are there re-entrancy issues that we should be
worried about? Most sanitizer code avoids calling libc directly, and most
functionality (printf) is implemented internally. Maybe you should link
sanitizer_common and use that.

> - Documentation for how to use/run XRay may need to live in a central
> location, but since the changes to the LLVM pieces are currently in three
> different places, are there suggestions for where the docs should live?
>
The sanitizer docs are kind of spread out, unfortunately. There's some in
clang, some on the github wiki, and probably some in LLVM. Personally I
think the documentation should be in clang in rst.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160707/51100ae0/attachment.html>

On Fri, Jul 8, 2016 at 9:57 AM Reid Kleckner <rnk at google.com> wrote:
> On Tue, Jun 28, 2016 at 12:31 AM, Dean Michael Berris via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> - What is the preferred way of controlling the behaviour of the runtime
>> library for something like XRay? Do the sanitizers check/use
environment
>> variables or commandline-flags to control behaviour at runtime/init?
>>
>
> The sanitizers each use a different, single environment variable:
> https://github.com/google/sanitizers/wiki/SanitizerCommonFlags
> IMO this works pretty well, and it supports multiple flags.
>
>
Awesome, thanks!

> - We would like to be able to trigger the patching/unpatching routines at
>> runtime in a portable manner. In Linux and other UNIX-like environments
>> signals (SIGUSR1 and SIGUSR2) might be good candidates for signalling
the
>> tracing infrastructure to start/stop at user-controlled times (useful
for
>> long-running servers). Is there a preference for this, or are there
>> alternatives in this space that might make better sense?
>>
>
> No idea, maybe others have a better idea.
>
>
I suspect the bare minimum here would be to expose a single function that
could be invoked to enable or disable tracing. Then that could be called in
many different ways according to the application or through some supported,
configurable, with sane defaults for the platforms supported.

> - Currently we are using 'printf' for the logging, but could use a
simple
>> thread-local in-memory log that flushes to disk when full. Any other
>> preferred ways of doing this?
>>
>
> Should XRay use libc? Are there re-entrancy issues that we should be
> worried about? Most sanitizer code avoids calling libc directly, and most
> functionality (printf) is implemented internally. Maybe you should link
> sanitizer_common and use that.
>
>
I wondered about that, and I think avoiding libc as much as possible is a
good thing to do.

I'll work with sanitizer_common and re-use stuff from there instead, thanks
for the pointer. :)

> - Documentation for how to use/run XRay may need to live in a central
>> location, but since the changes to the LLVM pieces are currently in
three
>> different places, are there suggestions for where the docs should live?
>>
>
> The sanitizer docs are kind of spread out, unfortunately. There's some
in
> clang, some on the github wiki, and probably some in LLVM. Personally I
> think the documentation should be in clang in rst.
>
I think following current practice is fine for consistency (i.e. document
the many different parts in many different places) but then have some sort
of high-level documentation to present the use-cases and user scenarios.
I'm fine with the user-facing documentation to be in clang too (i.e. how to
xray-enable your application/libraries, then how to configure, etc.).

Would love to know what others think about this particular point, but will
proceed to add documentation through clang in rst.

Cheers
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160708/0d83f2f7/attachment.html>