llvm dev - Oct 2017 - Policy for compiler-rt ABI stability and external dependencies?

Hi Kostya, Evgenii, and David,

Recently I've been making some incremental changes to the XRay runtime
implementation in compiler-rt to reduce the reliance on the C++ standard library
components that might have external linkage dependencies. This involves not
using containers from the STL and not using non-trivially destructible C++11
thread_local objects.

I was wondering whether the sanitisers follow an explicit policy of not
depending on the C++ standard library or avoiding linkage/ABI issues that come
up when building the sanitisers with C++ standard library dependencies. If it
does, do we document these policies somewhere? If we don't document it
somewhere, does it make sense for us to do so and maybe potentially enforce it
at build time?

One of the policies I can think of is not to use *anything* from the C++
standard library even if they are templates anyway (e.g.
std::unique_ptr<...>, std::shared_ptr<...>, std::tuple<...>,
std::aligned_storage<...>, etc.). The downside to this would be myriad,
where we'd need to implement analogues of these ourselves or resort to only
things we can use from libc. I'm not sure how this could be automatically
enforced, maybe potentially through a blacklist on `std::` types.

A slightly more permissive policy might be to allow using some of these types
but not in any of the exposed interfaces and not require any runtime
dependencies from the C++ standard library. This might be possible to enforce by
checking the external/dymamic symbols in the runtime archives. I'm not sure
though whether it's even safe to assume that the templates wouldn't
introduce some weird ABI constraint even if the types don't cross the API
boundaries.

I'm curious what the sanitisers currently do and what techniques/conventions
we can follow on the XRay implementation that would be helpful to emulate as
well.

In a similar vein, how do the sanitisers handle making ABI breaking changes? For
the published types, is there a symbol naming policy and explicit
deprecation/support horizon? How about API stability, do we intend to keep the
API stable across N stable releases?

Your thoughts and feedback would be most appreciated.

Cheers

-- Dean

Hi Dean.

The sanitizers don't use any of the C++ std library.
More than that, they don't include *any* system headers in most of the
sources (exception is some OS-dependent .cc files).

This rule is *somewhat* documented, e.g.
tsan/rtl/tsan_rtl.h:

//   - No system headers included in header files (*).
//   - Platform specific headres included only into platform-specific files
(*).

It's enforced only by code review, although if one attempts to use stl in
e.g. asan some tests will most likely break.
If you can suggest a simple way to enforce this at build time -- let's do
it.

libFuzzer on the contrary uses STL, including the containers, and it causes
all sorts of problems.
(I've anticipated it from the beginning, but just couldn't force myself
to
not use STL).

My recommendation is to ban all of C++ std lib in xray unless you find a
way to "privatize" the bits of STL you are using.
Ideally, those bits will be header-only and you'll be able to change the
namespace for STL.
See e.g. a long discussion in https://reviews.llvm.org/D37631 (trying to
privatize STL in libFuzzer)



On Wed, Oct 11, 2017 at 4:07 AM, Dean Michael Berris <dean.berris at
gmail.com>
wrote:
> Hi Kostya, Evgenii, and David,
>
> Recently I've been making some incremental changes to the XRay runtime
> implementation in compiler-rt to reduce the reliance on the C++ standard
> library components that might have external linkage dependencies. This
> involves not using containers from the STL and not using non-trivially
> destructible C++11 thread_local objects.
>
> I was wondering whether the sanitisers follow an explicit policy of not
> depending on the C++ standard library or avoiding linkage/ABI issues that
> come up when building the sanitisers with C++ standard library
> dependencies. If it does, do we document these policies somewhere? If we
> don't document it somewhere, does it make sense for us to do so and
maybe
> potentially enforce it at build time?
>
> One of the policies I can think of is not to use *anything* from the C++
> standard library even if they are templates anyway (e.g.
> std::unique_ptr<...>, std::shared_ptr<...>,
std::tuple<...>,
> std::aligned_storage<...>, etc.). The downside to this would be
myriad,
> where we'd need to implement analogues of these ourselves or resort to
only
> things we can use from libc. I'm not sure how this could be
automatically
> enforced, maybe potentially through a blacklist on `std::` types.
>
> A slightly more permissive policy might be to allow using some of these
> types but not in any of the exposed interfaces and not require any runtime
> dependencies from the C++ standard library. This might be possible to
> enforce by checking the external/dymamic symbols in the runtime archives.
> I'm not sure though whether it's even safe to assume that the
templates
> wouldn't introduce some weird ABI constraint even if the types
don't cross
> the API boundaries.
>
> I'm curious what the sanitisers currently do and what
> techniques/conventions we can follow on the XRay implementation that would
> be helpful to emulate as well.
>
> In a similar vein, how do the sanitisers handle making ABI breaking
> changes? For the published types, is there a symbol naming policy and
> explicit deprecation/support horizon? How about API stability, do we intend
> to keep the API stable across N stable releases?
>
We generally don't promise compatibility between the compiler and rt from
different revisions.
However, just in case, we do a version mismatch check (e.g. see
lib/asan/asan_init_version.h)

--kcc

>
> Your thoughts and feedback would be most appreciated.
>
> Cheers
>
> -- Dean
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20171011/f8a84c66/attachment.html>

> On 12 Oct 2017, at 03:30, Kostya Serebryany <kcc at google.com>
wrote:
> 
> Hi Dean. 
> 
> The sanitizers don't use any of the C++ std library. 
> More than that, they don't include *any* system headers in most of the
sources (exception is some OS-dependent .cc files).
> 
> This rule is *somewhat* documented, e.g. 
> tsan/rtl/tsan_rtl.h:
> 
> //   - No system headers included in header files (*).
> //   - Platform specific headres included only into platform-specific files
(*).
> 
> It's enforced only by code review, although if one attempts to use stl
in e.g. asan some tests will most likely break.
> If you can suggest a simple way to enforce this at build time -- let's
do it.
> 
Thanks for confirming Kostya, this really helps!

I'm not sure how to prevent this, but we can look at the generated
archives/shared-objects and find references to known symbols from the C++
standard library (or even if there are references to specific files/.so's).
Let me think about it a bit more, but it might be good to have this conversation
in person, perhaps this coming week.

Documentation would probably also help, if this is in some .rst document
that's available from llvm.org/docs <http://llvm.org/docs>.
> libFuzzer on the contrary uses STL, including the containers, and it causes
all sorts of problems.
> (I've anticipated it from the beginning, but just couldn't force
myself to not use STL).
> 
> My recommendation is to ban all of C++ std lib in xray unless you find a
way to "privatize" the bits of STL you are using.
> Ideally, those bits will be header-only and you'll be able to change
the namespace for STL.
> See e.g. a long discussion in https://reviews.llvm.org/D37631
<https://reviews.llvm.org/D37631> (trying to privatize STL in libFuzzer)
> 
I think this may be the best option for now as well.

I'll go ahead and spend some time to remove the std lib references in the
XRay runtime, and implement analogues for the things we actually really need in
the implementation.
> 
> 
> On Wed, Oct 11, 2017 at 4:07 AM, Dean Michael Berris <dean.berris at
gmail.com <mailto:dean.berris at gmail.com>> wrote:
> Hi Kostya, Evgenii, and David,
> 
> Recently I've been making some incremental changes to the XRay runtime
implementation in compiler-rt to reduce the reliance on the C++ standard library
components that might have external linkage dependencies. This involves not
using containers from the STL and not using non-trivially destructible C++11
thread_local objects.
> 
> I was wondering whether the sanitisers follow an explicit policy of not
depending on the C++ standard library or avoiding linkage/ABI issues that come
up when building the sanitisers with C++ standard library dependencies. If it
does, do we document these policies somewhere? If we don't document it
somewhere, does it make sense for us to do so and maybe potentially enforce it
at build time?
> 
> One of the policies I can think of is not to use *anything* from the C++
standard library even if they are templates anyway (e.g.
std::unique_ptr<...>, std::shared_ptr<...>, std::tuple<...>,
std::aligned_storage<...>, etc.). The downside to this would be myriad,
where we'd need to implement analogues of these ourselves or resort to only
things we can use from libc. I'm not sure how this could be automatically
enforced, maybe potentially through a blacklist on `std::` types.
> 
> A slightly more permissive policy might be to allow using some of these
types but not in any of the exposed interfaces and not require any runtime
dependencies from the C++ standard library. This might be possible to enforce by
checking the external/dymamic symbols in the runtime archives. I'm not sure
though whether it's even safe to assume that the templates wouldn't
introduce some weird ABI constraint even if the types don't cross the API
boundaries.
> 
> I'm curious what the sanitisers currently do and what
techniques/conventions we can follow on the XRay implementation that would be
helpful to emulate as well.
> 
> In a similar vein, how do the sanitisers handle making ABI breaking
changes? For the published types, is there a symbol naming policy and explicit
deprecation/support horizon? How about API stability, do we intend to keep the
API stable across N stable releases?
> 
> We generally don't promise compatibility between the compiler and rt
from different revisions.
> However, just in case, we do a version mismatch check (e.g. see
lib/asan/asan_init_version.h)
> 
Cool stuff, thanks!

It probably also helps to be explicit here, so that we can communicate it better
to users.

Right now, it's been implicit that XRay might change APIs.

Would be nice to have in some central documentation too. :)

Cheers

-- Dean

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20171014/5b758a7f/attachment.html>