llvm dev - Dec 2017 - pie and sanitizers

We recently set up a new powerpc system with Ubuntu 17.10 for llvm 
development work.  It works ok for llvm except that most of the msan and 
tsan tests fail.  From a bit of experimentation and reading it looks 
like pie is the culprit.


Ubuntu now uses pie by default and they even have a note about it:

"Failures using ThreadSanitizer

Another source of build failures occurs when building with 
ThreadSanitizer (aka TSAN), which doesn't work with pie binaries. These 
packages need to disable via -no-pie. "

(from https://wiki.ubuntu.com/SecurityTeam/PIE)


So I tried forcing -no-pie on the msan amd tsan tests and the msan tests 
all worked but one (which seems to be failing for some other reason) and 
many of the failing tsan ones work as well (I need to further 
investigate the other failures).

Looking at the virtual addresses that are used with pie on this new 
system I don't think that I can adjust the ranges that msan and tsan use 
to accommodate things. For example on an older Ubuntu 16.04 system (with 
pie forced on) a trivial little test program run multiple times sees low 
memory addresses in the range

0x0000000100000984 to 0x000000013FFF0984

while on the 17.10 system

0x00000001002409B4 to 0x00001000FFF409B4

which is orders of magnitude larger.

I am not sure why the range is so large.  On another non-Ubuntu system 
with similar kernel and binutils versions I don't see this.  So, any 
ideas on how to handle this?

For immediate needs is there some way to locally add options when 
testing besides modifying the lit.cfg files?
-- 

-Bill Seurer

On Fri, Dec 22, 2017 at 6:56 AM, Bill Seurer via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> We recently set up a new powerpc system with Ubuntu 17.10 for llvm
> development work.  It works ok for llvm except that most of the msan and
> tsan tests fail.  From a bit of experimentation and reading it looks like
> pie is the culprit.
>
>
> Ubuntu now uses pie by default and they even have a note about it:
>
> "Failures using ThreadSanitizer
>
> Another source of build failures occurs when building with ThreadSanitizer
> (aka TSAN), which doesn't work with pie binaries. These packages need
to
> disable via -no-pie. "
>
> (from https://wiki.ubuntu.com/SecurityTeam/PIE)
>
>
> So I tried forcing -no-pie on the msan amd tsan tests and the msan tests
all
> worked but one (which seems to be failing for some other reason) and many
of
> the failing tsan ones work as well (I need to further investigate the other
> failures).
>
> Looking at the virtual addresses that are used with pie on this new system
I
> don't think that I can adjust the ranges that msan and tsan use to
> accommodate things. For example on an older Ubuntu 16.04 system (with pie
> forced on) a trivial little test program run multiple times sees low memory
> addresses in the range
>
> 0x0000000100000984 to 0x000000013FFF0984
>
> while on the 17.10 system
>
> 0x00000001002409B4 to 0x00001000FFF409B4
>
> which is orders of magnitude larger.
>
> I am not sure why the range is so large.  On another non-Ubuntu system with
> similar kernel and binutils versions I don't see this.  So, any ideas
on how
> to handle this?
I'm pretty sure Ubuntu has cherry-picked a kernel change that
increases executable ASLR range. The same happened on x86_64 recently.
On x86 it only affected ASan, not MSan or TSan.

You two options are
(1) Fix the mapping somehow. 0x00001000FFF409B4 is still only ~1/8th
of the 48-bit address space, and theoretic upper limit is 1/3 (1x for
shadow and 1x for origins).
(2) Implement dynamic shadow. We've recently come up with a more
efficient way of doing it in ASan using ifunc resolver to put the
shadow start address directly in the GOT table, where it can be
accessed with just one PC-relative load. See PremapShadow(). That's
still a performance hit, of course.
>
> For immediate needs is there some way to locally add options when testing
> besides modifying the lit.cfg files?
> --
>
> -Bill Seurer
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev