llvm dev - Feb 2016 - Intel MPX support (instrumentation pass similar to gcc's Pointer Checker)

I've recently played with the GCC implementation of pointer checker on a
real hardware,
my recent impressions are here:
https://github.com/google/sanitizers/wiki/AddressSanitizerIntelMemoryProtectionExtensions

(there is also some old pre-hardware content).

In short, I totally agree with what David says above: MPX is a disaster.
(Usual disclaimer: my opinion here is too biased)

I am glad that LLVM already has the support for MPX instructions, but I see
no good reason to add the MPX-based checker to LLVM.
Yes, it will allow us to detect intra-object overflows, something that asan can
not do by default
<https://github.com/google/sanitizers/wiki/AddressSanitizerIntraObjectOverflow>,
but it's not worth the extreme complexity of the MPX-based checker.

--kcc

On Thu, Jan 28, 2016 at 9:14 AM, David Chisnall via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 28 Jan 2016, at 17:03, Dmitrii Kuvaiskii via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Interestingly, my
> > preliminary experiments indicate that adding MPX bounds checking via
> > Pointer Checker in gcc is usually slower than using software-only
> > AddressSanitizer.
>
> This corresponds with other results that I have seen.  The last time I
> looked at the output from gcc, it also did not generate pointer updates
> that were safe in the presence of concurrency (they must be bracketed in
> transactions if you want the MPX metadata and the pointer updates to be
> atomic) and the overhead of this is likely to be even more.
>
> I am particularly impressed with Intel for creating a hardware
> implementation that is both slower than a software-only version and can not
> (due to its fail-open policy being embedded in the hardware) be used for
> security.
>
> David
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160128/9fc58fb6/attachment.html>

I continue playing with Intel MPX and its support in modern compilers.
All experiments were done on the Alienware (Dell) 15 R2, Ubuntu 15.10
(linux 4.2.0), gcc version is 5.2.1, icc version 2016.1.150. The
benchmark suite is PARSEC 3.0, all versions with 1 thread and default
configs.

As I described previously, PointerChecker in gcc produces very
inefficient code. My experiments show overheads over native of up to
9.5X (on "raytrace"), with common overheads of 3X
("bodytrack",
"fluidanimate", "streamcluster"). At the same time,
AddressSanitizer
performs much better -- 1.3X on "raytrace", 1.7X on
"bodytrack" and so
on.

Recently I played with MPX support on Intel C/C++ Compiler (icc). This
implementation looks *much* better, with the following example
overheads: 1.2X on "raytrace", 1.25X on "bodytrack", 1.08X
on
"streamcluster". So the common overheads are in the range of 15%-25%!

Please note that gcc-mpx and gcc-asan versions were compared against
gcc-native, and icc-mpx version was compared against icc-native.

We would like to compile a small technical report with all our
measurements (performance and memory  overhead) and put it online.
We'll do it in the near future, I will write an update here when it's
done. Please tell me if anyone is interested in any specific
benchmarks (I want to test PARSEC and some case-studies: PostgreSQL,
Memcached, SQLite3). Any feedback is welcome.

On Thu, Jan 28, 2016 at 8:29 PM, Kostya Serebryany <kcc at google.com>
wrote:
> I've recently played with the GCC implementation of pointer checker on
a
> real hardware,
> my recent impressions are here:
>
https://github.com/google/sanitizers/wiki/AddressSanitizerIntelMemoryProtectionExtensions
> (there is also some old pre-hardware content).
>
> In short, I totally agree with what David says above: MPX is a disaster.
> (Usual disclaimer: my opinion here is too biased)
>
> I am glad that LLVM already has the support for MPX instructions, but I see
> no good reason to add the MPX-based checker to LLVM.
> Yes, it will allow us to detect intra-object overflows, something that asan
> can not do by default, but it's not worth the extreme complexity of the
> MPX-based checker.
>
> --kcc
>
> On Thu, Jan 28, 2016 at 9:14 AM, David Chisnall via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>
>> On 28 Jan 2016, at 17:03, Dmitrii Kuvaiskii via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>> >
>> > Interestingly, my
>> > preliminary experiments indicate that adding MPX bounds checking
via
>> > Pointer Checker in gcc is usually slower than using software-only
>> > AddressSanitizer.
>>
>> This corresponds with other results that I have seen.  The last time I
>> looked at the output from gcc, it also did not generate pointer updates
that
>> were safe in the presence of concurrency (they must be bracketed in
>> transactions if you want the MPX metadata and the pointer updates to be
>> atomic) and the overhead of this is likely to be even more.
>>
>> I am particularly impressed with Intel for creating a hardware
>> implementation that is both slower than a software-only version and can
not
>> (due to its fail-open policy being embedded in the hardware) be used
for
>> security.
>>
>> David
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>


-- 
Yours sincerely,
Dmitrii Kuvaiskii

On Wed, Feb 3, 2016 at 6:27 AM, Dmitrii Kuvaiskii <
Dmitrii.Kuvaiskii at tu-dresden.de> wrote:
> I continue playing with Intel MPX and its support in modern compilers.
> All experiments were done on the Alienware (Dell) 15 R2, Ubuntu 15.10
> (linux 4.2.0), gcc version is 5.2.1, icc version 2016.1.150. The
> benchmark suite is PARSEC 3.0, all versions with 1 thread and default
> configs.
>
> As I described previously, PointerChecker in gcc produces very
> inefficient code. My experiments show overheads over native of up to
> 9.5X (on "raytrace"), with common overheads of 3X
("bodytrack",
> "fluidanimate", "streamcluster"). At the same time,
AddressSanitizer
> performs much better -- 1.3X on "raytrace", 1.7X on
"bodytrack" and so
> on.
>
> Recently I played with MPX support on Intel C/C++ Compiler (icc). This
> implementation looks *much* better, with the following example
> overheads: 1.2X on "raytrace", 1.25X on "bodytrack",
1.08X on
> "streamcluster". So the common overheads are in the range of
15%-25%!
>
That's interesting.
Are you sure you are instrumenting both reads and writes with icc?

>  is not
> Please note that gcc-mpx and gcc-asan versions were compared against
> gcc-native, and icc-mpx version was compared against icc-native.
>
> We would like to compile a small technical report with all our
> measurements (performance and memory  overhead) and put it online.
> We'll do it in the near future, I will write an update here when
it's
> done. Please tell me if anyone is interested in any specific
> benchmarks (I want to test PARSEC and some case-studies: PostgreSQL,
> Memcached, SQLite3). Any feedback is welcome.
>
SPEC2006 is well know so it could be useful. Especially 483.xalancbmk
Besides, maybe you could take something that is not strictly a benchmark.
E.g. take pdfium_test (https://pdfium.googlesource.com/pdfium/) and feed
several large pdf files to it.

>
> On Thu, Jan 28, 2016 at 8:29 PM, Kostya Serebryany <kcc at
google.com> wrote:
> > I've recently played with the GCC implementation of pointer
checker on a
> > real hardware,
> > my recent impressions are here:
> >
>
https://github.com/google/sanitizers/wiki/AddressSanitizerIntelMemoryProtectionExtensions
> > (there is also some old pre-hardware content).
> >
> > In short, I totally agree with what David says above: MPX is a
disaster.
> > (Usual disclaimer: my opinion here is too biased)
> >
> > I am glad that LLVM already has the support for MPX instructions, but
I
> see
> > no good reason to add the MPX-based checker to LLVM.
> > Yes, it will allow us to detect intra-object overflows, something that
> asan
> > can not do by default, but it's not worth the extreme complexity
of the
> > MPX-based checker.
> >
> > --kcc
> >
> > On Thu, Jan 28, 2016 at 9:14 AM, David Chisnall via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> >>
> >> On 28 Jan 2016, at 17:03, Dmitrii Kuvaiskii via llvm-dev
> >> <llvm-dev at lists.llvm.org> wrote:
> >> >
> >> > Interestingly, my
> >> > preliminary experiments indicate that adding MPX bounds
checking via
> >> > Pointer Checker in gcc is usually slower than using
software-only
> >> > AddressSanitizer.
> >>
> >> This corresponds with other results that I have seen.  The last
time I
> >> looked at the output from gcc, it also did not generate pointer
updates
> that
> >> were safe in the presence of concurrency (they must be bracketed
in
> >> transactions if you want the MPX metadata and the pointer updates
to be
> >> atomic) and the overhead of this is likely to be even more.
> >>
> >> I am particularly impressed with Intel for creating a hardware
> >> implementation that is both slower than a software-only version
and can
> not
> >> (due to its fail-open policy being embedded in the hardware) be
used for
> >> security.
> >>
> >> David
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
> >
>
>
>
> --
> Yours sincerely,
> Dmitrii Kuvaiskii
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160203/e3949266/attachment.html>

On 2/3/16 6:27 AM, Dmitrii Kuvaiskii via llvm-dev wrote:
> I continue playing with Intel MPX and its support in modern compilers.
> All experiments were done on the Alienware (Dell) 15 R2, Ubuntu 15.10
> (linux 4.2.0), gcc version is 5.2.1, icc version 2016.1.150. The
> benchmark suite is PARSEC 3.0, all versions with 1 thread and default
> configs.
>
> As I described previously, PointerChecker in gcc produces very
> inefficient code. My experiments show overheads over native of up to
> 9.5X (on "raytrace"), with common overheads of 3X
("bodytrack",
> "fluidanimate", "streamcluster"). At the same time,
AddressSanitizer
> performs much better -- 1.3X on "raytrace", 1.7X on
"bodytrack" and so
> on.
>
> Recently I played with MPX support on Intel C/C++ Compiler (icc). This
> implementation looks *much* better, with the following example
> overheads: 1.2X on "raytrace", 1.25X on "bodytrack",
1.08X on
> "streamcluster". So the common overheads are in the range of
15%-25%!
>
> Please note that gcc-mpx and gcc-asan versions were compared against
> gcc-native, and icc-mpx version was compared against icc-native.
>
> We would like to compile a small technical report with all our
> measurements (performance and memory  overhead) and put it online.
> We'll do it in the near future, I will write an update here when
it's
> done. Please tell me if anyone is interested in any specific
> benchmarks (I want to test PARSEC and some case-studies: PostgreSQL,
> Memcached, SQLite3). Any feedback is welcome.
Two comments.

First, it would be interesting to compare SoftBound to MPX to see how 
the two compare.  To the best of my understanding, MPX is intended to be 
a hardware implementation that does what SoftBound does.

Second, when you write your report, please keep in mind that what Asan 
and MPX do are not really the same thing.  MPX is designed to track 
bounds information for each pointer while Asan only tracks where memory 
objects are located.  As a result, their security guarantees are not the 
same: MPX should be able to catch pointers that jump from one memory 
object into another while Asan does not (Asan distributes memory objects 
far away from each other in the address space so that out of bounds 
accesses are likely to point into unallocated memory).  The difference 
is subtle, but I think it's important.

Regards,

John Criswell

>
> On Thu, Jan 28, 2016 at 8:29 PM, Kostya Serebryany <kcc at
google.com> wrote:
>> I've recently played with the GCC implementation of pointer checker
on a
>> real hardware,
>> my recent impressions are here:
>>
https://github.com/google/sanitizers/wiki/AddressSanitizerIntelMemoryProtectionExtensions
>> (there is also some old pre-hardware content).
>>
>> In short, I totally agree with what David says above: MPX is a
disaster.
>> (Usual disclaimer: my opinion here is too biased)
>>
>> I am glad that LLVM already has the support for MPX instructions, but I
see
>> no good reason to add the MPX-based checker to LLVM.
>> Yes, it will allow us to detect intra-object overflows, something that
asan
>> can not do by default, but it's not worth the extreme complexity of
the
>> MPX-based checker.
>>
>> --kcc
>>
>> On Thu, Jan 28, 2016 at 9:14 AM, David Chisnall via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>>> On 28 Jan 2016, at 17:03, Dmitrii Kuvaiskii via llvm-dev
>>> <llvm-dev at lists.llvm.org> wrote:
>>>> Interestingly, my
>>>> preliminary experiments indicate that adding MPX bounds
checking via
>>>> Pointer Checker in gcc is usually slower than using
software-only
>>>> AddressSanitizer.
>>> This corresponds with other results that I have seen.  The last
time I
>>> looked at the output from gcc, it also did not generate pointer
updates that
>>> were safe in the presence of concurrency (they must be bracketed in
>>> transactions if you want the MPX metadata and the pointer updates
to be
>>> atomic) and the overhead of this is likely to be even more.
>>>
>>> I am particularly impressed with Intel for creating a hardware
>>> implementation that is both slower than a software-only version and
can not
>>> (due to its fail-open policy being embedded in the hardware) be
used for
>>> security.
>>>
>>> David
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>

-- 
John Criswell
Assistant Professor
Department of Computer Science, University of Rochester
http://www.cs.rochester.edu/u/criswell