llvm dev - Feb 2016 - Intel MPX support (instrumentation pass similar to gcc's Pointer Checker)

Dmitrii, all,

Please note, that GCC 5.3 had a significant update to the MPX code quality
- please, use this version as reference.

Regards,
Sergos

On Tue, Feb 9, 2016 at 12:49 AM, Kostya Serebryany via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>
> On Thu, Feb 4, 2016 at 10:40 AM, Kostya Serebryany <kcc at
google.com> wrote:
>
>>
>>
>> On Thu, Feb 4, 2016 at 4:59 AM, Dmitrii Kuvaiskii <
>> Dmitrii.Kuvaiskii at tu-dresden.de> wrote:
>>
>>> >> Recently I played with MPX support on Intel C/C++ Compiler
(icc). This
>>> >> implementation looks *much* better, with the following
example
>>> >> overheads: 1.2X on "raytrace", 1.25X on
"bodytrack", 1.08X on
>>> >> "streamcluster". So the common overheads are in
the range of 15%-25%!
>>> > That's interesting.
>>> > Are you sure you are instrumenting both reads and writes with
icc?
>>>
>>> Yes, here are the exact flags I add to the usual build
configuration:
>>>   -xHOST -check-pointers-mpx:rw
>>>
>>
>> Interesting, looking forward to reading your report!
>>
>>>
>>> Note "rw" which stands for protecting read and write
accesses. In the
>>> future, I will analyze how different flags affect ASan /
SoftBoundCETS
>>> / gcc-mpx / icc-mpx.
>>> I will also use a set of microbenchmarks/benchmarks (e.g., RIPE) to
>>> test the protection provided.
>>>
>>> > SPEC2006 is well know so it could be useful. Especially
483.xalancbmk
>>> > Besides, maybe you could take something that is not strictly a
>>> benchmark.
>>> > E.g. take pdfium_test
(https://pdfium.googlesource.com/pdfium/) and
>>> feed
>>> > several large pdf files to it.
>>>
>>> Thanks, I will report the SPEC2006 numbers as well.
>>>
>>>
>> Note that SPEC2006 has several know bugs that trigger under asan.
>>
>>
https://github.com/google/sanitizers/wiki/AddressSanitizerRunningSpecBenchmarks
>> has a patch that makes SPEC2006 pass with asan.
>> Some of these bugs and maybe others may also trigger with an MPX
checker.
>>
>
> Another note: please also try to document the memory footprint.
> One of unfortunate features of MPX is its large metadata storage which may
> in
> theory consume as much as 4x more RAM than the application itself.
>
> --kcc
>
>
>>
>> --kcc
>>
>> --
>>> Yours sincerely,
>>> Dmitrii Kuvaiskii
>>>
>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160209/a5042519/attachment.html>

Thank you Sergey and Konstantin for useful suggestions. We are
currently bootstrapping the infrastructure for our experiments. We
would like to make a sufficiently comprehensive report, with not only
the performance/memory overhead numbers, but also discussing and
evaluating security guarantees. I will also examine the available
source codes (ASan, gcc-mpx, SoftBound) and will spend some pages on a
discussion of the different approaches (trying to do science, you see
:)).

Btw, I will target only deterministic memory-safety no-code-changes
approaches that protect against spatial errors (I will probably
include also ASan and SoftBoundCETS with temporal errors' protection
in the results as well). The only technique (except Pointer Checker,
ASan, and SoftBound) I know of is Baggy Bounds Checking from MSR, but
it seems to be closed-source and Windows-oriented. If anyone can
suggest some other technique that could be evaluated here, please
inform me.

Anyway, before putting the techreport online, I will send the draft to
everyone who took part in this conversation, just to be on the safe
side and correct any bugs/wrong conclusions.

On Tue, Feb 9, 2016 at 3:24 PM, Sergey Ostanevich <sergos.gnu at
gmail.com> wrote:
> Dmitrii, all,
>
> Please note, that GCC 5.3 had a significant update to the MPX code quality
-
> please, use this version as reference.
>
> Regards,
> Sergos
>
> On Tue, Feb 9, 2016 at 12:49 AM, Kostya Serebryany via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>
>>
>>
>> On Thu, Feb 4, 2016 at 10:40 AM, Kostya Serebryany <kcc at
google.com> wrote:
>>>
>>>
>>>
>>> On Thu, Feb 4, 2016 at 4:59 AM, Dmitrii Kuvaiskii
>>> <Dmitrii.Kuvaiskii at tu-dresden.de> wrote:
>>>>
>>>> >> Recently I played with MPX support on Intel C/C++
Compiler (icc).
>>>> >> This
>>>> >> implementation looks *much* better, with the following
example
>>>> >> overheads: 1.2X on "raytrace", 1.25X on
"bodytrack", 1.08X on
>>>> >> "streamcluster". So the common overheads are
in the range of 15%-25%!
>>>> > That's interesting.
>>>> > Are you sure you are instrumenting both reads and writes
with icc?
>>>>
>>>> Yes, here are the exact flags I add to the usual build
configuration:
>>>>   -xHOST -check-pointers-mpx:rw
>>>
>>>
>>> Interesting, looking forward to reading your report!
>>>>
>>>>
>>>> Note "rw" which stands for protecting read and write
accesses. In the
>>>> future, I will analyze how different flags affect ASan /
SoftBoundCETS
>>>> / gcc-mpx / icc-mpx.
>>>> I will also use a set of microbenchmarks/benchmarks (e.g.,
RIPE) to
>>>> test the protection provided.
>>>>
>>>> > SPEC2006 is well know so it could be useful. Especially
483.xalancbmk
>>>> > Besides, maybe you could take something that is not
strictly a
>>>> > benchmark.
>>>> > E.g. take pdfium_test
(https://pdfium.googlesource.com/pdfium/) and
>>>> > feed
>>>> > several large pdf files to it.
>>>>
>>>> Thanks, I will report the SPEC2006 numbers as well.
>>>>
>>>
>>> Note that SPEC2006 has several know bugs that trigger under asan.
>>>
>>>
https://github.com/google/sanitizers/wiki/AddressSanitizerRunningSpecBenchmarks
>>> has a patch that makes SPEC2006 pass with asan.
>>> Some of these bugs and maybe others may also trigger with an MPX
checker.
>>
>>
>> Another note: please also try to document the memory footprint.
>> One of unfortunate features of MPX is its large metadata storage which
may
>> in
>> theory consume as much as 4x more RAM than the application itself.
>>
>> --kcc
>>
>>>
>>>
>>> --kcc
>>>
>>>> --
>>>> Yours sincerely,
>>>> Dmitrii Kuvaiskii
>>>
>>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-- 
Yours sincerely,
Dmitrii Kuvaiskii

On Tue, Feb 9, 2016 at 7:22 AM, Dmitrii Kuvaiskii <
Dmitrii.Kuvaiskii at tu-dresden.de> wrote:
> Thank you Sergey and Konstantin for useful suggestions. We are
> currently bootstrapping the infrastructure for our experiments. We
> would like to make a sufficiently comprehensive report, with not only
> the performance/memory overhead numbers, but also discussing and
> evaluating security guarantees. I will also examine the available
> source codes (ASan, gcc-mpx, SoftBound) and will spend some pages on a
> discussion of the different approaches (trying to do science, you see
> :)).
>
> Btw, I will target only deterministic memory-safety no-code-changes
> approaches that protect against spatial errors (I will probably
> include also ASan and SoftBoundCETS with temporal errors' protection
> in the results as well). The only technique (except Pointer Checker,
> ASan, and SoftBound) I know of is Baggy Bounds Checking from MSR, but
> it seems to be closed-source and Windows-oriented. If anyone can
> suggest some other technique that could be evaluated here, please
> inform me.
>
There is also a family of tools originated from Electric Fence
<http://elinux.org/Electric_Fence>,
they mostly have historical interest due to huge slowdown/memory
consumption.

Are you looking for bug detection mechanisms, or also for production
hardening techniques?
ASan is a bug detection tool. ASan can
<https://blog.torproject.org/blog/tor-browser-55a4-hardened-released> be
used for hardening, but that's not it's primary purpose.
Same is true (IMHO) about Pointer Checker and SoftBound.

Hardening is an entirely different subject, although there is a bit of
intersection,
e.g. I know that parts of UBSan (-fsanitize=signed-integer-overflow) are
used for hardening.
In LLVM, also have a look at clang.llvm.org/docs/ControlFlowIntegrity.html
and
http://clang.llvm.org/docs/SafeStack.html
>
> Anyway, before putting the techreport online, I will send the draft to
> everyone who took part in this conversation, just to be on the safe
> side and correct any bugs/wrong conclusions.
>
I would appreciate this.

--kcc

>
> On Tue, Feb 9, 2016 at 3:24 PM, Sergey Ostanevich <sergos.gnu at
gmail.com>
> wrote:
> > Dmitrii, all,
> >
> > Please note, that GCC 5.3 had a significant update to the MPX code
> quality -
> > please, use this version as reference.
> >
> > Regards,
> > Sergos
> >
> > On Tue, Feb 9, 2016 at 12:49 AM, Kostya Serebryany via llvm-dev
> > <llvm-dev at lists.llvm.org> wrote:
> >>
> >>
> >>
> >> On Thu, Feb 4, 2016 at 10:40 AM, Kostya Serebryany <kcc at
google.com>
> wrote:
> >>>
> >>>
> >>>
> >>> On Thu, Feb 4, 2016 at 4:59 AM, Dmitrii Kuvaiskii
> >>> <Dmitrii.Kuvaiskii at tu-dresden.de> wrote:
> >>>>
> >>>> >> Recently I played with MPX support on Intel C/C++
Compiler (icc).
> >>>> >> This
> >>>> >> implementation looks *much* better, with the
following example
> >>>> >> overheads: 1.2X on "raytrace", 1.25X on
"bodytrack", 1.08X on
> >>>> >> "streamcluster". So the common
overheads are in the range of
> 15%-25%!
> >>>> > That's interesting.
> >>>> > Are you sure you are instrumenting both reads and
writes with icc?
> >>>>
> >>>> Yes, here are the exact flags I add to the usual build
configuration:
> >>>>   -xHOST -check-pointers-mpx:rw
> >>>
> >>>
> >>> Interesting, looking forward to reading your report!
> >>>>
> >>>>
> >>>> Note "rw" which stands for protecting read and
write accesses. In the
> >>>> future, I will analyze how different flags affect ASan /
SoftBoundCETS
> >>>> / gcc-mpx / icc-mpx.
> >>>> I will also use a set of microbenchmarks/benchmarks (e.g.,
RIPE) to
> >>>> test the protection provided.
> >>>>
> >>>> > SPEC2006 is well know so it could be useful.
Especially
> 483.xalancbmk
> >>>> > Besides, maybe you could take something that is not
strictly a
> >>>> > benchmark.
> >>>> > E.g. take pdfium_test
(https://pdfium.googlesource.com/pdfium/) and
> >>>> > feed
> >>>> > several large pdf files to it.
> >>>>
> >>>> Thanks, I will report the SPEC2006 numbers as well.
> >>>>
> >>>
> >>> Note that SPEC2006 has several know bugs that trigger under
asan.
> >>>
> >>>
>
https://github.com/google/sanitizers/wiki/AddressSanitizerRunningSpecBenchmarks
> >>> has a patch that makes SPEC2006 pass with asan.
> >>> Some of these bugs and maybe others may also trigger with an
MPX
> checker.
> >>
> >>
> >> Another note: please also try to document the memory footprint.
> >> One of unfortunate features of MPX is its large metadata storage
which
> may
> >> in
> >> theory consume as much as 4x more RAM than the application itself.
> >>
> >> --kcc
> >>
> >>>
> >>>
> >>> --kcc
> >>>
> >>>> --
> >>>> Yours sincerely,
> >>>> Dmitrii Kuvaiskii
> >>>
> >>>
> >>
> >>
> >> _______________________________________________
> >> LLVM Developers mailing list
> >> llvm-dev at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >
>
> --
> Yours sincerely,
> Dmitrii Kuvaiskii
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160209/e0e65fbb/attachment.html>