llvm dev - Dec 2015 - Trouble supressing ASAN reported leaks

Hi,

I'm currently trying to find and fix memory leaks (compiling with
``-fsanitize=address``) in the KLEE tool [1] an having found some
leaks and I'm having trouble suppressing them.

I'm trying to suppress them using the
``-fsanitize-blacklist=blacklist.txt`` option as documented at
[2]. I'm using Clang 3.7 ( Arch Linux package 3.7.0-6).

The sort of reported leaks I see are

```
==9912==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x4df4a0 in operator new(unsigned long)
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x4df4a0)
    #1 0x4f76e1 in
klee::Array::CreateArray(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&,
unsigned long,
klee::ref<klee::ConstantExpr> const*, klee::ref<klee::ConstantExpr>
const*, unsigned int, unsigned int)
/home/dsl11/dev/klee/klee/src/lib/Expr/Expr.cpp:522:16
    #2 0x4e30d5 in (anonymous
namespace)::ExprTest_ConcatExtract_Test::TestBody()
/home/dsl11/dev/klee/klee/src/unittests/Expr/ExprTest.cpp:34:25
    #3 0x526410 in testing::Test::Run()
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x526410)

...

Indirect leak of 80 byte(s) in 1 object(s) allocated from:
    #0 0x4df4a0 in operator new(unsigned long)
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x4df4a0)
    #1 0x4f75ce in
klee::Array::CreateArray(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&,
unsigned long,
klee::ref<klee::ConstantExpr> const*, klee::ref<klee::ConstantExpr>
const*, unsigned int, u
nsigned int) /home/dsl11/dev/klee/klee/src/lib/Expr/Expr.cpp:506:25
    #2 0x4e2ff3 in (anonymous
namespace)::ExprTest_ConcatExtract_Test::TestBody()
/home/dsl11/dev/klee/klee/src/unittests/Expr/ExprTest.cpp:32:24
    #3 0x526410 in testing::Test::Run()
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x526410)
```

The source of the trouble is this static object.

```
std::map<unsigned, std::vector<const Array *> *>
Array::symbolicArraySingletonMap;
```

Neither the ``std::vector<const Array*>`` pointers or the ``const
Array`` pointers are being freed. Sure this code is bad (don't blame
me, I didn't write it), but I want to skip over this leak to find more
interesting issues.

I can't seem to suppress it though. I've tried putting the following
in the ``blacklist.txt`` file

* Explicitly naming the source file, like this

src:/home/dsl11/dev/klee/klee/src/lib/Expr/Expr.cpp

* Naming the function where the leak originates (demangled)

fun:klee::Array::CreateArray

* Naming the function where the leak originates (mangled)

fun:_ZN4klee5Array11CreateArrayERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmPKNS_3refINS_12ConstantExprEEESD_jj

* Naming the global (demangled)

global:klee::Array::symbolicArraySingletonMap

* Naming the global (mangled)

global:_ZN4klee5Array25symbolicArraySingletonMapE


None of these succeed in suppressing the error. Does anyone have any
idea what I'm doing wrong?

Note I'm also using a fairly new build of libstdc++ which is using a
new ABI [3]. I'm not sure if this would cause problems.

[1] https://github.com/klee/klee
[2]
http://clang.llvm.org/docs/AddressSanitizer.html#suppressing-errors-in-recompiled-code-blacklist
[3] https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html

Thanks,
Dan.

Hi Dan,

First of all, sorry for apparently misleading documentation. Currently you
*can't* use blacklist
to suppress memory leak reports: blacklist is applied at compile-time, and
leak detection
machinery brought up at run-time doesn't know about it.

We should probably fix a documentation, as a first step.

What you can use is:
1) Runtime suppression: see
https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions

You may pass environment variable LSAN_OPTIONS=suppressions=suppr.txt
and try to specify smth. like
leak:klee::Array::CreateArray
or
leak:lib/Expr/Expr.cpp
in suppr.txt

2) You see the error report because LSan is invoked *after* all global
destructors. So, you have
std::map<unsigned, std::vector<const Array *>
*> Array::symbolicArraySingletonMap;
which contains pointers as values. When the map is destroyed, it's cleared,
and the allocated objects are
no longer reachable. As a quick-fix, you can replace a global std::map with
*a pointer* to global std::map
which will also never be deleted. In that way all your objects will be
reachable until the shutdown, and ASan/LSan will be silent.

3) You may schedule to run leak detection at an earlier point - e.g. as the
last statement of main(): call __lsan_do_leak_check()
from <sanitizer/lsan_interface.h>. This would run the leak detection
before
the global destructors, i.e. before std::map is destroyed

Hope that helps.


On Tue, Dec 15, 2015 at 10:58 AM, Dan Liew via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> I'm currently trying to find and fix memory leaks (compiling with
> ``-fsanitize=address``) in the KLEE tool [1] an having found some
> leaks and I'm having trouble suppressing them.
>
> I'm trying to suppress them using the
> ``-fsanitize-blacklist=blacklist.txt`` option as documented at
> [2]. I'm using Clang 3.7 ( Arch Linux package 3.7.0-6).
>
> The sort of reported leaks I see are
>
> ```
> ==9912==ERROR: LeakSanitizer: detected memory leaks
>
> Direct leak of 24 byte(s) in 1 object(s) allocated from:
>     #0 0x4df4a0 in operator new(unsigned long)
>
>
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x4df4a0)
>     #1 0x4f76e1 in
> klee::Array::CreateArray(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
unsigned long,
> klee::ref<klee::ConstantExpr> const*,
klee::ref<klee::ConstantExpr>
> const*, unsigned int, unsigned int)
> /home/dsl11/dev/klee/klee/src/lib/Expr/Expr.cpp:522:16
>     #2 0x4e30d5 in (anonymous
> namespace)::ExprTest_ConcatExtract_Test::TestBody()
> /home/dsl11/dev/klee/klee/src/unittests/Expr/ExprTest.cpp:34:25
>     #3 0x526410 in testing::Test::Run()
>
>
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x526410)
>
> ...
>
> Indirect leak of 80 byte(s) in 1 object(s) allocated from:
>     #0 0x4df4a0 in operator new(unsigned long)
>
>
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x4df4a0)
>     #1 0x4f75ce in
> klee::Array::CreateArray(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
unsigned long,
> klee::ref<klee::ConstantExpr> const*,
klee::ref<klee::ConstantExpr>
> const*, unsigned int, u
> nsigned int) /home/dsl11/dev/klee/klee/src/lib/Expr/Expr.cpp:506:25
>     #2 0x4e2ff3 in (anonymous
> namespace)::ExprTest_ConcatExtract_Test::TestBody()
> /home/dsl11/dev/klee/klee/src/unittests/Expr/ExprTest.cpp:32:24
>     #3 0x526410 in testing::Test::Run()
>
>
(/home/dsl11/dev/klee/klee/build_asan/unittests/Expr/Release+Asserts/ExprTests+0x526410)
> ```
>
> The source of the trouble is this static object.
>
> ```
> std::map<unsigned, std::vector<const Array *> *>
> Array::symbolicArraySingletonMap;
> ```
>
> Neither the ``std::vector<const Array*>`` pointers or the ``const
> Array`` pointers are being freed. Sure this code is bad (don't blame
> me, I didn't write it), but I want to skip over this leak to find more
> interesting issues.
>
> I can't seem to suppress it though. I've tried putting the
following
> in the ``blacklist.txt`` file
>
> * Explicitly naming the source file, like this
>
> src:/home/dsl11/dev/klee/klee/src/lib/Expr/Expr.cpp
>
> * Naming the function where the leak originates (demangled)
>
> fun:klee::Array::CreateArray
>
> * Naming the function where the leak originates (mangled)
>
>
>
fun:_ZN4klee5Array11CreateArrayERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmPKNS_3refINS_12ConstantExprEEESD_jj
>
> * Naming the global (demangled)
>
> global:klee::Array::symbolicArraySingletonMap
>
> * Naming the global (mangled)
>
> global:_ZN4klee5Array25symbolicArraySingletonMapE
>
>
> None of these succeed in suppressing the error. Does anyone have any
> idea what I'm doing wrong?
>
> Note I'm also using a fairly new build of libstdc++ which is using a
> new ABI [3]. I'm not sure if this would cause problems.
>
> [1] https://github.com/klee/klee
> [2]
>
http://clang.llvm.org/docs/AddressSanitizer.html#suppressing-errors-in-recompiled-code-blacklist
> [3] https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html
>
> Thanks,
> Dan.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>


-- 
Alexey Samsonov
vonosmas at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151215/0eb94a5b/attachment.html>

Hi Alexey,

On 15 December 2015 at 20:20, Alexey Samsonov <vonosmas at gmail.com>
wrote:
> Hi Dan,
>
> First of all, sorry for apparently misleading documentation. Currently you
> *can't* use blacklist
> to suppress memory leak reports: blacklist is applied at compile-time, and
> leak detection
> machinery brought up at run-time doesn't know about it.
>
> We should probably fix a documentation, as a first step.
Thanks. If it helps the reason I tried using the compile time
blacklist is that [1] gave the impression that
suppressing ASan reported issues in your "re-compiled code" should be
done using ``-fsanitize-blacklist=``.
I didn't realise that the LeakSanitizer

I think it would be great to fix Clang's documentation here to make it
clear how to suppress each type
of error report from ASan.

As a side note [2] gives the impression you should use mangled names
in a blacklist file as the comment says "use mangled names".
Given that it seems you can use mangled or demangled names it might be
better to say in the comment "You can use either mangled or demangled
names".
> What you can use is:
> 1) Runtime suppression: see
>
https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions
>
> You may pass environment variable LSAN_OPTIONS=suppressions=suppr.txt
> and try to specify smth. like
> leak:klee::Array::CreateArray
> or
> leak:lib/Expr/Expr.cpp
> in suppr.txt
>
> 2) You see the error report because LSan is invoked *after* all global
> destructors. So, you have
> std::map<unsigned, std::vector<const Array *> *>
> Array::symbolicArraySingletonMap;
> which contains pointers as values. When the map is destroyed, it's
cleared,
> and the allocated objects are
> no longer reachable. As a quick-fix, you can replace a global std::map with
> *a pointer* to global std::map
> which will also never be deleted. In that way all your objects will be
> reachable until the shutdown, and ASan/LSan will be silent.
>
> 3) You may schedule to run leak detection at an earlier point - e.g. as the
> last statement of main(): call __lsan_do_leak_check()
> from <sanitizer/lsan_interface.h>. This would run the leak detection
before
> the global destructors, i.e. before std::map is destroyed
>
> Hope that helps.
It helps a lot. These are great answers and probably worthy of being
expanded on slightly and put into the documentation :)

Thanks,
Dan.