llvm dev - Dec 2013 - [LLVMdev] [cfe-dev] lsan for LLVM bootstrap; leaks in TableGen

Hi,

We are trying to enable LeakSanitizer on our asan/msan llvm bootstrap bot
(http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/).
In clang itself there are two leaks
(http://llvm.org/bugs/show_bug.cgi?id=18318,
http://llvm-reviews.chandlerc.com/D2472)
and one lsan-hostile feature (http://llvm.org/bugs/show_bug.cgi?id=18320),
all of which are easy to fix.

And there are also lots of leaks in TableGen.
Would anyone be interested in fixing TableGen leaks?
I'd guess not since TableGen is not a user-facing utility, leaks there are
not too harmful.
If so, I'd like to disable lsan for TableGen so that
we can enable lsan on the bootstrap bot soon. Objections?

--kcc

Index: utils/TableGen/TableGen.cpp
==================================================================---
utils/TableGen/TableGen.cpp (revision 198007)
+++ utils/TableGen/TableGen.cpp (working copy)
@@ -180,3 +180,7 @@

   return TableGenMain(argv[0], &LLVMTableGenMain);
 }
+
+extern "C" {
+int __lsan_is_turned_off() { return 1; }
+}  // extern "C"
Index: tools/clang/utils/TableGen/TableGen.cpp
==================================================================---
tools/clang/utils/TableGen/TableGen.cpp     (revision 198007)
+++ tools/clang/utils/TableGen/TableGen.cpp     (working copy)
@@ -248,3 +248,7 @@

   return TableGenMain(argv[0], &ClangTableGenMain);
 }
+
+extern "C" {
+int __lsan_is_turned_off() { return 1; }
+}  // extern "C"
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131225/cacd24c3/attachment.html>

Its generally been by design that tblgen leaks. Might be nice to fix one
day but it's been that way for a while now so don't expect it to be
fixed
soon.

If thats the suppression mechanism of choice (I would've expected a build
flag option) for lsan, I'd say go for it.

(Maybe there's some existing build flag handling for valgrind so as not to
leak check tblgen, but that would be a runtime flag, not build time)

On Wednesday, December 25, 2013 7:08:27 AM, Kostya Serebryany
<kcc<kcc at google.com>
@ <kcc at google.com>google.com <kcc at google.com>> wrote:

Hi,

We are trying to enable LeakSanitizer on our asan/msan llvm bootstrap bot

(http://<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
lab.llvm.org<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
:8011/builders/sanitizer-x86_64-<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
linux
<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
-bootstrap/<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
).

In clang itself there are two leaks

(http://
<http://llvm.org/bugs/show_bug.cgi?id=18318>llvm.org<http://llvm.org/bugs/show_bug.cgi?id=18318>
/bugs/show_bug.cgi?id=18318 <http://llvm.org/bugs/show_bug.cgi?id=18318>,
http://llvm-reviews.chandlerc.com/D2472)

and one lsan-hostile feature (http://llvm.org/bugs/show_bug.cgi?id=18320),

all of which are easy to fix.

And there are also lots of leaks in TableGen.

Would anyone be interested in fixing TableGen leaks?

I'd guess not since TableGen is not a user-facing utility, leaks there are
not too harmful.

If so, I'd like to disable lsan for TableGen so that

we can enable lsan on the bootstrap bot soon. Objections?

--kcc

Index: utils/TableGen/TableGen.cpp

==================================================================
--- utils/TableGen/TableGen.cpp (revision 198007)

+++ utils/TableGen/TableGen.cpp (working copy)

@@ -180,3 +180,7 @@



   return TableGenMain(argv[0], &LLVMTableGenMain);

 }

+

+extern "C" {

+int __lsan_is_turned_off() { return 1; }

+}  // extern "C"

Index: tools/clang/utils/TableGen/TableGen.cpp

==================================================================
--- tools/clang/utils/TableGen/TableGen.cpp     (revision 198007)

+++ tools/clang/utils/TableGen/TableGen.cpp     (working copy)

@@ -248,3 +248,7 @@



   return TableGenMain(argv[0], &ClangTableGenMain);

 }

+

+extern "C" {

+int __lsan_is_turned_off() { return 1; }

+}  // extern "C"
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131225/f3ff9f22/attachment.html>

On Wed, Dec 25, 2013 at 9:38 PM, dblaikie at gmail.com <dblaikie at
gmail.com> wrote:
> Its generally been by design that tblgen leaks. Might be nice to fix one
day
> but it's been that way for a while now so don't expect it to be
fixed soon.
>
> If thats the suppression mechanism of choice (I would've expected a
build
> flag option) for lsan, I'd say go for it.
>
> (Maybe there's some existing build flag handling for valgrind so as not
to
> leak check tblgen, but that would be a runtime flag, not build time)
>
> On Wednesday, December 25, 2013 7:08:27 AM, Kostya Serebryany
> <kcc at google.com> wrote:
>
> Hi,
>
> We are trying to enable LeakSanitizer on our asan/msan llvm bootstrap bot
>
> (http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/).
>
> In clang itself there are two leaks
>
> (http://llvm.org/bugs/show_bug.cgi?id=18318,
> http://llvm-reviews.chandlerc.com/D2472)
>
> and one lsan-hostile feature (http://llvm.org/bugs/show_bug.cgi?id=18320),
>
> all of which are easy to fix.
>
> And there are also lots of leaks in TableGen.
I think the problem with TableGen is that we call leak checking too
late - after the global destructors,
not before them. IIRC pointers to objects "leaked" in TableGen are
stored in a set with static storage duration.
If we're able to call __lsan_do_leak_check() right after main(), we
will treat that objects as reachable.

I considered adding a call to __lsan_do_leak_check into
llvm_shutdown() function, but that looks wrong -
for instance, llvm_shutdown() may be called when a dynamic library is unloaded.
>
> Would anyone be interested in fixing TableGen leaks?
>
> I'd guess not since TableGen is not a user-facing utility, leaks there
are
> not too harmful.
>
> If so, I'd like to disable lsan for TableGen so that
>
> we can enable lsan on the bootstrap bot soon. Objections?
>
> --kcc
>
> Index: utils/TableGen/TableGen.cpp
>
> ==================================================================>
> --- utils/TableGen/TableGen.cpp (revision 198007)
>
> +++ utils/TableGen/TableGen.cpp (working copy)
>
> @@ -180,3 +180,7 @@
>
>
>
>    return TableGenMain(argv[0], &LLVMTableGenMain);
>
>  }
>
> +
>
> +extern "C" {
>
> +int __lsan_is_turned_off() { return 1; }
>
> +}  // extern "C"
>
> Index: tools/clang/utils/TableGen/TableGen.cpp
>
> ==================================================================>
> --- tools/clang/utils/TableGen/TableGen.cpp     (revision 198007)
>
> +++ tools/clang/utils/TableGen/TableGen.cpp     (working copy)
>
> @@ -248,3 +248,7 @@
>
>
>
>    return TableGenMain(argv[0], &ClangTableGenMain);
>
>  }
>
> +
>
> +extern "C" {
>
> +int __lsan_is_turned_off() { return 1; }
>
> +}  // extern "C"
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>


-- 
Alexey Samsonov, MSK

On Wed, Dec 25, 2013 at 7:38 PM, dblaikie at gmail.com <dblaikie at
gmail.com>wrote:
> Its generally been by design that tblgen leaks. Might be nice to fix one
> day but it's been that way for a while now so don't expect it to be
fixed
> soon.
>
> If thats the suppression mechanism of choice
It is.
> (I would've expected a build flag option)

That would not be trivial.
The link-time flag is -fsanitize=address, and we do want to pass it because
we do want to check TableGen for addressability bugs
(and because if we don't pass it at link time we can not use it at compile
time, which will cause a huge cmake/make challenge).
Inventing a link time flag that will keep asan but remove lsan is not worth
it.
There is the run-time flag (ASAN_OPTIONS=detect_leaks=0|1) which turns lsan
on/off,
 but again passing it selectively to TableGen is harder than my simple
patch.

> for lsan, I'd say go for it.
>
Ok, unless someone objects.

--kcc

>
> (Maybe there's some existing build flag handling for valgrind so as not
to
> leak check tblgen, but that would be a runtime flag, not build time)
>
> On Wednesday, December 25, 2013 7:08:27 AM, Kostya Serebryany
<kcc<kcc at google.com>
> @ <kcc at google.com>google.com <kcc at google.com>> wrote:
>
> Hi,
>
> We are trying to enable LeakSanitizer on our asan/msan llvm bootstrap bot
>
>
(http://<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
>
lab.llvm.org<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
>
:8011/builders/sanitizer-x86_64-<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
>
linux<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
>
-bootstrap/<http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/>
> ).
>
> In clang itself there are two leaks
>
> (http://
<http://llvm.org/bugs/show_bug.cgi?id=18318>llvm.org<http://llvm.org/bugs/show_bug.cgi?id=18318>
> /bugs/show_bug.cgi?id=18318
<http://llvm.org/bugs/show_bug.cgi?id=18318>,
> http://llvm-reviews.chandlerc.com/D2472)
>
> and one lsan-hostile feature (http://llvm.org/bugs/show_bug.cgi?id=18320),
>
> all of which are easy to fix.
>
> And there are also lots of leaks in TableGen.
>
> Would anyone be interested in fixing TableGen leaks?
>
> I'd guess not since TableGen is not a user-facing utility, leaks there
are
> not too harmful.
>
> If so, I'd like to disable lsan for TableGen so that
>
> we can enable lsan on the bootstrap bot soon. Objections?
>
> --kcc
>
> Index: utils/TableGen/TableGen.cpp
>
> ==================================================================>
> --- utils/TableGen/TableGen.cpp (revision 198007)
>
> +++ utils/TableGen/TableGen.cpp (working copy)
>
> @@ -180,3 +180,7 @@
>
>
>
>    return TableGenMain(argv[0], &LLVMTableGenMain);
>
>  }
>
> +
>
> +extern "C" {
>
> +int __lsan_is_turned_off() { return 1; }
>
> +}  // extern "C"
>
> Index: tools/clang/utils/TableGen/TableGen.cpp
>
> ==================================================================>
> --- tools/clang/utils/TableGen/TableGen.cpp     (revision 198007)
>
> +++ tools/clang/utils/TableGen/TableGen.cpp     (working copy)
>
> @@ -248,3 +248,7 @@
>
>
>
>    return TableGenMain(argv[0], &ClangTableGenMain);
>
>  }
>
> +
>
> +extern "C" {
>
> +int __lsan_is_turned_off() { return 1; }
>
> +}  // extern "C"
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131225/a8193d4a/attachment.html>