llvm dev - Feb 2015 - [LLVMdev] RFC: Dropping support for building sanitizers with autotools

> On Feb 17, 2015, at 4:00 PM, Alexey Samsonov <vonosmas at gmail.com>
wrote:
> 
> 
> On Tue, Feb 17, 2015 at 3:37 PM, Anna Zaks <ganna at apple.com
<mailto:ganna at apple.com>> wrote:
> Unfortunately, we do rely on the Autotools to build for darwin. 
> 
> The lack of proper testing is very unfortunate. In the future, we will
either drop the dependency on Makefile/autoconf build or add the support for
sanitizer testing.
> 
> Sorry, for some reason I had the impression that you're also more
interested in CMake buiild :-/. Only now I noticed that recent Kuba's
patches about -isysroot fix both makefile and CMake
> build system (and he's probably not very happy about it).
> 
> I believe the effort required to port sanitizer unit tests to autotools is
not worth it, and it's better to spend that time migrating the release tools
to CMake. Keeping two build systems "working"
> was and is a manageable, but unpleasant burden. Providing a good test
coverage for two runtime versions is hard, and keeping two runtime versions
"equivalent" (built with the same compiler flags)
> is close to impossible.
I agree. The solution where we only rely on cmake build would be the best
option.
>  
> I understand that removing autotools support right now breaks your
integration. What if we just delete autotools support from all Linux builds?
> 
Yes. That is fine by me.
> Speaking of autotools support, I should lay out the reason why I raised the
question of removing it last week. We plan to make intrusive changes to ASan and
UBSan library layout:
> 1) embed UBSan runtime into ASan to solve various initialization and
integration problems of using ASan+UBSan (fixing
http://llvm.org/bugs/show_bug.cgi?id=21112
<http://llvm.org/bugs/show_bug.cgi?id=21112>)
> 2) make UBSan runtime standalone, and forbid to use it with another
sanitizers except for ASan.
> Unfortunately, it means changing a bunch of build rules in *both* CMake and
makefile build.
> 
> Ideally, I would be happy to transfer ownership of current autotools build
on Darwin to you, and ask for help in implementing/testing necessary changes on
Mac OS X side :)
> At the moment we don't even have a live buildbot running
autotools-built ASan on Mac, so you're in much better shape to catch and
notice regressions.
> 
We do have public buildbots set up for OS X:

Here is one that is testing a makefile build. There are probably others in
different configurations.
http://lab.llvm.org:8080/green/view/Clang/job/clang-stage1-configure-RA_build/
<http://lab.llvm.org:8080/green/view/Clang/job/clang-stage1-configure-RA_build/>

We also have a buildbot running cmake tests on OS X:
http://lab.llvm.org:8080/green/view/All/job/clang-stage1-cmake-RA_check/
<http://lab.llvm.org:8080/green/view/All/job/clang-stage1-cmake-RA_check/>
> 
> Anna.
> 
>> On Feb 17, 2015, at 10:54 AM, Kostya Serebryany <kcc at google.com
<mailto:kcc at google.com>> wrote:
>> 
>> +1 (in case you still need mine)
>> 
>> On Thu, Feb 12, 2015 at 6:31 PM, Alexey Samsonov <vonosmas at
gmail.com <mailto:vonosmas at gmail.com>> wrote:
>> Hi everyone,
>> 
>> TL;DR I plan to do subj unless anyone objects.
>> 
>> Autotools support for building sanitizers was never complete, is a
proper subset of CMake capabilities, and the gap between the features of CMake
build and autotools is quickly increasing:
>> 1) OS: autotools support Linux, Mac OS X and Android; CMake also
supports FreeBSD and Windows.
>> 2) Architectures: autotools support i386 and x86_64, CMake also
supports MIPS, ARM/AArch64, PowerPC).
>> 3) Library variants: autotools don't support building shared ASan
runtime on Linux.
>> 4 [!!!]) Autotools build doesn't have *any* support for building
and running tests.
>> 5) The sets of compiler flags we use to build runtimes are different in
two builds, and are hard to kept in sync.
>> 
>> Because of (4), sanitizer runtimes built with autotools are severely
undertested, and maintaining two different build systems is a burden I would
like to get rid of. Now seems to be a good time for that: we've cut off 3.6
release branch, and Hans assures me that 3.7 release process will use CMake.
>> 
>> Are there existing users of configure/make that want sanitizers to be
available in their build? Are there other arguments for keeping the autotools
build "working" I'm missing?
>> 
>> -- 
>> Alexey Samsonov
>> vonosmas at gmail.com <mailto:vonosmas at gmail.com>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>        
http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/>
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
<http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>
> 
> 
> 
> -- 
> Alexey Samsonov
> vonosmas at gmail.com <mailto:vonosmas at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150217/cc4e8694/attachment.html>

On Tue, Feb 17, 2015 at 6:23 PM, Anna Zaks <ganna at apple.com> wrote:
>
> We do have public buildbots set up for OS X:
>
> Here is one that is testing a makefile build. There are probably others in
> different configurations.
>
>
http://lab.llvm.org:8080/green/view/Clang/job/clang-stage1-configure-RA_build/
>
I may be missing something obvious (the jenkins UI is very unfamiliar to
me) but I don't see this build bot failing at any time because of this
change... Is it actually building the ASan runtimes?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150217/925e538b/attachment.html>

> On Feb 17, 2015, at 6:33 PM, Chandler Carruth <chandlerc at
google.com> wrote:
> 
> 
> On Tue, Feb 17, 2015 at 6:23 PM, Anna Zaks <ganna at apple.com
<mailto:ganna at apple.com>> wrote:
> 
> We do have public buildbots set up for OS X:
> 
> Here is one that is testing a makefile build. There are probably others in
different configurations.
>
http://lab.llvm.org:8080/green/view/Clang/job/clang-stage1-configure-RA_build/
<http://lab.llvm.org:8080/green/view/Clang/job/clang-stage1-configure-RA_build/>
> 
> I may be missing something obvious (the jenkins UI is very unfamiliar to
me) but I don't see this build bot failing at any time because of this
change... Is it actually building the ASan runtimes?
The change has been reverted shortly after the initial commit to allow for more
discussions. This is why the bot is green. You can see that the asan library is
being built if you choose "Console Output" menu and click "full
log", search for "libclang_rt.asan_osx_dynamic.dylib".

I don't think there is a specific reason why we need the sanitizers to be
built with autotools. I agree that the current setup is broken in several ways
and needs attention. Thank you for the suggestion to run cmake from an
autoconf/Makefile setting; we'll investigate if that could be an acceptable
solution.

Anna.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150217/04cfbffc/attachment.html>

On Tue, Feb 17, 2015 at 6:23 PM, Anna Zaks <ganna at apple.com> wrote:
>
> On Feb 17, 2015, at 4:00 PM, Alexey Samsonov <vonosmas at gmail.com>
wrote:
>
>
> On Tue, Feb 17, 2015 at 3:37 PM, Anna Zaks <ganna at apple.com>
wrote:
>
>> Unfortunately, we do rely on the Autotools to build for darwin.
>>
>
>> The lack of proper testing is very unfortunate. In the future, we will
>> either drop the dependency on Makefile/autoconf build or add the
support
>> for sanitizer testing.
>>
>
> Sorry, for some reason I had the impression that you're also more
> interested in CMake buiild :-/. Only now I noticed that recent Kuba's
> patches about -isysroot fix both makefile and CMake
> build system (and he's probably not very happy about it).
>
> I believe the effort required to port sanitizer unit tests to autotools is
> not worth it, and it's better to spend that time migrating the release
> tools to CMake. Keeping two build systems "working"
> was and is a manageable, but unpleasant burden. Providing a good test
> coverage for two runtime versions is hard, and keeping two runtime versions
> "equivalent" (built with the same compiler flags)
> is close to impossible.
>
>
> I agree. The solution where we only rely on cmake build would be the best
> option.
>
>
> I understand that removing autotools support right now breaks your
> integration. What if we just delete autotools support from all Linux
builds?
>
>
> Yes. That is fine by me.
>
OK, I've removed the support for sanitizers in makefile/autoconf builds on
Linux in r229754-r229756.

>
> Speaking of autotools support, I should lay out the reason why I raised
> the question of removing it last week. We plan to make intrusive changes to
> ASan and UBSan library layout:
> 1) embed UBSan runtime into ASan to solve various initialization and
> integration problems of using ASan+UBSan (fixing
> http://llvm.org/bugs/show_bug.cgi?id=21112)
> 2) make UBSan runtime standalone, and forbid to use it with another
> sanitizers except for ASan.
> Unfortunately, it means changing a bunch of build rules in *both* CMake
> and makefile build.
>
> Ideally, I would be happy to transfer ownership of current autotools build
> on Darwin to you, and ask for help in implementing/testing necessary
> changes on Mac OS X side :)
> At the moment we don't even have a live buildbot running
autotools-built
> ASan on Mac, so you're in much better shape to catch and notice
regressions.
>
>
> We do have public buildbots set up for OS X:
>
> Here is one that is testing a makefile build. There are probably others in
> different configurations.
>
>
http://lab.llvm.org:8080/green/view/Clang/job/clang-stage1-configure-RA_build/
>
> We also have a buildbot running cmake tests on OS X:
> http://lab.llvm.org:8080/green/view/All/job/clang-stage1-cmake-RA_check/
>
>
>> Anna.
>>
>> On Feb 17, 2015, at 10:54 AM, Kostya Serebryany <kcc at
google.com> wrote:
>>
>> +1 (in case you still need mine)
>>
>> On Thu, Feb 12, 2015 at 6:31 PM, Alexey Samsonov <vonosmas at
gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> TL;DR I plan to do subj unless anyone objects.
>>>
>>> Autotools support for building sanitizers was never complete, is a
>>> proper subset of CMake capabilities, and the gap between the
features of
>>> CMake build and autotools is quickly increasing:
>>> 1) OS: autotools support Linux, Mac OS X and Android; CMake also
>>> supports FreeBSD and Windows.
>>> 2) Architectures: autotools support i386 and x86_64, CMake also
supports
>>> MIPS, ARM/AArch64, PowerPC).
>>> 3) Library variants: autotools don't support building shared
ASan
>>> runtime on Linux.
>>> 4 [!!!]) Autotools build doesn't have *any* support for
building and
>>> running tests.
>>> 5) The sets of compiler flags we use to build runtimes are
different in
>>> two builds, and are hard to kept in sync.
>>>
>>> Because of (4), sanitizer runtimes built with autotools are
severely
>>> undertested, and maintaining two different build systems is a
burden I
>>> would like to get rid of. Now seems to be a good time for that:
we've cut
>>> off 3.6 release branch, and Hans assures me that 3.7 release
process will
>>> use CMake.
>>>
>>> Are there existing users of configure/make that want sanitizers to
be
>>> available in their build? Are there other arguments for keeping the
>>> autotools build "working" I'm missing?
>>>
>>> --
>>> Alexey Samsonov
>>> vonosmas at gmail.com
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>>
>
>
> --
> Alexey Samsonov
> vonosmas at gmail.com
>
>
>

-- 
Alexey Samsonov
vonosmas at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150218/3db0759a/attachment.html>

On 18/02/2015 23:29, Alexey Samsonov wrote:
> 
> On Tue, Feb 17, 2015 at 6:23 PM, Anna Zaks <ganna at apple.com
<mailto:ganna at apple.com>> wrote:
> 
> 
>>     On Feb 17, 2015, at 4:00 PM, Alexey Samsonov <vonosmas at
gmail.com <mailto:vonosmas at gmail.com>> wrote:
>>
>>
>>     On Tue, Feb 17, 2015 at 3:37 PM, Anna Zaks <ganna at apple.com
<mailto:ganna at apple.com>> wrote:
>>
>>         Unfortunately, we do rely on the Autotools to build for darwin.
>>
>>
>>         The lack of proper testing is very unfortunate. In the future,
we will either drop the dependency on Makefile/autoconf build or add the support
for sanitizer testing.
>>
>>
>>     Sorry, for some reason I had the impression that you're also
more interested in CMake buiild :-/. Only now I noticed that recent Kuba's
patches about -isysroot fix both makefile and CMake
>>     build system (and he's probably not very happy about it).
>>
>>     I believe the effort required to port sanitizer unit tests to
autotools is not worth it, and it's better to spend that time migrating the
release tools to CMake. Keeping two build systems "working"
>>     was and is a manageable, but unpleasant burden. Providing a good
test coverage for two runtime versions is hard, and keeping two runtime versions
"equivalent" (built with the same compiler flags)
>>     is close to impossible.
> 
>     I agree. The solution where we only rely on cmake build would be the
best option.
> 
>>      
>>     I understand that removing autotools support right now breaks your
integration. What if we just delete autotools support from all Linux builds?
>>
> 
>     Yes. That is fine by me.
> 
> 
> OK, I've removed the support for sanitizers in makefile/autoconf builds
on Linux in r229754-r229756.
It is possible to build LLVM & Clang with autotools and compiler-rt with
cmake?

Currently, dropping the autotools support is breaking the build of compiler-rt
for llvm.org/apt/ and
the Debian & Ubuntu packaging.

I agree that the autotools support in compiler-rt was limited but it was doing
the job correctly...

Could you consider reverting this changes?

Thanks,
Sylvestre
PS: I don't mind switching to cmake but here are still a few blocking bugs:
http://llvm.org/bugs/show_bug.cgi?id=15732