llvm dev - Feb 2017 - [RFC] Proposing libcxx-config.py: a configuration tool for building and using libc++

Hi All,

Configuring a compiler to target or build libc++ is tricky. It often
requires suppressing and rebuilding large parts of the CC1 compiler
invocation. Doing this manually can be prohibitive, even for experienced
users. Even doing it programmatically is hard and requires a lot of code
because of the sheer number of configurations libc++ support.

These problems affects both libc++ and it's users. For libc++ CMake's
statefulness makes it hard to generate the two different sets of flags
needed to initially build the library and then to target it afterwards.

Therefore I'm proposing a utility called `libcxx-config.py` which, similar
to llvm-config, when queried will output the flags required to compile and
link libc++. The tool will have two main purposes:

(1) Generating the compile and link flags required to *use the just-built
libc++ library*.
(2) Generating the compile and link flags required to *build the libc++
dylib.*

The goals for `libcxx-config.py` are:

(1) Reduce duplicate configuration logic between the CMake and test-suite.
(2) Allow non-CMake users to correctly generate the flags required to build
libc++. (Ex buildit)
(3) Make it easier for projects, including libc++, to correctly configure
to target the just-built libc++.
(4) Allow the creation of a `test-libc++` tool which can compile and link
test programs against libc++ (Often needed when debugging test suite
failures).

Would anybody else have a need for this? Are there use cases anybody would
like to see supported?

Some example usages I imagine for `libcxx-config.py` are:

  # Get the compile flags needed to build libc++.dylib
  $ libcxx-config.py --building-library --compile-flags --compiler=clang++
--cxx-abi-library=libcxxabi

  # output: -nostdinc++ -I <libcxx-headers> -D_LIBCPP_BUILDING_LIBRARY
-DLIBCXX_BUILDING_LIBCXXABI -std=c++11 -fvisibility-inlines-hidden

  # Get the link flags needed to use libc++ with the specified compiler
  $ libcxx-config.py --link-flags --compiler=g++

  # Output: -nodefaultlibs -L <path> -Wl,-rpath <path>
-lc++experimental
-lc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc

Any feedback is appreciated.

/Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170209/436d4468/attachment.html>

> On Feb 9, 2017, at 4:18 PM, Eric Fiselier via cfe-dev <cfe-dev at
lists.llvm.org> wrote:
> 
> Hi All,
> 
> Configuring a compiler to target or build libc++ is tricky. It often
requires suppressing and rebuilding large parts of the CC1 compiler invocation.
Doing this manually can be prohibitive, even for experienced users. Even doing
it programmatically is hard and requires a lot of code because of the sheer
number of configurations libc++ support.
> 
> These problems affects both libc++ and it's users. For libc++
CMake's statefulness makes it hard to generate the two different sets of
flags needed to initially build the library and then to target it afterwards.
Can you elaborate on this issue? Without looking deeply at the problem, it seems
difficult to me to figure exactly what the problem is.

> 
> Therefore I'm proposing a utility called `libcxx-config.py` which,
similar to llvm-config, when queried will output the flags required to compile
and link libc++. The tool will have two main purposes:
> 
> (1) Generating the compile and link flags required to use the just-built
libc++ library.
> (2) Generating the compile and link flags required to build the libc++
dylib.
> 
> The goals for `libcxx-config.py` are:
> 
> (1) Reduce duplicate configuration logic between the CMake and test-suite.
> (2) Allow non-CMake users to correctly generate the flags required to build
libc++. (Ex buildit)
> (3) Make it easier for projects, including libc++, to correctly configure
to target the just-built libc++.
> (4) Allow the creation of a `test-libc++` tool which can compile and link
test programs against libc++ (Often needed when debugging test suite failures).
> 
> Would anybody else have a need for this? Are there use cases anybody would
like to see supported?
> 
> Some example usages I imagine for `libcxx-config.py` are:
> 
>   # Get the compile flags needed to build libc++.dylib
>   $ libcxx-config.py --building-library --compile-flags --compiler=clang++
--cxx-abi-library=libcxxabi
>   
>   # output: -nostdinc++ -I <libcxx-headers>
-D_LIBCPP_BUILDING_LIBRARY -DLIBCXX_BUILDING_LIBCXXABI -std=c++11
-fvisibility-inlines-hidden
I’m not sure why it is not the role of CMake?
>   # Get the link flags needed to use libc++ with the specified compiler
>   $ libcxx-config.py --link-flags --compiler=g++
>   
>   # Output: -nodefaultlibs -L <path> -Wl,-rpath <path>
-lc++experimental -lc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc
Isn’t it the role of the clang driver when given -stdlib=libc++ ?

— 
Mehdi

> 
> Any feedback is appreciated.
> 
> /Eric
> 
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170211/c7c098bb/attachment.html>

On 2/9/17 5:18 PM, Eric Fiselier via llvm-dev wrote:
> Hi All,
>
> Configuring a compiler to target or build libc++ is tricky. It often
> requires suppressing and rebuilding large parts of the CC1 compiler
> invocation. Doing this manually can be prohibitive, even for experienced
> users. Even doing it programmatically is hard and requires a lot of code
> because of the sheer number of configurations libc++ support.
Can you elaborate on what the problems are here?

Is this something we could fix by adding flags to clang to query for the 
appropriate flags? (i.e. like the kind of support given by 
-print-multi-directory/-print-multi-lib, but tailored for the problems 
related to building/using a c++ STL)


Jon
>
> These problems affects both libc++ and it's users. For libc++
CMake's
> statefulness makes it hard to generate the two different sets of flags
> needed to initially build the library and then to target it afterwards.
>
> Therefore I'm proposing a utility called `libcxx-config.py` which,
> similar to llvm-config, when queried will output the flags required to
> compile and link libc++. The tool will have two main purposes:
>
> (1) Generating the compile and link flags required to *use the
> just-built libc++ library*.
> (2) Generating the compile and link flags required to *build the libc++
> dylib.*
> *
> *
> The goals for `libcxx-config.py` are:
>
> (1) Reduce duplicate configuration logic between the CMake and test-suite.
> (2) Allow non-CMake users to correctly generate the flags required to
> build libc++. (Ex buildit)
> (3) Make it easier for projects, including libc++, to correctly
> configure to target the just-built libc++.
> (4) Allow the creation of a `test-libc++` tool which can compile and
> link test programs against libc++ (Often needed when debugging test
> suite failures).
>
> Would anybody else have a need for this? Are there use cases anybody
> would like to see supported?
>
> Some example usages I imagine for `libcxx-config.py` are:
>
>   # Get the compile flags needed to build libc++.dylib
>   $ libcxx-config.py --building-library --compile-flags
> --compiler=clang++ --cxx-abi-library=libcxxabi
>
>   # output: -nostdinc++ -I <libcxx-headers>
-D_LIBCPP_BUILDING_LIBRARY
> -DLIBCXX_BUILDING_LIBCXXABI -std=c++11 -fvisibility-inlines-hidden
>
>   # Get the link flags needed to use libc++ with the specified compiler
>   $ libcxx-config.py --link-flags --compiler=g++
>
>   # Output: -nodefaultlibs -L <path> -Wl,-rpath <path>
-lc++experimental
> -lc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc
>
> Any feedback is appreciated.
>
> /Eric
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded

On 2017-02-09 18:18, Eric Fiselier via cfe-dev wrote:
> Hi All,
> 
> Configuring a compiler to target or build libc++ is tricky. It often
> requires suppressing and rebuilding large parts of the CC1 compiler
> invocation. Doing this manually can be prohibitive, even for
> experienced users. Even doing it programmatically is hard and requires
> a lot of code because of the sheer number of configurations libc++
> support.
Agreed: this is challenging to get right.
> compile and link libc++. The tool will have two main purposes:
> 
> (1) Generating the compile and link flags required to USE THE
> JUST-BUILT LIBC++ LIBRARY.
> (2) Generating the compile and link flags required to BUILD THE LIBC++
> DYLIB.
I've witnessed these challenges while building and testing libcxx for 
Hexagon.  There's been at least one occasion where I accidentally 
configured it to build and test against the baseline/bootstrap 
installation's libc++ and not the just-built one.
> The goals for `libcxx-config.py` are:
> 
> (1) Reduce duplicate configuration logic between the CMake and
> test-suite.
 From what I've seen, I thought there's a decent amount of cascade from 
CMake vars->lit.site.cfg vars, leveraged by the relevant targets' 
configs.  But IIRC I saw some flags in utils/libcxx/test/config.py that 
were the same as CMake ones.
> (2) Allow non-CMake users to correctly generate the flags required to
> build libc++. (Ex buildit)
> (3) Make it easier for projects, including libc++, to correctly
> configure to target the just-built libc++.
> (4) Allow the creation of a `test-libc++` tool which can compile and
> link test programs against libc++ (Often needed when debugging test
> suite failures).
> 
> Would anybody else have a need for this? Are there use cases anybody
> would like to see supported?
I don't see a need for #2 -- wasn't autoconf deprecated for the rest of 
LLVM at 3.8?  #3 is less of a concern for us because our local test 
infrastructure builds/tests our libc++ specifically from a compiler 
suite lacking a target libc++.  Someone clever must have figured that 
failure mode out before I started on this project.

I've hand-rolled a solution for #4 but having a canonical way to do this 
seems like a good idea.  I'd be worried that the critical element to the 
test failure is among the flags in the CMake config.  That said, it 
would be handy when working with other teams less familiar with 
building/testing LLVM-style projects.  It turns out that the 
libc++/c++abi test suites are great at evoking bugs at various layers 
throughout the system.

-Brian