Chandler Carruth
2011-Nov-23 00:10 UTC
[LLVMdev] RFC: How should Clang/LLVM runtime libraries be installed and found during link steps?
It has come up when reviewing Kostya's patch to add the necessary support fort linking in the Address Sanitizer runtime library that we need a proper scheme and plan for deploying runtime libraries along with Clang. I've CC'ed llvmdev on this for compiler-rt developers' input. The key issues I see when locating runtime libraries are the following: - These libraries should be shipped with Clang, not installed separately on the system. - They are likely to be somewhat tied to specific versions of Clang/LLVM. - To support cross-compiling, we should be able to install multiple copies of these libraries in a single Clang installation, and use the appropriate one when targeting a particular platform. - The above cross-compiling concern extends to ABI compatibility -- many of the ABIs are already fixed in this space. - These are different from builtin header files which can use the preprocessor to internally differentiate their contents based on the target platform. My proposed solution: - Base the path on the shared "resource directory" concept which already exists in Clang. - Builtin header files are at <ResourceDir>/include already. - Append a "base" triple directory name. - Append a "lib" directory name for runtime libraries - Place the runtime libraries as "libcompiler_rt_<sublib>.a" for each sub-library component "<sublib>". Example for "x/bin/clang" installation: x/bin/../lib/clang/3.1/x86_64-linux-gnu/lib/libcompiler_rt_asan.a What is a "base" triple? It is the simplest form we can reduce a triple to while guaranteeing compatibility. The concept is best explained by an example. All of the following triples reduce to the base triple of "x86_64-linux-gnu": x86_64-linux-gnu x86_64-pc-linux-gnu x86_64-unknown-linux-gnu x86_64-redhat-linux x86_64-redhat-linux6E ... A different ABI could be expressed much like ARM's is with the last element: "*-gnueabi". This would *not* reduce to the triple ending in '-gnu'. Due to the adhoc and poorly spec'ed nature of existing triples, this will largely be a fixed mapping much like already exists in the Clang driver, but consolidated into LLVM's core triple logic. Open Questions: How do we handle 'i686' vs 'i386'? Is it useful to have separate installed libraries for i386 and i686 in order to get better performance for the latter *and* maintain compatibility for the former? We could collapse to either i386 or i686, and define either to mean whatever we want (for example, Debian uses i386-linux-gnu for its "base" triple in multiarch, and does *not* support i386 processors). This scheme closely mirrors GCC's existing scheme with one exception: GCC puts the triple first, and the version number second. I don't think this matters greatly either way, but currently Clang puts its version number first. I think this reflects the inherent design of Clang to be cross-compiling by default, but it would be good to consider (even if we reject) matching GCC's behavior here. Should runtime libraries be installed as archives? .o files? .so files? (gasp) bitcode? Some mixture of these? What mixture, and how do we decide? I lean toward .o files as bitcode where the linker supports it, normal .o files where it supports those, and .a files only as a fallback. Not very confident of these preferences though. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20111122/865fb8b1/attachment.html>
Daniel Dunbar
2011-Nov-28 20:53 UTC
[LLVMdev] RFC: How should Clang/LLVM runtime libraries be installed and found during link steps?
Hey Chandler, We already have a certain precedent for how we do this on Darwin. The current library set is: -- $ find lib/clang/3.1/lib lib/clang/3.1/lib lib/clang/3.1/lib/darwin lib/clang/3.1/lib/darwin/libclang_rt.10.4.a lib/clang/3.1/lib/darwin/libclang_rt.cc_kext.a lib/clang/3.1/lib/darwin/libclang_rt.eprintf.a lib/clang/3.1/lib/darwin/libclang_rt.ios.a lib/clang/3.1/lib/darwin/libclang_rt.osx.a lib/clang/3.1/lib/darwin/libclang_rt.profile_ios.a lib/clang/3.1/lib/darwin/libclang_rt.profile_osx.a -- It's entirely custom based on what the driver needs, and I find this to be very workable. In general, I view this as very much an "implementation detail" and see no need to over engineer here w.r.t. defining some formal schema up front. We can easily change it over time as we figure out what works. Unlike GCC we *never* want users to use these library names directly, and they are rev-locked to the compiler, which is why we can get away with treating the exact layout as an implementation detail. My preference would be that runtime libraries for broad platform classes (basically, darwin, linux, win32) fall under top level directories in the lib resource dir. Beyond that, any further subdivision should match whatever is easiest for the platform maintainers and driver implementation. I think we should just wait and review the code as it comes in. Note that I'm going to be checking in some code soonish to start setting these things up for Linux... it will be minimal and simple, and we can refine it over time as need be. - Daniel On Tue, Nov 22, 2011 at 4:10 PM, Chandler Carruth <chandlerc at google.com> wrote:> It has come up when reviewing Kostya's patch to add the necessary support > fort linking in the Address Sanitizer runtime library that we need a proper > scheme and plan for deploying runtime libraries along with Clang. > I've CC'ed llvmdev on this for compiler-rt developers' input. > The key issues I see when locating runtime libraries are the following: > - These libraries should be shipped with Clang, not installed separately on > the system. > - They are likely to be somewhat tied to specific versions of Clang/LLVM. > - To support cross-compiling, we should be able to install multiple copies > of these libraries in a single Clang installation, and use the appropriate > one when targeting a particular platform. > - The above cross-compiling concern extends to ABI compatibility -- many of > the ABIs are already fixed in this space. > - These are different from builtin header files which can use the > preprocessor to internally differentiate their contents based on the target > platform. > My proposed solution: > - Base the path on the shared "resource directory" concept which already > exists in Clang. > - Builtin header files are at <ResourceDir>/include already. > - Append a "base" triple directory name. > - Append a "lib" directory name for runtime libraries > - Place the runtime libraries as "libcompiler_rt_<sublib>.a" for each > sub-library component "<sublib>". > Example for "x/bin/clang" installation: > x/bin/../lib/clang/3.1/x86_64-linux-gnu/lib/libcompiler_rt_asan.a > What is a "base" triple? It is the simplest form we can reduce a triple to > while guaranteeing compatibility. The concept is best explained by an > example. All of the following triples reduce to the base triple of > "x86_64-linux-gnu": > x86_64-linux-gnu > x86_64-pc-linux-gnu > x86_64-unknown-linux-gnu > x86_64-redhat-linux > x86_64-redhat-linux6E > ... > A different ABI could be expressed much like ARM's is with the last element: > "*-gnueabi". This would *not* reduce to the triple ending in '-gnu'. Due to > the adhoc and poorly spec'ed nature of existing triples, this will largely > be a fixed mapping much like already exists in the Clang driver, but > consolidated into LLVM's core triple logic. > > Open Questions: > How do we handle 'i686' vs 'i386'? Is it useful to have separate installed > libraries for i386 and i686 in order to get better performance for the > latter *and* maintain compatibility for the former? We could collapse to > either i386 or i686, and define either to mean whatever we want (for > example, Debian uses i386-linux-gnu for its "base" triple in multiarch, and > does *not* support i386 processors). > This scheme closely mirrors GCC's existing scheme with one exception: GCC > puts the triple first, and the version number second. I don't think this > matters greatly either way, but currently Clang puts its version number > first. I think this reflects the inherent design of Clang to be > cross-compiling by default, but it would be good to consider (even if we > reject) matching GCC's behavior here. > Should runtime libraries be installed as archives? .o files? .so files? > (gasp) bitcode? Some mixture of these? What mixture, and how do we decide? I > lean toward .o files as bitcode where the linker supports it, normal .o > files where it supports those, and .a files only as a fallback. Not very > confident of these preferences though.
Kostya Serebryany
2011-Nov-29 17:58 UTC
[LLVMdev] RFC: How should Clang/LLVM runtime libraries be installed and found during link steps?
So, is there an agreement now? In particular, is it fine to have the asan run-time for linux x86/x86_64 at lib/clang/linux/TC.getArchName())/libclang_rt.asan.a ? Thanks, --kcc On Mon, Nov 28, 2011 at 12:53 PM, Daniel Dunbar <daniel at zuster.org> wrote:> Hey Chandler, > > We already have a certain precedent for how we do this on Darwin. The > current library set is: > -- > $ find lib/clang/3.1/lib > lib/clang/3.1/lib > lib/clang/3.1/lib/darwin > lib/clang/3.1/lib/darwin/libclang_rt.10.4.a > lib/clang/3.1/lib/darwin/libclang_rt.cc_kext.a > lib/clang/3.1/lib/darwin/libclang_rt.eprintf.a > lib/clang/3.1/lib/darwin/libclang_rt.ios.a > lib/clang/3.1/lib/darwin/libclang_rt.osx.a > lib/clang/3.1/lib/darwin/libclang_rt.profile_ios.a > lib/clang/3.1/lib/darwin/libclang_rt.profile_osx.a > -- > It's entirely custom based on what the driver needs, and I find this > to be very workable. > > In general, I view this as very much an "implementation detail" and > see no need to over engineer here w.r.t. defining some formal schema > up front. We can easily change it over time as we figure out what > works. Unlike GCC we *never* want users to use these library names > directly, and they are rev-locked to the compiler, which is why we can > get away with treating the exact layout as an implementation detail. > > My preference would be that runtime libraries for broad platform > classes (basically, darwin, linux, win32) fall under top level > directories in the lib resource dir. Beyond that, any further > subdivision should match whatever is easiest for the platform > maintainers and driver implementation. I think we should just wait and > review the code as it comes in. > > Note that I'm going to be checking in some code soonish to start > setting these things up for Linux... it will be minimal and simple, > and we can refine it over time as need be. > > - Daniel > > On Tue, Nov 22, 2011 at 4:10 PM, Chandler Carruth <chandlerc at google.com> > wrote: > > It has come up when reviewing Kostya's patch to add the necessary support > > fort linking in the Address Sanitizer runtime library that we need a > proper > > scheme and plan for deploying runtime libraries along with Clang. > > I've CC'ed llvmdev on this for compiler-rt developers' input. > > The key issues I see when locating runtime libraries are the following: > > - These libraries should be shipped with Clang, not installed separately > on > > the system. > > - They are likely to be somewhat tied to specific versions of Clang/LLVM. > > - To support cross-compiling, we should be able to install multiple > copies > > of these libraries in a single Clang installation, and use the > appropriate > > one when targeting a particular platform. > > - The above cross-compiling concern extends to ABI compatibility -- many > of > > the ABIs are already fixed in this space. > > - These are different from builtin header files which can use the > > preprocessor to internally differentiate their contents based on the > target > > platform. > > My proposed solution: > > - Base the path on the shared "resource directory" concept which already > > exists in Clang. > > - Builtin header files are at <ResourceDir>/include already. > > - Append a "base" triple directory name. > > - Append a "lib" directory name for runtime libraries > > - Place the runtime libraries as "libcompiler_rt_<sublib>.a" for each > > sub-library component "<sublib>". > > Example for "x/bin/clang" installation: > > x/bin/../lib/clang/3.1/x86_64-linux-gnu/lib/libcompiler_rt_asan.a > > What is a "base" triple? It is the simplest form we can reduce a triple > to > > while guaranteeing compatibility. The concept is best explained by an > > example. All of the following triples reduce to the base triple of > > "x86_64-linux-gnu": > > x86_64-linux-gnu > > x86_64-pc-linux-gnu > > x86_64-unknown-linux-gnu > > x86_64-redhat-linux > > x86_64-redhat-linux6E > > ... > > A different ABI could be expressed much like ARM's is with the last > element: > > "*-gnueabi". This would *not* reduce to the triple ending in '-gnu'. Due > to > > the adhoc and poorly spec'ed nature of existing triples, this will > largely > > be a fixed mapping much like already exists in the Clang driver, but > > consolidated into LLVM's core triple logic. > > > > Open Questions: > > How do we handle 'i686' vs 'i386'? Is it useful to have separate > installed > > libraries for i386 and i686 in order to get better performance for the > > latter *and* maintain compatibility for the former? We could collapse to > > either i386 or i686, and define either to mean whatever we want (for > > example, Debian uses i386-linux-gnu for its "base" triple in multiarch, > and > > does *not* support i386 processors). > > This scheme closely mirrors GCC's existing scheme with one exception: GCC > > puts the triple first, and the version number second. I don't think this > > matters greatly either way, but currently Clang puts its version number > > first. I think this reflects the inherent design of Clang to be > > cross-compiling by default, but it would be good to consider (even if we > > reject) matching GCC's behavior here. > > Should runtime libraries be installed as archives? .o files? .so files? > > (gasp) bitcode? Some mixture of these? What mixture, and how do we > decide? I > > lean toward .o files as bitcode where the linker supports it, normal .o > > files where it supports those, and .a files only as a fallback. Not very > > confident of these preferences though. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20111129/453939e4/attachment.html>
Reasonably Related Threads
- [LLVMdev] RFC: How should Clang/LLVM runtime libraries be installed and found during link steps?
- [LLVMdev] RFC: How should Clang/LLVM runtime libraries be installed and found during link steps?
- [LLVMdev] [cfe-dev] RFC: How should Clang/LLVM runtime libraries be installed and found during link steps?
- using emulated-tls on Darwin 8, 9, 10
- RFC: Default path for cross-compiled runtimes