Chandler, Thanks for the great overview. On Fri, Jan 31, 2014 at 12:12 PM, Chandler Carruth <chandlerc at google.com>wrote:> > On Thu, Jan 30, 2014 at 1:54 PM, Renato Golin <renato.golin at linaro.org>wrote: > >> On 30 January 2014 21:50, Reid Kleckner <rnk at google.com> wrote: >>> >>> I don't see any compelling reason to split the sanitizers out today. >>> >> >> Clear, next. >> > > Just as a side note, I had some thoughts about the organization of this > stuff a while back that I wanted to replay here. > > Fundamentally, I feel like we're in particular getting a few things > conflated. Maybe separating them out helps: > > - There is the repository "compiler-rt" that holds all of the runtime > libraries that (if desired) need to be shipped along side the compiler. > They're separated so that they can be omitted when they aren't needed, and > potentially to isolate an unusual desired property: we would really like to > (eventually) build them with the just-built compiler rather than the host > compiler (where possible, clearly this requires the target compiler to be > executable on the host). > > - There is the core runtime library. Historically this was called > 'compiler-rt' informally, but perhaps better called 'libclang_rt', which > provides the core necessary runtime library facilities to compile C or C++ > applications. It's analogous to libgcc but without some of the unwinding > code (as I understand it, there may be details I'm wrong about here or > glossing over, but it's not relevant to the organization of things). > > - There are several optional runtime libraries to support specific > compiler / toolchain features. These include the sanitizers and profiling > libraries. > > I think all of the libraries here make sense in the same repository > because of the shared concerns of building runtime libraries. For example, > it would be useful to compile them with the just-built-clang when target is > executable on the host, and it would be useful to automatically cross > compile versions of them for as many targets as are possible on the given > host and supported by the host compiler (if used) and the just-built-clang. > > However, the organization of the tree is ... very hard to understand. > originally, there was only the one 'libclang_rt' library that had its > generic C99 implementation in 'lib', and architecture-specific assembly > routines when desirable in architecture subdirectories of lib. When we > added new libraries, we put them in subdirectories of lib, making the whole > thing kind of a mess. My suggestion to fix this was to create a "core" > subdirectory of lib to contain the code used for 'libclang_rt'. "core" is a > terrible name, but i've no better. suggestions welcome there. Then we would > have a more sensible organization of the 'lib' tree. >I can't suggest a better name than "core" (maybe, "runtime"?), but would *love* to put 'libclang_rt' into a separate directory under /compiler-rt/lib :) We should've done it long ago, IMO.> > I also think it might be useful to have a single large test tree (much > like with llvm or clang) that has subdirectories for the various tests > rather than test directories under lib/asan/ and friends, but maybe the > sanitizer folks have objections to that. consistency seems a compelling > reason here, but there might be other compelling concerns. >Yes, I don't like the way testing is organized either. Originally it made sense to put sanitizer lit tests under lib/xsan/, but now we've got a lot of sanitizers, and a lot of duplicating configuration code. As Renato mentioned, browsing through lit configs is a bit painful. I would like to proceed with re-organizing the tests for sanitizers, unless anyone objects. Kostya?> Now, the build system has always been problematic because the build system > for this tree is *hard* and no one who has worked on it has really had the > time to do an extremely thorough job and finish all aspects. It's a huge > project. Right now, the makefile system does a good job of using the > just-built-clang and can do a limited amount of cross-building runtimes for > other targets. But the makefile system of compiler-rt is also terribly, > terribly complex, doesn't follow the conventions of LLVM's makefiles, and > is generally painful to maintain and update, so folks have been reluctant > to flesh out its support for new libraries and other new things. >Note that it would be really hard to add support for running test suites into Makefile build system. It's also good for the one-time build only, not for continuous development - I don't think it respects dependencies correctly.> > The CMake system is much cleaner in some respects (a bit less opaque to > the folks trying to maintain it), but CMake makes it much harder to use the > just-built-clang, especially for the C++ runtime code. The consequence is > that we've never finished either the cross-building or > just-built-clang-hosting features that are desirable. >I once tried to implement a pseudo-build-system for compiler-rt on top of CMake, so that we use "just-built" Clang instead of a host compiler, but failed miserably. Maybe I was doing wrong things, but I got the impression that CMake isn't suited for switching the compiler on the fly :)> Any work toward these would be really awesome to see, but is a huge pile > of work. Finally, when I was originally doing the CMake build for this I > didn't understand what really needed to be done to build and use the core > 'libclang_rt' library, and so I don't think I got it right. Some folks have > sent patches to improve it, but I suspect it still really needs more work > to be a solid system to use instead of libgcc. So I'm really excited about > your emails. =] Having a more self-contained stack would be a significant > improvement. > > > I think the obvious incremental steps are to disable building any parts of > compiler-rt that don't build cleanly. >As I mentioned above, currently in CMake build system we try to avoid building anything if we're not sure we can produce a working and correct library on the host platform. That is, I still don't see what the problem is - it's relatively easy to enable building just the compiler-rt library on ARM and not enable building sanitizers on ARM.> There should never be a requirement for you to port an optional runtime > library unless you need it. =] I think having good ports is important, but > that should never block progress getting other thinsg ported and working > well. > > If it helps to reorganize things, I'm happy to even help there as much as > I can. I agree that the organization isn't great, but I *really* didn't > want to fight the makefile build system to do the reorganization myself, so > its something that has lingered too long. > > > Sorry for the long ramble, but hopefully this gives you some of tho > context. > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > >-- Alexey Samsonov, MSK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140131/c987d752/attachment.html>
Kostya Serebryany
2014-Jan-31 08:55 UTC
[LLVMdev] [cfe-dev] Sanitizers libs in Compiler-RT
> > > Yes, I don't like the way testing is organized either. Originally it made > sense to put sanitizer lit tests under lib/xsan/, but now we've got a lot > of sanitizers, and a lot of duplicating configuration code. As Renato > mentioned, browsing > through lit configs is a bit painful. I would like to proceed with > re-organizing the tests for sanitizers, unless anyone objects. Kostya? > > Any such changes would horrify me, but if you are willing to do them --great. Once thing which I'd like to have improved: we currently have several lit tests in asan or msan or tsan or lsan which really ought to be run for all of the tools (e.g. tests for interceptors) --kcc -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140131/0af533bf/attachment.html>
Hi Alexey, On 31 Jan 2014, at 08:50, Alexey Samsonov wrote:> Thanks for the great overview.+1 ( and +1 for David's comment about moving the unwind stuff into this area)> > On Fri, Jan 31, 2014 at 12:12 PM, Chandler Carruth <chandlerc at google.com> wrote: > > On Thu, Jan 30, 2014 at 1:54 PM, Renato Golin <renato.golin at linaro.org> wrote: > On 30 January 2014 21:50, Reid Kleckner <rnk at google.com> wrote: > I don't see any compelling reason to split the sanitizers out today. > > Clear, next. > > Just as a side note, I had some thoughts about the organization of this stuff a while back that I wanted to replay here. > > Fundamentally, I feel like we're in particular getting a few things conflated. Maybe separating them out helps: > > - There is the repository "compiler-rt" that holds all of the runtime libraries that (if desired) need to be shipped along side the compiler. They're separated so that they can be omitted when they aren't needed, and potentially to isolate an unusual desired property: we would really like to (eventually) build them with the just-built compiler rather than the host compiler (where possible, clearly this requires the target compiler to be executable on the host). > > - There is the core runtime library. Historically this was called 'compiler-rt' informally, but perhaps better called 'libclang_rt', which provides the core necessary runtime library facilities to compile C or C++ applications. It's analogous to libgcc but without some of the unwinding code (as I understand it, there may be details I'm wrong about here or glossing over, but it's not relevant to the organization of things). > > - There are several optional runtime libraries to support specific compiler / toolchain features. These include the sanitizers and profiling libraries. > > I think all of the libraries here make sense in the same repository because of the shared concerns of building runtime libraries. For example, it would be useful to compile them with the just-built-clang when target is executable on the host, and it would be useful to automatically cross compile versions of them for as many targets as are possible on the given host and supported by the host compiler (if used) and the just-built-clang. > > However, the organization of the tree is ... very hard to understand. originally, there was only the one 'libclang_rt' library that had its generic C99 implementation in 'lib', and architecture-specific assembly routines when desirable in architecture subdirectories of lib. When we added new libraries, we put them in subdirectories of lib, making the whole thing kind of a mess. My suggestion to fix this was to create a "core" subdirectory of lib to contain the code used for 'libclang_rt'. "core" is a terrible name, but i've no better. suggestions welcome there. Then we would have a more sensible organization of the 'lib' tree. > > I can't suggest a better name than "core" (maybe, "runtime"?), but would *love* to put 'libclang_rt' into a separate directory under /compiler-rt/lib :) We should've done it long ago, IMO. > > I also think it might be useful to have a single large test tree (much like with llvm or clang) that has subdirectories for the various tests rather than test directories under lib/asan/ and friends, but maybe the sanitizer folks have objections to that. consistency seems a compelling reason here, but there might be other compelling concerns. > > Yes, I don't like the way testing is organized either. Originally it made sense to put sanitizer lit tests under lib/xsan/, but now we've got a lot of sanitizers, and a lot of duplicating configuration code. As Renato mentioned, browsing > through lit configs is a bit painful. I would like to proceed with re-organizing the tests for sanitizers, unless anyone objects. Kostya? > > > Now, the build system has always been problematic because the build system for this tree is *hard* and no one who has worked on it has really had the time to do an extremely thorough job and finish all aspects. It's a huge project. Right now, the makefile system does a good job of using the just-built-clang and can do a limited amount of cross-building runtimes for other targets. But the makefile system of compiler-rt is also terribly, terribly complex, doesn't follow the conventions of LLVM's makefiles, and is generally painful to maintain and update, so folks have been reluctant to flesh out its support for new libraries and other new things. > > Note that it would be really hard to add support for running test suites into Makefile build system. It's also good for the one-time build only, not for continuous development - I don't think it respects dependencies correctly.Please could you expand upon this? Certainly, one can expect to test the runtime on the host? likewise, any pointers to where you see wrong dependencies would be appreciated. At present, config & make appears to build a sensible lib using the just-built compiler [and in the general case, that's the only sensible solution, since the host might not have cross-tools for the set of archs you want to support with llvm/clang] In some future ideal world, we might cook up a cross-testing environment - of course, it's a given that that would require testers to have access to the target hardware they wanted to test on. Iain> > > The CMake system is much cleaner in some respects (a bit less opaque to the folks trying to maintain it), but CMake makes it much harder to use the just-built-clang, especially for the C++ runtime code. The consequence is that we've never finished either the cross-building or just-built-clang-hosting features that are desirable. > > I once tried to implement a pseudo-build-system for compiler-rt on top of CMake, so that we use "just-built" Clang instead of a host compiler, but failed miserably. Maybe I was doing wrong things, but I got the impression > that CMake isn't suited for switching the compiler on the fly :) > > Any work toward these would be really awesome to see, but is a huge pile of work. Finally, when I was originally doing the CMake build for this I didn't understand what really needed to be done to build and use the core 'libclang_rt' library, and so I don't think I got it right. Some folks have sent patches to improve it, but I suspect it still really needs more work to be a solid system to use instead of libgcc. So I'm really excited about your emails. =] Having a more self-contained stack would be a significant improvement. > > > I think the obvious incremental steps are to disable building any parts of compiler-rt that don't build cleanly. > > As I mentioned above, currently in CMake build system we try to avoid building anything if we're not sure we can produce a working and correct library on the host platform. That is, I still don't see what the problem is - it's relatively easy to enable building just the compiler-rt library on ARM and not enable building sanitizers on ARM. > > There should never be a requirement for you to port an optional runtime library unless you need it. =] I think having good ports is important, but that should never block progress getting other thinsg ported and working well. > > If it helps to reorganize things, I'm happy to even help there as much as I can. I agree that the organization isn't great, but I *really* didn't want to fight the makefile build system to do the reorganization myself, so its something that has lingered too long. > > > Sorry for the long ramble, but hopefully this gives you some of tho context. > > _______________________________________________ > cfe-dev mailing list > cfe-dev at cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > > -- > Alexey Samsonov, MSK > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On Fri, Jan 31, 2014 at 1:04 PM, Iain Sandoe <iain at codesourcery.com> wrote:> Hi Alexey, > > On 31 Jan 2014, at 08:50, Alexey Samsonov wrote: > > > Thanks for the great overview. > +1 > ( and +1 for David's comment about moving the unwind stuff into this area) > > > > > On Fri, Jan 31, 2014 at 12:12 PM, Chandler Carruth <chandlerc at google.com> > wrote: > > > > On Thu, Jan 30, 2014 at 1:54 PM, Renato Golin <renato.golin at linaro.org> > wrote: > > On 30 January 2014 21:50, Reid Kleckner <rnk at google.com> wrote: > > I don't see any compelling reason to split the sanitizers out today. > > > > Clear, next. > > > > Just as a side note, I had some thoughts about the organization of this > stuff a while back that I wanted to replay here. > > > > Fundamentally, I feel like we're in particular getting a few things > conflated. Maybe separating them out helps: > > > > - There is the repository "compiler-rt" that holds all of the runtime > libraries that (if desired) need to be shipped along side the compiler. > They're separated so that they can be omitted when they aren't needed, and > potentially to isolate an unusual desired property: we would really like to > (eventually) build them with the just-built compiler rather than the host > compiler (where possible, clearly this requires the target compiler to be > executable on the host). > > > > - There is the core runtime library. Historically this was called > 'compiler-rt' informally, but perhaps better called 'libclang_rt', which > provides the core necessary runtime library facilities to compile C or C++ > applications. It's analogous to libgcc but without some of the unwinding > code (as I understand it, there may be details I'm wrong about here or > glossing over, but it's not relevant to the organization of things). > > > > - There are several optional runtime libraries to support specific > compiler / toolchain features. These include the sanitizers and profiling > libraries. > > > > I think all of the libraries here make sense in the same repository > because of the shared concerns of building runtime libraries. For example, > it would be useful to compile them with the just-built-clang when target is > executable on the host, and it would be useful to automatically cross > compile versions of them for as many targets as are possible on the given > host and supported by the host compiler (if used) and the just-built-clang. > > > > However, the organization of the tree is ... very hard to understand. > originally, there was only the one 'libclang_rt' library that had its > generic C99 implementation in 'lib', and architecture-specific assembly > routines when desirable in architecture subdirectories of lib. When we > added new libraries, we put them in subdirectories of lib, making the whole > thing kind of a mess. My suggestion to fix this was to create a "core" > subdirectory of lib to contain the code used for 'libclang_rt'. "core" is a > terrible name, but i've no better. suggestions welcome there. Then we would > have a more sensible organization of the 'lib' tree. > > > > I can't suggest a better name than "core" (maybe, "runtime"?), but would > *love* to put 'libclang_rt' into a separate directory under > /compiler-rt/lib :) We should've done it long ago, IMO. > > > > I also think it might be useful to have a single large test tree (much > like with llvm or clang) that has subdirectories for the various tests > rather than test directories under lib/asan/ and friends, but maybe the > sanitizer folks have objections to that. consistency seems a compelling > reason here, but there might be other compelling concerns. > > > > Yes, I don't like the way testing is organized either. Originally it > made sense to put sanitizer lit tests under lib/xsan/, but now we've got a > lot of sanitizers, and a lot of duplicating configuration code. As Renato > mentioned, browsing > > through lit configs is a bit painful. I would like to proceed with > re-organizing the tests for sanitizers, unless anyone objects. Kostya? > > > > > > Now, the build system has always been problematic because the build > system for this tree is *hard* and no one who has worked on it has really > had the time to do an extremely thorough job and finish all aspects. It's a > huge project. Right now, the makefile system does a good job of using the > just-built-clang and can do a limited amount of cross-building runtimes for > other targets. But the makefile system of compiler-rt is also terribly, > terribly complex, doesn't follow the conventions of LLVM's makefiles, and > is generally painful to maintain and update, so folks have been reluctant > to flesh out its support for new libraries and other new things. > > > > Note that it would be really hard to add support for running test suites > into Makefile build system. It's also good for the one-time build only, not > for continuous development - I don't think it respects dependencies > correctly. > > Please could you expand upon this? >Well, LLVM/Clang's configure+make and compiler-rt's make are disjoint - when you run "make" in Clang build tree, at one point it simply invokes, the Makefile in compiler-rt directory. But if we want more functionality, like running the tests, we ought to have a single build system, single set of "targets" (binaries, libraries, test suites) with dependencies between them. For example, I want to have "make check-asan" command which I can run from the root of the LLVM/Clang's build tree. More, if I run "make check-asan", I need to scan the dependencies of that test suite, and: 1) If ASan runtime has changed, re-build it, and re-run tests. 2) If FileCheck or lit sources have changed, I should re-build them (but not ASan runtime) and re-run tests. 3) If Clang sources have changed, I should re-build Clang, re-build ASan runtime (*) and re-run tests. With CMake we're able to do all that (except for (*), as we build runtimes with host compiler) at the moment.> Certainly, one can expect to test the runtime on the host? > likewise, any pointers to where you see wrong dependencies would be > appreciated. >IIRC, "make check-all" in configure+make build tree doesn't re-build the necessary libraries automatically.> > At present, config & make appears to build a sensible lib using the > just-built compiler [and in the general case, that's the only sensible > solution, since the host might not have cross-tools for the set of archs > you want to support with llvm/clang] >Yes, that's what configure+make is good at.> In some future ideal world, we might cook up a cross-testing environment - > of course, it's a given that that would require testers to have access to > the target hardware they wanted to test on. > > Iain > > > > > > > The CMake system is much cleaner in some respects (a bit less opaque to > the folks trying to maintain it), but CMake makes it much harder to use the > just-built-clang, especially for the C++ runtime code. The consequence is > that we've never finished either the cross-building or > just-built-clang-hosting features that are desirable. > > > > I once tried to implement a pseudo-build-system for compiler-rt on top > of CMake, so that we use "just-built" Clang instead of a host compiler, but > failed miserably. Maybe I was doing wrong things, but I got the impression > > that CMake isn't suited for switching the compiler on the fly :) > > > > Any work toward these would be really awesome to see, but is a huge pile > of work. Finally, when I was originally doing the CMake build for this I > didn't understand what really needed to be done to build and use the core > 'libclang_rt' library, and so I don't think I got it right. Some folks have > sent patches to improve it, but I suspect it still really needs more work > to be a solid system to use instead of libgcc. So I'm really excited about > your emails. =] Having a more self-contained stack would be a significant > improvement. > > > > > > I think the obvious incremental steps are to disable building any parts > of compiler-rt that don't build cleanly. > > > > As I mentioned above, currently in CMake build system we try to avoid > building anything if we're not sure we can produce a working and correct > library on the host platform. That is, I still don't see what the problem > is - it's relatively easy to enable building just the compiler-rt library > on ARM and not enable building sanitizers on ARM. > > > > There should never be a requirement for you to port an optional runtime > library unless you need it. =] I think having good ports is important, but > that should never block progress getting other thinsg ported and working > well. > > > > If it helps to reorganize things, I'm happy to even help there as much > as I can. I agree that the organization isn't great, but I *really* didn't > want to fight the makefile build system to do the reorganization myself, so > its something that has lingered too long. > > > > > > Sorry for the long ramble, but hopefully this gives you some of tho > context. > > > > _______________________________________________ > > cfe-dev mailing list > > cfe-dev at cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev > > > > > > > > > > -- > > Alexey Samsonov, MSK > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-- Alexey Samsonov, MSK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140131/c75185d8/attachment.html>
As Kostya mentioned, there are many tests in compiler-rt that should be shared between several sanitizers. This has bitten us more than once in the past. On Fri, Jan 31, 2014 at 1:04 PM, Iain Sandoe <iain at codesourcery.com> wrote:> Hi Alexey, > > On 31 Jan 2014, at 08:50, Alexey Samsonov wrote: > >> Thanks for the great overview. > +1 > ( and +1 for David's comment about moving the unwind stuff into this area) > >> >> On Fri, Jan 31, 2014 at 12:12 PM, Chandler Carruth <chandlerc at google.com> wrote: >> >> On Thu, Jan 30, 2014 at 1:54 PM, Renato Golin <renato.golin at linaro.org> wrote: >> On 30 January 2014 21:50, Reid Kleckner <rnk at google.com> wrote: >> I don't see any compelling reason to split the sanitizers out today. >> >> Clear, next. >> >> Just as a side note, I had some thoughts about the organization of this stuff a while back that I wanted to replay here. >> >> Fundamentally, I feel like we're in particular getting a few things conflated. Maybe separating them out helps: >> >> - There is the repository "compiler-rt" that holds all of the runtime libraries that (if desired) need to be shipped along side the compiler. They're separated so that they can be omitted when they aren't needed, and potentially to isolate an unusual desired property: we would really like to (eventually) build them with the just-built compiler rather than the host compiler (where possible, clearly this requires the target compiler to be executable on the host). >> >> - There is the core runtime library. Historically this was called 'compiler-rt' informally, but perhaps better called 'libclang_rt', which provides the core necessary runtime library facilities to compile C or C++ applications. It's analogous to libgcc but without some of the unwinding code (as I understand it, there may be details I'm wrong about here or glossing over, but it's not relevant to the organization of things). >> >> - There are several optional runtime libraries to support specific compiler / toolchain features. These include the sanitizers and profiling libraries. >> >> I think all of the libraries here make sense in the same repository because of the shared concerns of building runtime libraries. For example, it would be useful to compile them with the just-built-clang when target is executable on the host, and it would be useful to automatically cross compile versions of them for as many targets as are possible on the given host and supported by the host compiler (if used) and the just-built-clang. >> >> However, the organization of the tree is ... very hard to understand. originally, there was only the one 'libclang_rt' library that had its generic C99 implementation in 'lib', and architecture-specific assembly routines when desirable in architecture subdirectories of lib. When we added new libraries, we put them in subdirectories of lib, making the whole thing kind of a mess. My suggestion to fix this was to create a "core" subdirectory of lib to contain the code used for 'libclang_rt'. "core" is a terrible name, but i've no better. suggestions welcome there. Then we would have a more sensible organization of the 'lib' tree. >> >> I can't suggest a better name than "core" (maybe, "runtime"?), but would *love* to put 'libclang_rt' into a separate directory under /compiler-rt/lib :) We should've done it long ago, IMO. >> >> I also think it might be useful to have a single large test tree (much like with llvm or clang) that has subdirectories for the various tests rather than test directories under lib/asan/ and friends, but maybe the sanitizer folks have objections to that. consistency seems a compelling reason here, but there might be other compelling concerns. >> >> Yes, I don't like the way testing is organized either. Originally it made sense to put sanitizer lit tests under lib/xsan/, but now we've got a lot of sanitizers, and a lot of duplicating configuration code. As Renato mentioned, browsing >> through lit configs is a bit painful. I would like to proceed with re-organizing the tests for sanitizers, unless anyone objects. Kostya? >> >> >> Now, the build system has always been problematic because the build system for this tree is *hard* and no one who has worked on it has really had the time to do an extremely thorough job and finish all aspects. It's a huge project. Right now, the makefile system does a good job of using the just-built-clang and can do a limited amount of cross-building runtimes for other targets. But the makefile system of compiler-rt is also terribly, terribly complex, doesn't follow the conventions of LLVM's makefiles, and is generally painful to maintain and update, so folks have been reluctant to flesh out its support for new libraries and other new things. >> >> Note that it would be really hard to add support for running test suites into Makefile build system. It's also good for the one-time build only, not for continuous development - I don't think it respects dependencies correctly. > > Please could you expand upon this? > Certainly, one can expect to test the runtime on the host? > likewise, any pointers to where you see wrong dependencies would be appreciated. > > At present, config & make appears to build a sensible lib using the just-built compiler [and in the general case, that's the only sensible solution, since the host might not have cross-tools for the set of archs you want to support with llvm/clang] > > In some future ideal world, we might cook up a cross-testing environment - of course, it's a given that that would require testers to have access to the target hardware they wanted to test on.I'd very much like to have this for Android target.> > Iain > >> >> >> The CMake system is much cleaner in some respects (a bit less opaque to the folks trying to maintain it), but CMake makes it much harder to use the just-built-clang, especially for the C++ runtime code. The consequence is that we've never finished either the cross-building or just-built-clang-hosting features that are desirable. >> >> I once tried to implement a pseudo-build-system for compiler-rt on top of CMake, so that we use "just-built" Clang instead of a host compiler, but failed miserably. Maybe I was doing wrong things, but I got the impression >> that CMake isn't suited for switching the compiler on the fly :) >> >> Any work toward these would be really awesome to see, but is a huge pile of work. Finally, when I was originally doing the CMake build for this I didn't understand what really needed to be done to build and use the core 'libclang_rt' library, and so I don't think I got it right. Some folks have sent patches to improve it, but I suspect it still really needs more work to be a solid system to use instead of libgcc. So I'm really excited about your emails. =] Having a more self-contained stack would be a significant improvement. >> >> >> I think the obvious incremental steps are to disable building any parts of compiler-rt that don't build cleanly. >> >> As I mentioned above, currently in CMake build system we try to avoid building anything if we're not sure we can produce a working and correct library on the host platform. That is, I still don't see what the problem is - it's relatively easy to enable building just the compiler-rt library on ARM and not enable building sanitizers on ARM. >> >> There should never be a requirement for you to port an optional runtime library unless you need it. =] I think having good ports is important, but that should never block progress getting other thinsg ported and working well. >> >> If it helps to reorganize things, I'm happy to even help there as much as I can. I agree that the organization isn't great, but I *really* didn't want to fight the makefile build system to do the reorganization myself, so its something that has lingered too long. >> >> >> Sorry for the long ramble, but hopefully this gives you some of tho context. >> >> _______________________________________________ >> cfe-dev mailing list >> cfe-dev at cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev >> >> >> >> >> -- >> Alexey Samsonov, MSK >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On 31 January 2014 08:50, Alexey Samsonov <samsonov at google.com> wrote:> That is, I still don't see what the problem is - it's relatively easy to > enable building just the compiler-rt library on ARM and not enable building > sanitizers on ARM. >For some reason, when I added the compiler-rt directory (even before my CMake changes), the Clang tests *required* the Asan libraries in lib/clang/3.5/linux, which is why I added it in the first place. I think this is wrong and should be fixed (though I have no idea how) in the Clang CMake files. With that fixed, I could just enable the RT libs and worry about the Asan later. Right now, I can't. --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140131/180fb7d5/attachment.html>
On Fri, Jan 31, 2014 at 1:47 PM, Renato Golin <renato.golin at linaro.org>wrote:> On 31 January 2014 08:50, Alexey Samsonov <samsonov at google.com> wrote: > >> That is, I still don't see what the problem is - it's relatively easy to >> enable building just the compiler-rt library on ARM and not enable building >> sanitizers on ARM. >> > > For some reason, when I added the compiler-rt directory (even before my > CMake changes), the Clang tests *required* the Asan libraries in > lib/clang/3.5/linux, which is why I added it in the first place. I think > this is wrong and should be fixed (though I have no idea how) in the Clang > CMake files. >Huh? What tests are you talking about? I thought that tests for Clang driver verifiy that the command "clang++ -fsanitize=address ...." produce a correct linker invocation (with path to /lib/clang/3.5/linux/libclang_rt.asan-arm.a), but don't actually require the libraries to be built there. If it's not the case, we should fix that.> > With that fixed, I could just enable the RT libs and worry about the Asan > later. Right now, I can't. > > --renato >-- Alexey Samsonov, MSK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140131/7f786168/attachment.html>