Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup as an LLVM subproject is sustainable going forward without some form of testing support, automated or otherwise. The motivation: It's difficult to make changes to the source without some way to see if a commit breaks features or indeed works at all. This has a chilling effect on contributions -- without means to test changes the only verification strategy is visual inspection and finger crossing which isn't really a viable way to develop platform libraries. My understanding is that the proprietary OpenMP runtime test suite at Intel contains customer code so there's little prospect of getting released for use within LLVM. So we can rule that out. As for the prospect of growing a test suite organically, there's a very low level of activity in LLVM OpenMP SVN characterised by occasional Intel code drops by Jim and my own work to get ToT tree building and running, with the previous commit being in December. So it's clear that a test suite, even a small one, isn't going to just "show up" through casual effort. As such I'd like to propose that we set aside a few days, perhaps with developers from Nuanti and Intel and anyone else who wants to lend a hand, to adapt a branch of the libgomp test suite to be hosted externally, that can be used to regression test the LLVM OpenMP runtime. This shouldn't be much work given that the libraries are ABI/API-compatible and will at least show a way forward. From a licensing point of view it'll be similar to the way we point clang users to external gcc documentation, or the way dragonegg plugs into external gcc builds -- the runtime sources themselves are unaffected other than a workflow change. I don't see alternatives but I'll open the floor to other suggestions keeping in mind that without tests the project is starting to look dead in the water. (This mail touches on the OpenMP runtime that affects LLVM and clang so I'm copying in those lists.) Alp. -- http://www.nuanti.com the browser experts
On Feb 25, 2014, at 15:13 , Alp Toker <alp at nuanti.com> wrote:> Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup as an LLVM subproject is sustainable going forward without some form of testing support, automated or otherwise....> Alp.I work at a small start up that writes a lot of computationally-intensive code. The guys who work on that use OpenMP, but they're primarily targeting our Linux-based servers, and typically build with GCC. However, a big chunk of that code also goes into the iOS app that I'm responsible for, and thus OpenMP support is very much something we desire. Using macros that either use OpenMP or GCD, we addressed our biggest hot spots, but only for a very particular class of OpenMP constructs. I'd love to see it fully integrated into Xcode. Not sure how much I can help, but if you want me to try to build our stuff with it as it moves forward, let me know. Mostly I just wanted to voice my support for the effort. -- Rick -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 204 bytes Desc: Message signed with OpenPGP using GPGMail URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140225/77cb77e7/attachment.sig>
On 25/02/2014 23:58, Rick Mann wrote:> On Feb 25, 2014, at 15:13 , Alp Toker <alp at nuanti.com> wrote: > >> Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup as an LLVM subproject is sustainable going forward without some form of testing support, automated or otherwise. > ... >> Alp. > I work at a small start up that writes a lot of computationally-intensive code. The guys who work on that use OpenMP, but they're primarily targeting our Linux-based servers, and typically build with GCC. However, a big chunk of that code also goes into the iOS app that I'm responsible for, and thus OpenMP support is very much something we desire. > > Using macros that either use OpenMP or GCD, we addressed our biggest hot spots, but only for a very particular class of OpenMP constructs. I'd love to see it fully integrated into Xcode.Hi Rick, Keep in mind that LLVM integration doesn't imply or even indicate Xcode support, but you're right, pulling things together is a step in the right direction. Achieving a complete stack is likely to encourage the ongoing effort towards OpenMP language support in the clang frontend as well.> > Not sure how much I can help, but if you want me to try to build our stuff with it as it moves forward, let me know. Mostly I just wanted to voice my support for the effort. >As for trying it out, I suspect the infrastructure isn't quite there yet to respond to bug reports if you hit issues (no bugs reports or fixes have been made yet since the runtime project launched) but James Cownie might be able to answer better. It's easy to build and there haven't been success/failure reports in public yet so sure, if you have a minute to try the runtime with gcc or icc that could be useful! Thanks Alp. -- http://www.nuanti.com the browser experts
On 25 Feb 2014, at 23:13, Alp Toker <alp at nuanti.com> wrote:> Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup as an LLVM subproject is sustainable going forward without some form of testing support, automated or otherwise.I'd add that fairly high up the TODO list should probably be a build system that's a bit easier to modify. We're very interested in getting the runtime working on FreeBSD with a view to importing it into the base system at some point in the future (we've removed the GNU OpenMP runtime and would quite like a replacement). Unfortunately, the current build system is a complex structure of Makefile driving Perl scripts and vice versa. After half an hour of trying to persuade it that FreeBSD was, in fact, a valid target, I gave up. The description of what things are valid targets is spread over multiple files with subtle interactions between them. We have a number of people producing experimental manycore 64-bit MIPS systems running FreeBSD, so we'd also be interested in doing MIPS bring-up, but the build system is currently something of a show-stopper for us. David
> Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal > to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup > as an LLVM subproject is sustainable going forward without some form of testing support, automated > or otherwise.> The motivation: It's difficult to make changes to the source without some way to see if a commit breaks > features or indeed works at all. This has a chilling effect on contributions -- without means to test > changes the only verification strategy is visual inspection and finger crossing which isn't really > a viable way to develop platform libraries.I agree with all of that!> My understanding is that the proprietary OpenMP runtime test suite at Intel contains customer code so > there's little prospect of getting released for use within LLVM. So we can rule that out.Correct. It *may* contain code snippets from customers for regression testing, and we have lawyers, so even "may" is enough to seriously worry them. Attempting to audit it would be such a big job that starting from somewhere else is likely to be more productive.> As such I'd like to propose that we set aside a few days, perhaps with developers from Nuanti > and Intel and anyone else who wants to lend a hand, to adapt a branch of the libgomp test suite to be > hosted externally, that can be used to regression test the LLVM OpenMP runtime. > This shouldn't be much work given that the libraries are ABI/API-compatible and will at least > show a way forward.Another potential solution would be to take the validation suite from the OpenUH compiler http://web.cs.uh.edu/~openuh/download/ (which is BSD licensed), and start to build a more comprehensive set of tests from that. Since it is BSD I believe we could take it without asking, however I'd like to get an ACK from the authors before doing that. The other issues to consider here are 1) Which compiler is used to compile the tests?. Without the OpenMP support in clang we still don't have our own compiler that can do that, so we'd presumably have to use gcc. 2) If we build the runtime with clang and run tests built with gcc we need to be aware that there will be a potential incompatibility, since gcc supports a 128 bit float, and clang/llvm don't. Therefore it is impossible to compile the support for 128 bit reductions (which exists in the runtime source) using clang. (Though gcc may not be calling the runtime to do reductions anyway, in which case the issue is a more general one of test coverage). -- Jim James Cownie <james.h.cownie at intel.com> SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes) Tel: +44 117 9071438 -----Original Message----- From: Alp Toker [mailto:alp at nuanti.com] Sent: Tuesday, February 25, 2014 11:14 PM To: openmp-dev at dcs-maillist2.engr.illinois.edu Cc: llvmdev at cs.uiuc.edu; clang-dev Developers; Cownie, James H; Hal Finkel; Dmitri Gribenko Subject: Future of the LLVM OpenMP runtime Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup as an LLVM subproject is sustainable going forward without some form of testing support, automated or otherwise. The motivation: It's difficult to make changes to the source without some way to see if a commit breaks features or indeed works at all. This has a chilling effect on contributions -- without means to test changes the only verification strategy is visual inspection and finger crossing which isn't really a viable way to develop platform libraries. My understanding is that the proprietary OpenMP runtime test suite at Intel contains customer code so there's little prospect of getting released for use within LLVM. So we can rule that out. As for the prospect of growing a test suite organically, there's a very low level of activity in LLVM OpenMP SVN characterised by occasional Intel code drops by Jim and my own work to get ToT tree building and running, with the previous commit being in December. So it's clear that a test suite, even a small one, isn't going to just "show up" through casual effort. As such I'd like to propose that we set aside a few days, perhaps with developers from Nuanti and Intel and anyone else who wants to lend a hand, to adapt a branch of the libgomp test suite to be hosted externally, that can be used to regression test the LLVM OpenMP runtime. This shouldn't be much work given that the libraries are ABI/API-compatible and will at least show a way forward. From a licensing point of view it'll be similar to the way we point clang users to external gcc documentation, or the way dragonegg plugs into external gcc builds -- the runtime sources themselves are unaffected other than a workflow change. I don't see alternatives but I'll open the floor to other suggestions keeping in mind that without tests the project is starting to look dead in the water. (This mail touches on the OpenMP runtime that affects LLVM and clang so I'm copying in those lists.) Alp. -- http://www.nuanti.com the browser experts --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
John Leidel (jleidel)
2014-Feb-26 12:53 UTC
[LLVMdev] [Openmp-dev] Future of the LLVM OpenMP runtime
James, I believe the UofHouston folks would be quite pleased that this project would utilize their test suite. Having used it for previous projects, I can say that its quite thorough once you get your arms wrapped around the infrastructure. We'll likely need to do a bit of work in order to get tests running for OpenMP 4.0 additions. If anyone is interested, Sunita Chandrasekaran presented a paper at IWOMP'12 describing the infrastructure. http://dl.acm.org/citation.cfm?id=2345831 cheers john On Feb 26, 2014, at 4:29 AM, "Cownie, James H" <james.h.cownie at intel.com> wrote:>> Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal >> to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup >> as an LLVM subproject is sustainable going forward without some form of testing support, automated >> or otherwise. > >> The motivation: It's difficult to make changes to the source without some way to see if a commit breaks >> features or indeed works at all. This has a chilling effect on contributions -- without means to test >> changes the only verification strategy is visual inspection and finger crossing which isn't really >> a viable way to develop platform libraries. > > I agree with all of that! > >> My understanding is that the proprietary OpenMP runtime test suite at Intel contains customer code so >> there's little prospect of getting released for use within LLVM. So we can rule that out. > > Correct. It *may* contain code snippets from customers for regression testing, and we have lawyers, so > even "may" is enough to seriously worry them. Attempting to audit it would be such a big job that > starting from somewhere else is likely to be more productive. > >> As such I'd like to propose that we set aside a few days, perhaps with developers from Nuanti >> and Intel and anyone else who wants to lend a hand, to adapt a branch of the libgomp test suite to be >> hosted externally, that can be used to regression test the LLVM OpenMP runtime. >> This shouldn't be much work given that the libraries are ABI/API-compatible and will at least >> show a way forward. > > Another potential solution would be to take the validation suite from the OpenUH compiler > http://web.cs.uh.edu/~openuh/download/ (which is BSD licensed), and start to build a more > comprehensive set of tests from that. Since it is BSD I believe we could take it without asking, > however I'd like to get an ACK from the authors before doing that. > > The other issues to consider here are > 1) Which compiler is used to compile the tests?. > Without the OpenMP support in clang we still don't have our own compiler that can do that, so > we'd presumably have to use gcc. > 2) If we build the runtime with clang and run tests built with gcc we need to be aware that there will be > a potential incompatibility, since gcc supports a 128 bit float, and clang/llvm don't. Therefore > it is impossible to compile the support for 128 bit reductions (which exists in the runtime source) > using clang. (Though gcc may not be calling the runtime to do reductions anyway, in which case the issue > is a more general one of test coverage). > > > -- Jim > > James Cownie <james.h.cownie at intel.com> > SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes) > Tel: +44 117 9071438 > > > -----Original Message----- > From: Alp Toker [mailto:alp at nuanti.com] > Sent: Tuesday, February 25, 2014 11:14 PM > To: openmp-dev at dcs-maillist2.engr.illinois.edu > Cc: llvmdev at cs.uiuc.edu; clang-dev Developers; Cownie, James H; Hal Finkel; Dmitri Gribenko > Subject: Future of the LLVM OpenMP runtime > > Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup as an LLVM subproject is sustainable going forward without some form of testing support, automated or otherwise. > > The motivation: It's difficult to make changes to the source without some way to see if a commit breaks features or indeed works at all. This has a chilling effect on contributions -- without means to test changes the only verification strategy is visual inspection and finger crossing which isn't really a viable way to develop platform libraries. > > My understanding is that the proprietary OpenMP runtime test suite at Intel contains customer code so there's little prospect of getting released for use within LLVM. So we can rule that out. > > As for the prospect of growing a test suite organically, there's a very low level of activity in LLVM OpenMP SVN characterised by occasional Intel code drops by Jim and my own work to get ToT tree building and running, with the previous commit being in December. So it's clear that a test suite, even a small one, isn't going to just "show up" through casual effort. > > As such I'd like to propose that we set aside a few days, perhaps with developers from Nuanti and Intel and anyone else who wants to lend a hand, to adapt a branch of the libgomp test suite to be hosted externally, that can be used to regression test the LLVM OpenMP runtime. > This shouldn't be much work given that the libraries are ABI/API-compatible and will at least show a way forward. > > From a licensing point of view it'll be similar to the way we point clang users to external gcc documentation, or the way dragonegg plugs into external gcc builds -- the runtime sources themselves are unaffected other than a workflow change. I don't see alternatives but I'll open the floor to other suggestions keeping in mind that without tests the project is starting to look dead in the water. > > (This mail touches on the OpenMP runtime that affects LLVM and clang so I'm copying in those lists.) > > Alp. > > -- > http://www.nuanti.com > the browser experts > > --------------------------------------------------------------------- > Intel Corporation (UK) Limited > Registered No. 1134945 (England) > Registered Office: Pipers Way, Swindon SN3 1RJ > VAT No: 860 2173 47 > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. > > > _______________________________________________ > Openmp-dev mailing list > Openmp-dev at dcs-maillist2.engr.illinois.edu > http://lists.cs.uiuc.edu/mailman/listinfo/openmp-dev
On 26/02/2014 09:03, David Chisnall wrote:> On 25 Feb 2014, at 23:13, Alp Toker <alp at nuanti.com> wrote: > >> Now that we've kick-started the LLVM OpenMP runtime discussion, I want to make a concrete proposal to get a test suite up and running for the LLVM OpenMP runtime. I don't think the current setup as an LLVM subproject is sustainable going forward without some form of testing support, automated or otherwise. > I'd add that fairly high up the TODO list should probably be a build system that's a bit easier to modify. We're very interested in getting the runtime working on FreeBSD with a view to importing it into the base system at some point in the future (we've removed the GNU OpenMP runtime and would quite like a replacement).Some good news: We have a FreeBSD port covering both the build system and the OpenMP runtime itself. I'll see if we can contribute this upstream in the next few days. This work generalises a few abstractions to ease porting to other platforms as well.> > Unfortunately, the current build system is a complex structure of Makefile driving Perl scripts and vice versa. After half an hour of trying to persuade it that FreeBSD was, in fact, a valid target, I gave up. The description of what things are valid targets is spread over multiple files with subtle interactions between them.Agree a CMake build system is desirable. Some of the perl configure and generation code is ingrained and might have to stay around longer.> > We have a number of people producing experimental manycore 64-bit MIPS systems running FreeBSD, so we'd also be interested in doing MIPS bring-up, but the build system is currently something of a show-stopper for us.The port is working on FreeBSD 10 x86_64 (modulo CPU affinity support) so MIPS shouldn't be a large leap from there, hopefully this time with some test coverage. Alp. -- http://www.nuanti.com the browser experts