Jason Henline via llvm-dev
2016-May-09 17:32 UTC
[llvm-dev] [cfe-dev] [Openmp-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries
I talked to Chandler about the name "offload_libs" vs "parallel_libs" and he said he thinks "offload" is too narrow of a term for the scope he sees for this subproject. One example he brought up was AVX 512. He thinks that code explicitly targeting CPU parallelism should also be included in this project, even though it doesn't fit in the category of "offloading". So that is an argument in favor of "parallel_libs" instead of "offload_libs". Chandler, please correct me if I misrepresented your thoughts on this. On Thu, Apr 28, 2016 at 9:55 AM Jason Henline <jhen at google.com> wrote:> Alexandre, > > Thanks for your thoughts on this. I was thinking that each subproject > library would be responsible for handling its own name and any associated > branding. For example, when evangelizing for StreamExecutor, I plan to > refer to it as StreamExecutor, not parallel_libs-StreamExecutor or > something like that. So I think it is OK for the top-level container > project to have an undistinguished name. Does that sound reasonable? > > On Thu, Apr 28, 2016 at 7:09 AM Alexandre Eichenberger <alexe at us.ibm.com> > wrote: > >> A suggestion on naming: by choosing too generic a name, you don't get any >> branding. >> >> My point for the omp library: if someone talks about GOMP, KMPC / IOMP, >> or LOMP, they would know we are talking about the GNU, Intel, IBM >> implementation. I don't think we get that with OMP, which was selected for >> the OMP runtime. >> >> So I would suggest to append "llvm" or "lr" for LLVM runtime, or >> something distinctive that you like. >> >> This make sense because users will have choices of runtimes, gcc linking >> to GOMP, LLVM runtime, LOMP on POWER; GOMP, LLVM, IOMP on Intel... and we >> will need to educate our users on which one to use. >> >> Alexandre >> >> >> ----------------------------------------------------------------------------------------------------- >> Alexandre Eichenberger, Master Inventor, Advanced Compiler Technologies >> - research: compiler optimization (OpenMP, multithreading, SIMD) >> - info: alexe at us.ibm.com http://www.research.ibm.com/people/a/alexe >> - phone: 914-945-1812 (work) 914-312-3618 (cell) >> >> >> >> ----- Original message ----- >> From: Andrey Bokhanko via cfe-dev <cfe-dev at lists.llvm.org> >> Sent by: "cfe-dev" <cfe-dev-bounces at lists.llvm.org> >> To: Jason Henline <jhen at google.com> >> >> Cc: llvm-dev <llvm-dev at lists.llvm.org>, clang developer list < >> cfe-dev at lists.llvm.org>, "openmp-dev at lists.llvm.org" < >> openmp-dev at lists.llvm.org> >> Subject: Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM >> subproject for parallelism runtime and support libraries >> Date: Thu, Apr 28, 2016 9:23 AM >> >> Jason, >> >> Very nice write-up. Well done! >> >> My two kopecks (before you asked, one kopeck is the smallest item of >> currency in Russia ;-)): >> * Bugzilla and mailing list requirements are not covered. Do you want to >> have a component in Bugzilla for each project? One for all of them? Mailing >> list -- a separate list for each project? (IMHO, an overkill) One for all >> of them? >> * Build system -- I suppose you expect all of the libs to be built >> separately, without a single unified cmake file. Correct? Also, I expect >> you want all of the libs to be integrated into LLVM build -- correct? This >> should be spelled out explicitly. >> * I don't really like "parallel" in the name. Both SE and libomptarget >> are libraries that handle offloading, not parallelism. I understand other >> libraries, to be added in the future, might deal with parallelism, but >> maybe we need a separate project for them? (Something Chris already >> hinted.) How about "offloading_lib"? >> >> Yours, >> Andrey >> >> >> On Wed, Apr 27, 2016 at 7:58 PM, Jason Henline via Openmp-dev < >> openmp-dev at lists.llvm.org> wrote: >> >> I've put together a proposed "charter" for this new project, which I am >> calling parallel_utils (although I'm very open to suggestions for a better >> name). The text of my charter is below, and I welcome any input on how it >> can be improved. >> >> >> ====================================================>> LLVM Parallel Utils Subproject Charter >> ====================================================>> >> ---------------------------------------------- >> Description >> ---------------------------------------------- >> The LLVM open source project will contain a subproject named >> `parallel_utils` which will host the development of libraries which are >> aimed at enabling parallelism in code and which are also closely tied to >> compiler technology. Examples of libraries suitable for hosting within the >> `parallel_utils` subproject are runtime libraries and parallel math >> libraries. The initial candidates for inclusion in this subproject are >> StreamExecutor and libomptarget which would live in the `streamexecutor` >> and `libomptarget` subdirectories of `parallel_utils`, respectively. >> >> The `parallel_utils` project will host a collection of libraries where >> each library may be dependent on other libraries from the project or may be >> completely independent of any other libraries in the project. The rationale >> for hosting independent libraries within the same subproject is that all >> libraries in the project are providing related functionality that lives at >> the intersection of parallelism and compiler technology. It is expected >> that some libraries which initially began as independent will develop >> dependencies over time either between existing libraries or by extracting >> common code that can be used by each. One of the purposes of this >> subproject is to provide a working space where such refactoring and code >> sharing can take place. >> >> Libraries in the `parallel_utils` subproject may also depend on the LLVM >> core libraries. This will be useful for avoiding duplication of code within >> the LLVM project for common utilities such as those found in the LLVM >> support library. >> >> >> ---------------------------------------------- >> Requirements >> ---------------------------------------------- >> Libraries included in the `parallel_utils` subproject must strive to >> achieve the following requirements: >> >> 1. Adhere to the LLVM coding standards. >> 2. Use the LLVM build and test infrastructure. >> 3. Be released under LLVM's license. >> >> >> Coding standards >> ---------------- >> Libraries in `parallel_utils` will match the LLVM coding standards. For >> existing projects being checked into the subproject as-is, an exception >> will be made during the initial check-in, with the understanding that the >> code will be promptly updated to follow the standards. Therefore, a three >> month grace period will be allowed for new libraries to meet the LLVM >> coding standards. >> >> Additional exceptions to strict adherence to the LLVM coding standards >> may be allowed in certain other cases, but the reasons for such exceptions >> must be discussed and documented on a case-by-case basis. >> >> >> LLVM build and test infrastructure >> ---------------------------------- >> Using the LLVM build and test infrastructure currently means using >> `cmake` for building, `lit` for testing, and `buildbot` for automating >> build and testing. This project will follow the main LLVM project >> conventions here and track them as they evolve. >> >> >> LLVM license >> ------------ >> For simplicity, the `parallel_utils` project will use the normal LLVM >> license. While some runtime libraries use a dual license scheme in LLVM, we >> anticipate the project removing the need for this eventually and in the >> interim follow the simpler but still permissive license. Among other >> things, this makes it straightforward for these libraries to re-use core >> LLVM libraries where appropriate. >> >> On Wed, Apr 27, 2016 at 9:52 AM Jason Henline <jhen at google.com> wrote: >> >> Andrey, >> >> I may have misunderstood Chandler, but I think it would be good to house >> libomptarget together with SE in the new project. I suspect he means to >> leave the option open for the libomptarget developers to choose the best >> option as they see fit. >> >> In terms of organization, I would expect the project initially to contain >> a StreamExecutor directory and a libomptarget directory where each project >> could work separately. >> >> When it comes to the difference in review process between the two >> projects, I don't know the answer. I would expect the new project to handle >> reviews similar to the way they are handled in the LLVM project itself, so >> I don't know if there would be any difference compared to what is happening >> now with libomptarget. >> >> On Wed, Apr 27, 2016 at 9:32 AM C Bergström <openmp-dev at lists.llvm.org> >> wrote: >> >> I respect Hal's more tactful approach and response.. >> >> Let me play devils advocate for a minute >> >> 1) Yet another programming model - Is the advantage spelled out >> somewhere? (I know there are reasons, but I'd like to see a FAQ or >> this clearly documented. Examples pretty please.. More for long term >> than my own selfish benefit) >> 2) Is this an "open standard" - If I wanted to propose a major change >> to SE - how would I or someone else go about it? OpenMP/ACC have more >> or less clearly defined paths for new features.. What's the governing >> policy here.. Bug fixes are easy to deal with, but does Google have >> final say on the roadmap.. >> 3) When the project is created - will it include lots of good tests? >> 4) It's probably used internally @google - who else will be using >> this? Is the target HPC, Android.. etc >> >> Lastly - sorry, but I don't like this kick-the-can approach to what >> should be proper engineering and planning upfront. Can someone @google >> gentleman's promise to actively work on playing nice with other >> projects, specifically OpenMP and Intel. From my perspective nothing >> stops Google from tossing it up on github or google code and letting >> it stay there until all the pieces are in the correct place. Why it >> *must* be an llvm project now doesn't make sense to me. When the shoe >> was on the other foot (OpenMP) there was all sorts of shit and redtape >> Intel (and others) had to jump around to get it included. Google has a >> lot of good karma in the llvm community and maybe that's the >> difference.. >> _______________________________________________ >> Openmp-dev mailing list >> Openmp-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev >> >> >> _______________________________________________ >> Openmp-dev mailing list >> Openmp-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev >> >> >> _______________________________________________ >> >> >> cfe-dev mailing list >> cfe-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev >> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160509/1703213d/attachment.html>
Chandler Carruth via llvm-dev
2016-May-09 17:45 UTC
[llvm-dev] [cfe-dev] [Openmp-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries
On Mon, May 9, 2016 at 11:33 AM Jason Henline via cfe-dev < cfe-dev at lists.llvm.org> wrote:> I talked to Chandler about the name "offload_libs" vs "parallel_libs" and > he said he thinks "offload" is too narrow of a term for the scope he sees > for this subproject. One example he brought up was AVX 512. He thinks that > code explicitly targeting CPU parallelism should also be included in this > project, even though it doesn't fit in the category of "offloading". So > that is an argument in favor of "parallel_libs" instead of "offload_libs". > > Chandler, please correct me if I misrepresented your thoughts on this. >No, you summed it up. And sorry I didn't promptly send those thoughts here, but thanks for relaying our in-person conversation.> > On Thu, Apr 28, 2016 at 9:55 AM Jason Henline <jhen at google.com> wrote: > >> Alexandre, >> >> Thanks for your thoughts on this. I was thinking that each subproject >> library would be responsible for handling its own name and any associated >> branding. For example, when evangelizing for StreamExecutor, I plan to >> refer to it as StreamExecutor, not parallel_libs-StreamExecutor or >> something like that. So I think it is OK for the top-level container >> project to have an undistinguished name. Does that sound reasonable? >> >> On Thu, Apr 28, 2016 at 7:09 AM Alexandre Eichenberger <alexe at us.ibm.com> >> wrote: >> >>> A suggestion on naming: by choosing too generic a name, you don't get >>> any branding. >>> >>> My point for the omp library: if someone talks about GOMP, KMPC / IOMP, >>> or LOMP, they would know we are talking about the GNU, Intel, IBM >>> implementation. I don't think we get that with OMP, which was selected for >>> the OMP runtime. >>> >>> So I would suggest to append "llvm" or "lr" for LLVM runtime, or >>> something distinctive that you like. >>> >>> This make sense because users will have choices of runtimes, gcc linking >>> to GOMP, LLVM runtime, LOMP on POWER; GOMP, LLVM, IOMP on Intel... and we >>> will need to educate our users on which one to use. >>> >>> Alexandre >>> >>> >>> ----------------------------------------------------------------------------------------------------- >>> Alexandre Eichenberger, Master Inventor, Advanced Compiler Technologies >>> - research: compiler optimization (OpenMP, multithreading, SIMD) >>> - info: alexe at us.ibm.com http://www.research.ibm.com/people/a/alexe >>> - phone: 914-945-1812 (work) 914-312-3618 (cell) >>> >>> >>> >>> ----- Original message ----- >>> From: Andrey Bokhanko via cfe-dev <cfe-dev at lists.llvm.org> >>> Sent by: "cfe-dev" <cfe-dev-bounces at lists.llvm.org> >>> To: Jason Henline <jhen at google.com> >>> >>> Cc: llvm-dev <llvm-dev at lists.llvm.org>, clang developer list < >>> cfe-dev at lists.llvm.org>, "openmp-dev at lists.llvm.org" < >>> openmp-dev at lists.llvm.org> >>> Subject: Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM >>> subproject for parallelism runtime and support libraries >>> Date: Thu, Apr 28, 2016 9:23 AM >>> >>> Jason, >>> >>> Very nice write-up. Well done! >>> >>> My two kopecks (before you asked, one kopeck is the smallest item of >>> currency in Russia ;-)): >>> * Bugzilla and mailing list requirements are not covered. Do you want to >>> have a component in Bugzilla for each project? One for all of them? Mailing >>> list -- a separate list for each project? (IMHO, an overkill) One for all >>> of them? >>> * Build system -- I suppose you expect all of the libs to be built >>> separately, without a single unified cmake file. Correct? Also, I expect >>> you want all of the libs to be integrated into LLVM build -- correct? This >>> should be spelled out explicitly. >>> * I don't really like "parallel" in the name. Both SE and libomptarget >>> are libraries that handle offloading, not parallelism. I understand other >>> libraries, to be added in the future, might deal with parallelism, but >>> maybe we need a separate project for them? (Something Chris already >>> hinted.) How about "offloading_lib"? >>> >>> Yours, >>> Andrey >>> >>> >>> On Wed, Apr 27, 2016 at 7:58 PM, Jason Henline via Openmp-dev < >>> openmp-dev at lists.llvm.org> wrote: >>> >>> I've put together a proposed "charter" for this new project, which I am >>> calling parallel_utils (although I'm very open to suggestions for a better >>> name). The text of my charter is below, and I welcome any input on how it >>> can be improved. >>> >>> >>> ====================================================>>> LLVM Parallel Utils Subproject Charter >>> ====================================================>>> >>> ---------------------------------------------- >>> Description >>> ---------------------------------------------- >>> The LLVM open source project will contain a subproject named >>> `parallel_utils` which will host the development of libraries which are >>> aimed at enabling parallelism in code and which are also closely tied to >>> compiler technology. Examples of libraries suitable for hosting within the >>> `parallel_utils` subproject are runtime libraries and parallel math >>> libraries. The initial candidates for inclusion in this subproject are >>> StreamExecutor and libomptarget which would live in the `streamexecutor` >>> and `libomptarget` subdirectories of `parallel_utils`, respectively. >>> >>> The `parallel_utils` project will host a collection of libraries where >>> each library may be dependent on other libraries from the project or may be >>> completely independent of any other libraries in the project. The rationale >>> for hosting independent libraries within the same subproject is that all >>> libraries in the project are providing related functionality that lives at >>> the intersection of parallelism and compiler technology. It is expected >>> that some libraries which initially began as independent will develop >>> dependencies over time either between existing libraries or by extracting >>> common code that can be used by each. One of the purposes of this >>> subproject is to provide a working space where such refactoring and code >>> sharing can take place. >>> >>> Libraries in the `parallel_utils` subproject may also depend on the LLVM >>> core libraries. This will be useful for avoiding duplication of code within >>> the LLVM project for common utilities such as those found in the LLVM >>> support library. >>> >>> >>> ---------------------------------------------- >>> Requirements >>> ---------------------------------------------- >>> Libraries included in the `parallel_utils` subproject must strive to >>> achieve the following requirements: >>> >>> 1. Adhere to the LLVM coding standards. >>> 2. Use the LLVM build and test infrastructure. >>> 3. Be released under LLVM's license. >>> >>> >>> Coding standards >>> ---------------- >>> Libraries in `parallel_utils` will match the LLVM coding standards. For >>> existing projects being checked into the subproject as-is, an exception >>> will be made during the initial check-in, with the understanding that the >>> code will be promptly updated to follow the standards. Therefore, a three >>> month grace period will be allowed for new libraries to meet the LLVM >>> coding standards. >>> >>> Additional exceptions to strict adherence to the LLVM coding standards >>> may be allowed in certain other cases, but the reasons for such exceptions >>> must be discussed and documented on a case-by-case basis. >>> >>> >>> LLVM build and test infrastructure >>> ---------------------------------- >>> Using the LLVM build and test infrastructure currently means using >>> `cmake` for building, `lit` for testing, and `buildbot` for automating >>> build and testing. This project will follow the main LLVM project >>> conventions here and track them as they evolve. >>> >>> >>> LLVM license >>> ------------ >>> For simplicity, the `parallel_utils` project will use the normal LLVM >>> license. While some runtime libraries use a dual license scheme in LLVM, we >>> anticipate the project removing the need for this eventually and in the >>> interim follow the simpler but still permissive license. Among other >>> things, this makes it straightforward for these libraries to re-use core >>> LLVM libraries where appropriate. >>> >>> On Wed, Apr 27, 2016 at 9:52 AM Jason Henline <jhen at google.com> wrote: >>> >>> Andrey, >>> >>> I may have misunderstood Chandler, but I think it would be good to house >>> libomptarget together with SE in the new project. I suspect he means to >>> leave the option open for the libomptarget developers to choose the best >>> option as they see fit. >>> >>> In terms of organization, I would expect the project initially to >>> contain a StreamExecutor directory and a libomptarget directory where each >>> project could work separately. >>> >>> When it comes to the difference in review process between the two >>> projects, I don't know the answer. I would expect the new project to handle >>> reviews similar to the way they are handled in the LLVM project itself, so >>> I don't know if there would be any difference compared to what is happening >>> now with libomptarget. >>> >>> On Wed, Apr 27, 2016 at 9:32 AM C Bergström <openmp-dev at lists.llvm.org> >>> wrote: >>> >>> I respect Hal's more tactful approach and response.. >>> >>> Let me play devils advocate for a minute >>> >>> 1) Yet another programming model - Is the advantage spelled out >>> somewhere? (I know there are reasons, but I'd like to see a FAQ or >>> this clearly documented. Examples pretty please.. More for long term >>> than my own selfish benefit) >>> 2) Is this an "open standard" - If I wanted to propose a major change >>> to SE - how would I or someone else go about it? OpenMP/ACC have more >>> or less clearly defined paths for new features.. What's the governing >>> policy here.. Bug fixes are easy to deal with, but does Google have >>> final say on the roadmap.. >>> 3) When the project is created - will it include lots of good tests? >>> 4) It's probably used internally @google - who else will be using >>> this? Is the target HPC, Android.. etc >>> >>> Lastly - sorry, but I don't like this kick-the-can approach to what >>> should be proper engineering and planning upfront. Can someone @google >>> gentleman's promise to actively work on playing nice with other >>> projects, specifically OpenMP and Intel. From my perspective nothing >>> stops Google from tossing it up on github or google code and letting >>> it stay there until all the pieces are in the correct place. Why it >>> *must* be an llvm project now doesn't make sense to me. When the shoe >>> was on the other foot (OpenMP) there was all sorts of shit and redtape >>> Intel (and others) had to jump around to get it included. Google has a >>> lot of good karma in the llvm community and maybe that's the >>> difference.. >>> _______________________________________________ >>> Openmp-dev mailing list >>> Openmp-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev >>> >>> >>> _______________________________________________ >>> Openmp-dev mailing list >>> Openmp-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev >>> >>> >>> _______________________________________________ >>> >>> >>> cfe-dev mailing list >>> cfe-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev >>> >>> _______________________________________________ > cfe-dev mailing list > cfe-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160509/90bd3ff0/attachment-0001.html>
via llvm-dev
2016-May-09 18:00 UTC
[llvm-dev] [cfe-dev] [Openmp-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries
<html><head></head><body lang="en-US" style="background-color: rgb(255, 255, 255); line-height: initial;"> <div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">I'd still argue that offload is more accurate because </div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><br></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">1) AVX512 on the CPU is (should) likely to be outlined similar to if it was actually offloaded using a discrete device. I imagine there's going to be a lot of similarities in the flow. (latencies and other things will change but not the point) </div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><br></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;">2) simd vec isn't strictly speaking parallelism in my mind. I'll try to avoid the whole is simd a thread or GPU "threads"</span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;"><br></span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;">3) people see offload and think gpu.</span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;"><br></span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;">I'm not strongly against the name parallel, but not sure it really fits what I see falling under this umbrella.</span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;"><br></span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;">Ultimately, will you end up with a lib that exposes an api for both.. execution as well as mechanism to actually "offload" it somewhere.</span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;"><br></span></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><span style="font-size: initial; text-align: initial; line-height: initial;">As a counter example, the kernels you offload to a GPU are actually scalar.. the hw turns them into something parallel.</span></div> <div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><br></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">Fun stuff.. red or blue paint?</div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><br></div> <div style="font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"></div> <table width="100%" style="background-color:white;border-spacing:0px;"> <tbody><tr><td colspan="2" style="font-size: initial; text-align: initial; background-color: rgb(255, 255, 255);"> <div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0in 0in; font-family: Tahoma, 'BB Alpha Sans', 'Slate Pro'; font-size: 10pt;"> <div><b>From: </b>Chandler Carruth via llvm-dev</div><div><b>Sent: </b>Tuesday, May 10, 2016 00:45</div><div><b>To: </b>Jason Henline; Alexandre Eichenberger; andreybokhanko@gmail.com</div><div><b>Reply To: </b>Chandler Carruth</div><div><b>Cc: </b>llvm-dev@lists.llvm.org; cfe-dev@lists.llvm.org; openmp-dev@lists.llvm.org</div><div><b>Subject: </b>Re: [llvm-dev] [cfe-dev] [Openmp-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries</div></div></td></tr></tbody></table><div style="border-style: solid none none; border-top-color: rgb(186, 188, 209); border-top-width: 1pt; font-size: initial; text-align: initial; background-color: rgb(255, 255, 255);"></div><br><div id="_originalContent" style=""><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Mon, May 9, 2016 at 11:33 AM Jason Henline via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I talked to Chandler about the name "offload_libs" vs "parallel_libs" and he said he thinks "offload" is too narrow of a term for the scope he sees for this subproject. One example he brought up was AVX 512. He thinks that code explicitly targeting CPU parallelism should also be included in this project, even though it doesn't fit in the category of "offloading". So that is an argument in favor of "parallel_libs" instead of "offload_libs".<div><br></div><div>Chandler, please correct me if I misrepresented your thoughts on this.</div></div></blockquote><div><br></div><div>No, you summed it up. And sorry I didn't promptly send those thoughts here, but thanks for relaying our in-person conversation.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br><div class="gmail_quote"><div dir="ltr">On Thu, Apr 28, 2016 at 9:55 AM Jason Henline <<a href="mailto:jhen@google.com" target="_blank">jhen@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Alexandre,<div><br></div><div>Thanks for your thoughts on this. I was thinking that each subproject library would be responsible for handling its own name and any associated branding. For example, when evangelizing for StreamExecutor, I plan to refer to it as StreamExecutor, not parallel_libs-StreamExecutor or something like that. So I think it is OK for the top-level container project to have an undistinguished name. Does that sound reasonable?<br><br><div class="gmail_quote"></div></div></div><div dir="ltr"><div><div class="gmail_quote"><div dir="ltr">On Thu, Apr 28, 2016 at 7:09 AM Alexandre Eichenberger <<a href="mailto:alexe@us.ibm.com" target="_blank">alexe@us.ibm.com</a>> wrote:<br></div></div></div></div><div dir="ltr"><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr">A suggestion on naming: by choosing too generic a name, you don't get any branding.</div> <div dir="ltr"> </div> <div dir="ltr">My point for the omp library: if someone talks about GOMP, KMPC / IOMP, or LOMP, they would know we are talking about the GNU, Intel, IBM implementation. I don't think we get that with OMP, which was selected for the OMP runtime.</div> <div dir="ltr"> </div> <div dir="ltr">So I would suggest to append "llvm" or "lr" for LLVM runtime, or something distinctive that you like. </div> <div dir="ltr"> </div> <div dir="ltr">This make sense because users will have choices of runtimes, gcc linking to GOMP, LLVM runtime, LOMP on POWER; GOMP, LLVM, IOMP on Intel... and we will need to educate our users on which one to use.</div></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"> <div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr"><br>Alexandre<br><br>-----------------------------------------------------------------------------------------------------<br><span style="color:#0000cd">Alexandre Eichenberger, Master Inventor, Advanced Compiler Technologies</span><br><span style="color:#0000cd">- research</span>: compiler optimization (OpenMP, multithreading, SIMD)<br><span style="color:#0000cd">- info:</span> <a href="mailto:alexe@us.ibm.com" target="_blank">alexe@us.ibm.com</a> <a href="http://www.research.ibm.com/people/a/alexe" target="_blank">http://www.research.ibm.com/people/a/alexe</a><br><span style="color:#0000cd">- phone</span>: 914-945-1812 (work) 914-312-3618 (cell)</div></div></div></div></div> <div dir="ltr"> </div> <div dir="ltr"> </div> </div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:solid #aaaaaa 2px;margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px">----- Original message -----<br>From: Andrey Bokhanko via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>Sent by: "cfe-dev" <<a href="mailto:cfe-dev-bounces@lists.llvm.org" target="_blank">cfe-dev-bounces@lists.llvm.org</a>><br>To: Jason Henline <<a href="mailto:jhen@google.com" target="_blank">jhen@google.com</a>><br></blockquote></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:solid #aaaaaa 2px;margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px">Cc: llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>, clang developer list <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>>, "<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>" <<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>><br>Subject: Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries<br>Date: Thu, Apr 28, 2016 9:23 AM<br> <div dir="ltr">Jason, <div><br>Very nice write-up. Well done!</div> <div> </div> <div>My two kopecks (before you asked, one kopeck is the smallest item of currency in Russia ;-)):</div> <div>* Bugzilla and mailing list requirements are not covered. Do you want to have a component in Bugzilla for each project? One for all of them? Mailing list -- a separate list for each project? (IMHO, an overkill) One for all of them?</div> <div>* Build system -- I suppose you expect all of the libs to be built separately, without a single unified cmake file. Correct? Also, I expect you want all of the libs to be integrated into LLVM build -- correct? This should be spelled out explicitly.</div> <div>* I don't really like "parallel" in the name. Both SE and libomptarget are libraries that handle offloading, not parallelism. I understand other libraries, to be added in the future, might deal with parallelism, but maybe we need a separate project for them? (Something Chris already hinted.) How about "offloading_lib"?</div> <div> </div> <div>Yours,</div> <div>Andrey</div> <div> </div></div> <div> <div>On Wed, Apr 27, 2016 at 7:58 PM, Jason Henline via Openmp-dev <span dir="ltr"><<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>></span> wrote: <blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I've put together a proposed "charter" for this new project, which I am calling parallel_utils (although I'm very open to suggestions for a better name). The text of my charter is below, and I welcome any input on how it can be improved. <div> </div> <div> </div> <div><div>=====================================================</div> <div>LLVM Parallel Utils Subproject Charter</div> <div>=====================================================</div> <div> </div> <div>----------------------------------------------</div> <div>Description</div> <div>----------------------------------------------</div> <div>The LLVM open source project will contain a subproject named `parallel_utils` which will host the development of libraries which are aimed at enabling parallelism in code and which are also closely tied to compiler technology. Examples of libraries suitable for hosting within the `parallel_utils` subproject are runtime libraries and parallel math libraries. The initial candidates for inclusion in this subproject are StreamExecutor and libomptarget which would live in the `streamexecutor` and `libomptarget` subdirectories of `parallel_utils`, respectively.</div> <div> </div> <div>The `parallel_utils` project will host a collection of libraries where each library may be dependent on other libraries from the project or may be completely independent of any other libraries in the project. The rationale for hosting independent libraries within the same subproject is that all libraries in the project are providing related functionality that lives at the intersection of parallelism and compiler technology. It is expected that some libraries which initially began as independent will develop dependencies over time either between existing libraries or by extracting common code that can be used by each. One of the purposes of this subproject is to provide a working space where such refactoring and code sharing can take place.</div> <div> </div> <div>Libraries in the `parallel_utils` subproject may also depend on the LLVM core libraries. This will be useful for avoiding duplication of code within the LLVM project for common utilities such as those found in the LLVM support library.</div> <div> </div> <div> </div> <div>----------------------------------------------</div> <div>Requirements</div> <div>----------------------------------------------</div> <div>Libraries included in the `parallel_utils` subproject must strive to achieve the following requirements:</div> <div> </div> <div>1. Adhere to the LLVM coding standards.</div> <div>2. Use the LLVM build and test infrastructure.</div> <div>3. Be released under LLVM's license.</div> <div> </div> <div> </div> <div>Coding standards</div> <div>----------------</div> <div>Libraries in `parallel_utils` will match the LLVM coding standards. For existing projects being checked into the subproject as-is, an exception will be made during the initial check-in, with the understanding that the code will be promptly updated to follow the standards. Therefore, a three month grace period will be allowed for new libraries to meet the LLVM coding standards.</div> <div> </div> <div>Additional exceptions to strict adherence to the LLVM coding standards may be allowed in certain other cases, but the reasons for such exceptions must be discussed and documented on a case-by-case basis.</div> <div> </div> <div> </div> <div>LLVM build and test infrastructure</div> <div>----------------------------------</div> <div>Using the LLVM build and test infrastructure currently means using `cmake` for building, `lit` for testing, and `buildbot` for automating build and testing. This project will follow the main LLVM project conventions here and track them as they evolve.</div> <div> </div> <div> </div> <div>LLVM license</div> <div>------------</div> <div>For simplicity, the `parallel_utils` project will use the normal LLVM license. While some runtime libraries use a dual license scheme in LLVM, we anticipate the project removing the need for this eventually and in the interim follow the simpler but still permissive license. Among other things, this makes it straightforward for these libraries to re-use core LLVM libraries where appropriate.</div></div> <div><div> <div><div dir="ltr">On Wed, Apr 27, 2016 at 9:52 AM Jason Henline <<a href="mailto:jhen@google.com" target="_blank">jhen@google.com</a>> wrote:</div> <blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Andrey, <div> </div> <div>I may have misunderstood Chandler, but I think it would be good to house libomptarget together with SE in the new project. I suspect he means to leave the option open for the libomptarget developers to choose the best option as they see fit.</div> <div> </div> <div>In terms of organization, I would expect the project initially to contain a StreamExecutor directory and a libomptarget directory where each project could work separately.</div> <div> </div> <div>When it comes to the difference in review process between the two projects, I don't know the answer. I would expect the new project to handle reviews similar to the way they are handled in the LLVM project itself, so I don't know if there would be any difference compared to what is happening now with libomptarget.</div></div> <div><div dir="ltr">On Wed, Apr 27, 2016 at 9:32 AM C Bergström <<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>> wrote:</div> <blockquote style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I respect Hal's more tactful approach and response..<br><br>Let me play devils advocate for a minute<br><br>1) Yet another programming model - Is the advantage spelled out<br>somewhere? (I know there are reasons, but I'd like to see a FAQ or<br>this clearly documented. Examples pretty please.. More for long term<br>than my own selfish benefit)<br>2) Is this an "open standard" - If I wanted to propose a major change<br>to SE - how would I or someone else go about it? OpenMP/ACC have more<br>or less clearly defined paths for new features.. What's the governing<br>policy here.. Bug fixes are easy to deal with, but does Google have<br>final say on the roadmap..<br>3) When the project is created - will it include lots of good tests?<br>4) It's probably used internally @google - who else will be using<br>this? Is the target HPC, Android.. etc<br><br>Lastly - sorry, but I don't like this kick-the-can approach to what<br>should be proper engineering and planning upfront. Can someone @google<br>gentleman's promise to actively work on playing nice with other<br>projects, specifically OpenMP and Intel. From my perspective nothing<br>stops Google from tossing it up on github or google code and letting<br>it stay there until all the pieces are in the correct place. Why it<br>*must* be an llvm project now doesn't make sense to me. When the shoe<br>was on the other foot (OpenMP) there was all sorts of shit and redtape<br>Intel (and others) had to jump around to get it included. Google has a<br>lot of good karma in the llvm community and maybe that's the<br>difference..<br>_______________________________________________<br>Openmp-dev mailing list<br><a href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a></blockquote></div></blockquote></div></div></div></div><br>_______________________________________________<br>Openmp-dev mailing list<br><a href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a><br> </blockquote></div></div> </blockquote></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:solid #aaaaaa 2px;margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px"><div><font face="Default Monospace,Courier New,Courier,monospace" size="2">_______________________________________________</font></div></blockquote></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:solid #aaaaaa 2px;margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px"><div><font face="Default Monospace,Courier New,Courier,monospace" size="2"><br>cfe-dev mailing list<br><a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a></font></div></blockquote></div></blockquote></div></div></div></blockquote></div> _______________________________________________<br> cfe-dev mailing list<br> <a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br> </blockquote></div></div> <br><!--end of _originalContent --></div></body></html>
Andrey Bokhanko via llvm-dev
2016-May-09 18:03 UTC
[llvm-dev] [cfe-dev] [Openmp-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries
Thanks Jason (and Chandler) to tell Chandler's opinion, though it still doesn't answer my original question:> Both SE and libomptarget are libraries that handle offloading, notparallelism. I understand other libraries, to be added in the future, might deal with parallelism, but maybe we need a separate project for them? (Something Chris already hinted.) The example simply adds confusion> One example he brought up was AVX 512. He thinks that code explicitlytargeting CPU parallelism should also be included in this project, even though it doesn't fit in the category of "offloading". Do we want to add a vectorizer to this project as well? Yours, Andrey On Mon, May 9, 2016 at 8:32 PM, Jason Henline <jhen at google.com> wrote:> I talked to Chandler about the name "offload_libs" vs "parallel_libs" and > he said he thinks "offload" is too narrow of a term for the scope he sees > for this subproject. One example he brought up was AVX 512. He thinks that > code explicitly targeting CPU parallelism should also be included in this > project, even though it doesn't fit in the category of "offloading". So > that is an argument in favor of "parallel_libs" instead of "offload_libs". > > Chandler, please correct me if I misrepresented your thoughts on this. > > On Thu, Apr 28, 2016 at 9:55 AM Jason Henline <jhen at google.com> wrote: > >> Alexandre, >> >> Thanks for your thoughts on this. I was thinking that each subproject >> library would be responsible for handling its own name and any associated >> branding. For example, when evangelizing for StreamExecutor, I plan to >> refer to it as StreamExecutor, not parallel_libs-StreamExecutor or >> something like that. So I think it is OK for the top-level container >> project to have an undistinguished name. Does that sound reasonable? >> >> On Thu, Apr 28, 2016 at 7:09 AM Alexandre Eichenberger <alexe at us.ibm.com> >> wrote: >> >>> A suggestion on naming: by choosing too generic a name, you don't get >>> any branding. >>> >>> My point for the omp library: if someone talks about GOMP, KMPC / IOMP, >>> or LOMP, they would know we are talking about the GNU, Intel, IBM >>> implementation. I don't think we get that with OMP, which was selected for >>> the OMP runtime. >>> >>> So I would suggest to append "llvm" or "lr" for LLVM runtime, or >>> something distinctive that you like. >>> >>> This make sense because users will have choices of runtimes, gcc linking >>> to GOMP, LLVM runtime, LOMP on POWER; GOMP, LLVM, IOMP on Intel... and we >>> will need to educate our users on which one to use. >>> >>> Alexandre >>> >>> >>> ----------------------------------------------------------------------------------------------------- >>> Alexandre Eichenberger, Master Inventor, Advanced Compiler Technologies >>> - research: compiler optimization (OpenMP, multithreading, SIMD) >>> - info: alexe at us.ibm.com http://www.research.ibm.com/people/a/alexe >>> - phone: 914-945-1812 (work) 914-312-3618 (cell) >>> >>> >>> >>> ----- Original message ----- >>> From: Andrey Bokhanko via cfe-dev <cfe-dev at lists.llvm.org> >>> Sent by: "cfe-dev" <cfe-dev-bounces at lists.llvm.org> >>> To: Jason Henline <jhen at google.com> >>> >>> Cc: llvm-dev <llvm-dev at lists.llvm.org>, clang developer list < >>> cfe-dev at lists.llvm.org>, "openmp-dev at lists.llvm.org" < >>> openmp-dev at lists.llvm.org> >>> Subject: Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM >>> subproject for parallelism runtime and support libraries >>> Date: Thu, Apr 28, 2016 9:23 AM >>> >>> Jason, >>> >>> Very nice write-up. Well done! >>> >>> My two kopecks (before you asked, one kopeck is the smallest item of >>> currency in Russia ;-)): >>> * Bugzilla and mailing list requirements are not covered. Do you want to >>> have a component in Bugzilla for each project? One for all of them? Mailing >>> list -- a separate list for each project? (IMHO, an overkill) One for all >>> of them? >>> * Build system -- I suppose you expect all of the libs to be built >>> separately, without a single unified cmake file. Correct? Also, I expect >>> you want all of the libs to be integrated into LLVM build -- correct? This >>> should be spelled out explicitly. >>> * I don't really like "parallel" in the name. Both SE and libomptarget >>> are libraries that handle offloading, not parallelism. I understand other >>> libraries, to be added in the future, might deal with parallelism, but >>> maybe we need a separate project for them? (Something Chris already >>> hinted.) How about "offloading_lib"? >>> >>> Yours, >>> Andrey >>> >>> >>> On Wed, Apr 27, 2016 at 7:58 PM, Jason Henline via Openmp-dev < >>> openmp-dev at lists.llvm.org> wrote: >>> >>> I've put together a proposed "charter" for this new project, which I am >>> calling parallel_utils (although I'm very open to suggestions for a better >>> name). The text of my charter is below, and I welcome any input on how it >>> can be improved. >>> >>> >>> ====================================================>>> LLVM Parallel Utils Subproject Charter >>> ====================================================>>> >>> ---------------------------------------------- >>> Description >>> ---------------------------------------------- >>> The LLVM open source project will contain a subproject named >>> `parallel_utils` which will host the development of libraries which are >>> aimed at enabling parallelism in code and which are also closely tied to >>> compiler technology. Examples of libraries suitable for hosting within the >>> `parallel_utils` subproject are runtime libraries and parallel math >>> libraries. The initial candidates for inclusion in this subproject are >>> StreamExecutor and libomptarget which would live in the `streamexecutor` >>> and `libomptarget` subdirectories of `parallel_utils`, respectively. >>> >>> The `parallel_utils` project will host a collection of libraries where >>> each library may be dependent on other libraries from the project or may be >>> completely independent of any other libraries in the project. The rationale >>> for hosting independent libraries within the same subproject is that all >>> libraries in the project are providing related functionality that lives at >>> the intersection of parallelism and compiler technology. It is expected >>> that some libraries which initially began as independent will develop >>> dependencies over time either between existing libraries or by extracting >>> common code that can be used by each. One of the purposes of this >>> subproject is to provide a working space where such refactoring and code >>> sharing can take place. >>> >>> Libraries in the `parallel_utils` subproject may also depend on the LLVM >>> core libraries. This will be useful for avoiding duplication of code within >>> the LLVM project for common utilities such as those found in the LLVM >>> support library. >>> >>> >>> ---------------------------------------------- >>> Requirements >>> ---------------------------------------------- >>> Libraries included in the `parallel_utils` subproject must strive to >>> achieve the following requirements: >>> >>> 1. Adhere to the LLVM coding standards. >>> 2. Use the LLVM build and test infrastructure. >>> 3. Be released under LLVM's license. >>> >>> >>> Coding standards >>> ---------------- >>> Libraries in `parallel_utils` will match the LLVM coding standards. For >>> existing projects being checked into the subproject as-is, an exception >>> will be made during the initial check-in, with the understanding that the >>> code will be promptly updated to follow the standards. Therefore, a three >>> month grace period will be allowed for new libraries to meet the LLVM >>> coding standards. >>> >>> Additional exceptions to strict adherence to the LLVM coding standards >>> may be allowed in certain other cases, but the reasons for such exceptions >>> must be discussed and documented on a case-by-case basis. >>> >>> >>> LLVM build and test infrastructure >>> ---------------------------------- >>> Using the LLVM build and test infrastructure currently means using >>> `cmake` for building, `lit` for testing, and `buildbot` for automating >>> build and testing. This project will follow the main LLVM project >>> conventions here and track them as they evolve. >>> >>> >>> LLVM license >>> ------------ >>> For simplicity, the `parallel_utils` project will use the normal LLVM >>> license. While some runtime libraries use a dual license scheme in LLVM, we >>> anticipate the project removing the need for this eventually and in the >>> interim follow the simpler but still permissive license. Among other >>> things, this makes it straightforward for these libraries to re-use core >>> LLVM libraries where appropriate. >>> >>> On Wed, Apr 27, 2016 at 9:52 AM Jason Henline <jhen at google.com> wrote: >>> >>> Andrey, >>> >>> I may have misunderstood Chandler, but I think it would be good to house >>> libomptarget together with SE in the new project. I suspect he means to >>> leave the option open for the libomptarget developers to choose the best >>> option as they see fit. >>> >>> In terms of organization, I would expect the project initially to >>> contain a StreamExecutor directory and a libomptarget directory where each >>> project could work separately. >>> >>> When it comes to the difference in review process between the two >>> projects, I don't know the answer. I would expect the new project to handle >>> reviews similar to the way they are handled in the LLVM project itself, so >>> I don't know if there would be any difference compared to what is happening >>> now with libomptarget. >>> >>> On Wed, Apr 27, 2016 at 9:32 AM C Bergström <openmp-dev at lists.llvm.org> >>> wrote: >>> >>> I respect Hal's more tactful approach and response.. >>> >>> Let me play devils advocate for a minute >>> >>> 1) Yet another programming model - Is the advantage spelled out >>> somewhere? (I know there are reasons, but I'd like to see a FAQ or >>> this clearly documented. Examples pretty please.. More for long term >>> than my own selfish benefit) >>> 2) Is this an "open standard" - If I wanted to propose a major change >>> to SE - how would I or someone else go about it? OpenMP/ACC have more >>> or less clearly defined paths for new features.. What's the governing >>> policy here.. Bug fixes are easy to deal with, but does Google have >>> final say on the roadmap.. >>> 3) When the project is created - will it include lots of good tests? >>> 4) It's probably used internally @google - who else will be using >>> this? Is the target HPC, Android.. etc >>> >>> Lastly - sorry, but I don't like this kick-the-can approach to what >>> should be proper engineering and planning upfront. Can someone @google >>> gentleman's promise to actively work on playing nice with other >>> projects, specifically OpenMP and Intel. From my perspective nothing >>> stops Google from tossing it up on github or google code and letting >>> it stay there until all the pieces are in the correct place. Why it >>> *must* be an llvm project now doesn't make sense to me. When the shoe >>> was on the other foot (OpenMP) there was all sorts of shit and redtape >>> Intel (and others) had to jump around to get it included. Google has a >>> lot of good karma in the llvm community and maybe that's the >>> difference.. >>> _______________________________________________ >>> Openmp-dev mailing list >>> Openmp-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev >>> >>> >>> _______________________________________________ >>> Openmp-dev mailing list >>> Openmp-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev >>> >>> >>> _______________________________________________ >>> >>> >>> cfe-dev mailing list >>> cfe-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev >>> >>>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160509/a6f7f7e5/attachment.html>
Chandler Carruth via llvm-dev
2016-May-09 18:07 UTC
[llvm-dev] [cfe-dev] [Openmp-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries
On Mon, May 9, 2016 at 12:04 PM Andrey Bokhanko via cfe-dev < cfe-dev at lists.llvm.org> wrote:> Thanks Jason (and Chandler) to tell Chandler's opinion, though it still > doesn't answer my original question: > > > > Both SE and libomptarget are libraries that handle offloading, not > parallelism. I understand other libraries, to be added in the future, might > deal with parallelism, but maybe we need a separate project for them? > (Something Chris already hinted.) > > The example simply adds confusion > > > > One example he brought up was AVX 512. He thinks that code explicitly > targeting CPU parallelism should also be included in this project, even > though it doesn't fit in the category of "offloading". > > Do we want to add a vectorizer to this project as well? >I would not expect a vectorizer to really fit here, no. But I would like to have this be a viable home for support libraries used by vectorizers if that proves useful. I thought this was usefully captured by the '_rt' or '_libs' suffix. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160509/4f152675/attachment.html>
via llvm-dev
2016-May-09 18:07 UTC
[llvm-dev] [cfe-dev] [Openmp-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries
<html><head></head><body lang="en-US" style="background-color: rgb(255, 255, 255); line-height: initial;"> <div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">The api for Xeon PHI, which needs vectorization and that of a GPU, which is scalar can be unified at a higher level. This is target specific and as long as the layering is done correctly it won't or shouldn't be a problem </div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><br></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">Again 3 layers</div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">Programming model specific </div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">Common layer</div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">Device and target specific stuff below that.</div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><br></div><div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);">As long as programming models don't be naughty and call target specific stuff It's all cool.</div> <div style="width: 100%; font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"><br style="display:initial"></div> <div style="font-size: initial; font-family: Calibri, 'Slate Pro', sans-serif, sans-serif; color: rgb(31, 73, 125); text-align: initial; background-color: rgb(255, 255, 255);"></div> <table width="100%" style="background-color:white;border-spacing:0px;"> <tbody><tr><td colspan="2" style="font-size: initial; text-align: initial; background-color: rgb(255, 255, 255);"> <div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0in 0in; font-family: Tahoma, 'BB Alpha Sans', 'Slate Pro'; font-size: 10pt;"> <div><b>From: </b>Andrey Bokhanko via cfe-dev</div><div><b>Sent: </b>Tuesday, May 10, 2016 01:03</div><div><b>To: </b>Jason Henline</div><div><b>Reply To: </b>Andrey Bokhanko</div><div><b>Cc: </b>llvm-dev; cfe-dev; openmp-dev@lists.llvm.org</div><div><b>Subject: </b>Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries</div></div></td></tr></tbody></table><div style="border-style: solid none none; border-top-color: rgb(186, 188, 209); border-top-width: 1pt; font-size: initial; text-align: initial; background-color: rgb(255, 255, 255);"></div><br><div id="_originalContent" style=""><div dir="ltr"><div><div><div><div>Thanks Jason (and Chandler) to tell Chandler's opinion, though it still doesn't answer my original question:<br><br>> Both SE and libomptarget are libraries that handle offloading, not parallelism. I understand other libraries, to be added in the future, might deal with parallelism, but maybe we need a separate project for them? (Something Chris already hinted.)<br><br></div>The example simply adds confusion<br><br>> One example he brought up was AVX 512. He thinks that code explicitly targeting CPU parallelism should also be included in this project, even though it doesn't fit in the category of "offloading".<br><br></div>Do we want to add a vectorizer to this project as well?<br><br></div>Yours,<br></div>Andrey<br><div><div><div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, May 9, 2016 at 8:32 PM, Jason Henline <span dir="ltr"><<a href="mailto:jhen@google.com" target="_blank">jhen@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I talked to Chandler about the name "offload_libs" vs "parallel_libs" and he said he thinks "offload" is too narrow of a term for the scope he sees for this subproject. One example he brought up was AVX 512. He thinks that code explicitly targeting CPU parallelism should also be included in this project, even though it doesn't fit in the category of "offloading". So that is an argument in favor of "parallel_libs" instead of "offload_libs".<div><br></div><div>Chandler, please correct me if I misrepresented your thoughts on this.</div></div><div><div><br><div class="gmail_quote"><div dir="ltr">On Thu, Apr 28, 2016 at 9:55 AM Jason Henline <<a href="mailto:jhen@google.com" target="_blank">jhen@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Alexandre,<div><br></div><div>Thanks for your thoughts on this. I was thinking that each subproject library would be responsible for handling its own name and any associated branding. For example, when evangelizing for StreamExecutor, I plan to refer to it as StreamExecutor, not parallel_libs-StreamExecutor or something like that. So I think it is OK for the top-level container project to have an undistinguished name. Does that sound reasonable?<br><br><div class="gmail_quote"></div></div></div><div dir="ltr"><div><div class="gmail_quote"><div dir="ltr">On Thu, Apr 28, 2016 at 7:09 AM Alexandre Eichenberger <<a href="mailto:alexe@us.ibm.com" target="_blank">alexe@us.ibm.com</a>> wrote:<br></div></div></div></div><div dir="ltr"><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr">A suggestion on naming: by choosing too generic a name, you don't get any branding.</div> <div dir="ltr"> </div> <div dir="ltr">My point for the omp library: if someone talks about GOMP, KMPC / IOMP, or LOMP, they would know we are talking about the GNU, Intel, IBM implementation. I don't think we get that with OMP, which was selected for the OMP runtime.</div> <div dir="ltr"> </div> <div dir="ltr">So I would suggest to append "llvm" or "lr" for LLVM runtime, or something distinctive that you like. </div> <div dir="ltr"> </div> <div dir="ltr">This make sense because users will have choices of runtimes, gcc linking to GOMP, LLVM runtime, LOMP on POWER; GOMP, LLVM, IOMP on Intel... and we will need to educate our users on which one to use.</div></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"> <div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><div dir="ltr"><br>Alexandre<br><br>-----------------------------------------------------------------------------------------------------<br><span style="color:rgb(0,0,205)">Alexandre Eichenberger, Master Inventor, Advanced Compiler Technologies</span><br><span style="color:rgb(0,0,205)">- research</span>: compiler optimization (OpenMP, multithreading, SIMD)<br><span style="color:rgb(0,0,205)">- info:</span> <a href="mailto:alexe@us.ibm.com" target="_blank">alexe@us.ibm.com</a> <a href="http://www.research.ibm.com/people/a/alexe" target="_blank">http://www.research.ibm.com/people/a/alexe</a><br><span style="color:rgb(0,0,205)">- phone</span>: 914-945-1812 (work) 914-312-3618 (cell)</div></div></div></div></div> <div dir="ltr"> </div> <div dir="ltr"> </div> </div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:2px solid rgb(170,170,170);margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px">----- Original message -----<br>From: Andrey Bokhanko via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>Sent by: "cfe-dev" <<a href="mailto:cfe-dev-bounces@lists.llvm.org" target="_blank">cfe-dev-bounces@lists.llvm.org</a>><br>To: Jason Henline <<a href="mailto:jhen@google.com" target="_blank">jhen@google.com</a>><br></blockquote></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:2px solid rgb(170,170,170);margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px">Cc: llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>, clang developer list <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>>, "<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>" <<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>><br>Subject: Re: [cfe-dev] [Openmp-dev] [llvm-dev] RFC: Proposing an LLVM subproject for parallelism runtime and support libraries<br>Date: Thu, Apr 28, 2016 9:23 AM<br> <div dir="ltr">Jason, <div><br>Very nice write-up. Well done!</div> <div> </div> <div>My two kopecks (before you asked, one kopeck is the smallest item of currency in Russia ;-)):</div> <div>* Bugzilla and mailing list requirements are not covered. Do you want to have a component in Bugzilla for each project? One for all of them? Mailing list -- a separate list for each project? (IMHO, an overkill) One for all of them?</div> <div>* Build system -- I suppose you expect all of the libs to be built separately, without a single unified cmake file. Correct? Also, I expect you want all of the libs to be integrated into LLVM build -- correct? This should be spelled out explicitly.</div> <div>* I don't really like "parallel" in the name. Both SE and libomptarget are libraries that handle offloading, not parallelism. I understand other libraries, to be added in the future, might deal with parallelism, but maybe we need a separate project for them? (Something Chris already hinted.) How about "offloading_lib"?</div> <div> </div> <div>Yours,</div> <div>Andrey</div> <div> </div></div> <div> <div>On Wed, Apr 27, 2016 at 7:58 PM, Jason Henline via Openmp-dev <span dir="ltr"><<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>></span> wrote: <blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I've put together a proposed "charter" for this new project, which I am calling parallel_utils (although I'm very open to suggestions for a better name). The text of my charter is below, and I welcome any input on how it can be improved. <div> </div> <div> </div> <div><div>=====================================================</div> <div>LLVM Parallel Utils Subproject Charter</div> <div>=====================================================</div> <div> </div> <div>----------------------------------------------</div> <div>Description</div> <div>----------------------------------------------</div> <div>The LLVM open source project will contain a subproject named `parallel_utils` which will host the development of libraries which are aimed at enabling parallelism in code and which are also closely tied to compiler technology. Examples of libraries suitable for hosting within the `parallel_utils` subproject are runtime libraries and parallel math libraries. The initial candidates for inclusion in this subproject are StreamExecutor and libomptarget which would live in the `streamexecutor` and `libomptarget` subdirectories of `parallel_utils`, respectively.</div> <div> </div> <div>The `parallel_utils` project will host a collection of libraries where each library may be dependent on other libraries from the project or may be completely independent of any other libraries in the project. The rationale for hosting independent libraries within the same subproject is that all libraries in the project are providing related functionality that lives at the intersection of parallelism and compiler technology. It is expected that some libraries which initially began as independent will develop dependencies over time either between existing libraries or by extracting common code that can be used by each. One of the purposes of this subproject is to provide a working space where such refactoring and code sharing can take place.</div> <div> </div> <div>Libraries in the `parallel_utils` subproject may also depend on the LLVM core libraries. This will be useful for avoiding duplication of code within the LLVM project for common utilities such as those found in the LLVM support library.</div> <div> </div> <div> </div> <div>----------------------------------------------</div> <div>Requirements</div> <div>----------------------------------------------</div> <div>Libraries included in the `parallel_utils` subproject must strive to achieve the following requirements:</div> <div> </div> <div>1. Adhere to the LLVM coding standards.</div> <div>2. Use the LLVM build and test infrastructure.</div> <div>3. Be released under LLVM's license.</div> <div> </div> <div> </div> <div>Coding standards</div> <div>----------------</div> <div>Libraries in `parallel_utils` will match the LLVM coding standards. For existing projects being checked into the subproject as-is, an exception will be made during the initial check-in, with the understanding that the code will be promptly updated to follow the standards. Therefore, a three month grace period will be allowed for new libraries to meet the LLVM coding standards.</div> <div> </div> <div>Additional exceptions to strict adherence to the LLVM coding standards may be allowed in certain other cases, but the reasons for such exceptions must be discussed and documented on a case-by-case basis.</div> <div> </div> <div> </div> <div>LLVM build and test infrastructure</div> <div>----------------------------------</div> <div>Using the LLVM build and test infrastructure currently means using `cmake` for building, `lit` for testing, and `buildbot` for automating build and testing. This project will follow the main LLVM project conventions here and track them as they evolve.</div> <div> </div> <div> </div> <div>LLVM license</div> <div>------------</div> <div>For simplicity, the `parallel_utils` project will use the normal LLVM license. While some runtime libraries use a dual license scheme in LLVM, we anticipate the project removing the need for this eventually and in the interim follow the simpler but still permissive license. Among other things, this makes it straightforward for these libraries to re-use core LLVM libraries where appropriate.</div></div> <div><div> <div><div dir="ltr">On Wed, Apr 27, 2016 at 9:52 AM Jason Henline <<a href="mailto:jhen@google.com" target="_blank">jhen@google.com</a>> wrote:</div> <blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Andrey, <div> </div> <div>I may have misunderstood Chandler, but I think it would be good to house libomptarget together with SE in the new project. I suspect he means to leave the option open for the libomptarget developers to choose the best option as they see fit.</div> <div> </div> <div>In terms of organization, I would expect the project initially to contain a StreamExecutor directory and a libomptarget directory where each project could work separately.</div> <div> </div> <div>When it comes to the difference in review process between the two projects, I don't know the answer. I would expect the new project to handle reviews similar to the way they are handled in the LLVM project itself, so I don't know if there would be any difference compared to what is happening now with libomptarget.</div></div> <div><div dir="ltr">On Wed, Apr 27, 2016 at 9:32 AM C Bergström <<a href="mailto:openmp-dev@lists.llvm.org" target="_blank">openmp-dev@lists.llvm.org</a>> wrote:</div> <blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I respect Hal's more tactful approach and response..<br><br>Let me play devils advocate for a minute<br><br>1) Yet another programming model - Is the advantage spelled out<br>somewhere? (I know there are reasons, but I'd like to see a FAQ or<br>this clearly documented. Examples pretty please.. More for long term<br>than my own selfish benefit)<br>2) Is this an "open standard" - If I wanted to propose a major change<br>to SE - how would I or someone else go about it? OpenMP/ACC have more<br>or less clearly defined paths for new features.. What's the governing<br>policy here.. Bug fixes are easy to deal with, but does Google have<br>final say on the roadmap..<br>3) When the project is created - will it include lots of good tests?<br>4) It's probably used internally @google - who else will be using<br>this? Is the target HPC, Android.. etc<br><br>Lastly - sorry, but I don't like this kick-the-can approach to what<br>should be proper engineering and planning upfront. Can someone @google<br>gentleman's promise to actively work on playing nice with other<br>projects, specifically OpenMP and Intel. From my perspective nothing<br>stops Google from tossing it up on github or google code and letting<br>it stay there until all the pieces are in the correct place. Why it<br>*must* be an llvm project now doesn't make sense to me. When the shoe<br>was on the other foot (OpenMP) there was all sorts of shit and redtape<br>Intel (and others) had to jump around to get it included. Google has a<br>lot of good karma in the llvm community and maybe that's the<br>difference..<br>_______________________________________________<br>Openmp-dev mailing list<br><a href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a></blockquote></div></blockquote></div></div></div></div><br>_______________________________________________<br>Openmp-dev mailing list<br><a href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a><br> </blockquote></div></div> </blockquote></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:2px solid rgb(170,170,170);margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px"><div><font face="Default Monospace,Courier New,Courier,monospace" size="2">_______________________________________________</font></div></blockquote></div><div dir="ltr" style="font-family:Arial;font-size:10.5pt"><blockquote dir="ltr" style="border-left:2px solid rgb(170,170,170);margin-left:5px;padding-left:5px;direction:ltr;margin-right:0px"><div><font face="Default Monospace,Courier New,Courier,monospace" size="2"><br>cfe-dev mailing list<br><a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a></font></div></blockquote></div></blockquote></div></div></div></blockquote></div> </div></div></blockquote></div><br></div></div></div></div></div></div></div> <br><!--end of _originalContent --></div></body></html>