llvm dev - May 2018 - A Short Policy Proposal Regarding Host Compilers

I'd be opposed to 6/5, given where it would leave us.
It's simply hard to see a compelling reason to leave things that long.

In particular, given this is about what it takes to produce a binary
release of clang/llvm from trunk (and not what it takes to use one), i'd
like to see some evidence/argument that using 3/1.5 would actually have a
material affect on the number of contributions, etc.
(I have doubts it would have any affect on the abliity of new developers to
start contributing, etc).
All of the clang/llvm based tools i have around (cquery, rtags, you name
it) all download and ship binary releases of clang/llvm (and FWIW, they
ship and use 1-2 year old releases).
It's also unclear to me it makes sense to try to make sure any user can
compile the latest version - for example, researchers using it almost never
keep up with trunk, even with our current policy that supports things for
longer.  They stick with the version that existed when they started.
So it's unclear that we are doing a thing users actually want in practice
anyway :)

Finally,  given the rate of support for newer C++ standards in LLVM/GCC
seems to be accelerating and not slowing down (AFAICT), keeping a time
period this long will just put you farther and farther behind over time.

It may be better to simply express it in terms of releases, and say "we
support the past 2/3 major gcc releases, the past 2/3 major clang releases,
and the past 2 major msvc releases"





On Fri, May 11, 2018 at 8:58 AM, Andrew Kelley via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I second this proposal, and I make a motion to lengthen 3/1.5 to 6/5.
>
> On Fri, May 11, 2018 at 9:37 AM, Keane, Erich via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi All-
>> As we all know, the C++14 discussion is flaring up again.  Chandler
>> brought up that he would like a concrete plan to switch.  In my
opinion,
>> this is insufficient, as it will result in us simply having this
discussion
>> AGAIN next release.  Instead, I would prefer us to have a concrete
Policy
>> on our host compilers.  That way, changes like this are unsurprising to
our
>> users, and advance our codebase sufficiently.  I believe the arguments
>> for/against upgrading have been made repeatedly, so I won't repeat
them
>> here.  My proposal is thus:
>>
>> Starting with the Clang 7.0 release, we will officially support any
major
>> release of our host compilers (MSVC, GCC, Clang, ?ICC?) released in the
>> past 3* years from our previous branch date to give trunk-developers
time
>> to transition (so for 7.0, 3 years before January 3, 2018).  This will
be
>> enforced via the CMake CheckCompilerVersion script (ala
>> https://reviews.llvm.org/D46723).  ADDITIONALLY, a CMake warning will
be
>> issued for any major release less than 1.5* years old to give our users
>> sufficient time to transition/upgrade their compilers.  Finally, our
>> dependent C++ version will be the best released standard officially
>> supported by the collection of compilers (for example, we'd support
-C++20
>> if all compilers had std=c++20 or eqiv, but NOT std=c++2a).
>>
>> The 3-years/1.5 years would result in our minimum GCC/Clang becoming:
>> GCC5.1/Clang3.6.  We would WARN on anything older than GCC7.1/Clang3.8
>>
>> /End Proposal
>>
>>
>> *: To Be Bikeshed
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180511/d26368a6/attachment.html>

Hi,

@Erich, thanks for putting this together :).
> On May 11, 2018, at 9:54 AM, Daniel Berlin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> I'd be opposed to 6/5, given where it would leave us.
> It's simply hard to see a compelling reason to leave things that long.
> 
> In particular, given this is about what it takes to produce a binary
release of clang/llvm from trunk (and not what it takes to use one), i'd
like to see some evidence/argument that using 3/1.5 would actually have a
material affect on the number of contributions, etc.
> (I have doubts it would have any affect on the abliity of new developers to
start contributing, etc).
+ 1.
> All of the clang/llvm based tools i have around (cquery, rtags, you name
it) all download and ship binary releases of clang/llvm (and FWIW, they ship and
use 1-2 year old releases).
> It's also unclear to me it makes sense to try to make sure any user can
compile the latest version - for example, researchers using it almost never keep
up with trunk, even with our current policy that supports things for longer. 
They stick with the version that existed when they started.
> 
> So it's unclear that we are doing a thing users actually want in
practice anyway :)
Yep, I have the same doubts. Anecdotally, I've got a few hobby projects I
haven't rebased since.. 3.4? I don't have a list on hand, but I've
definitely seen papers from groups that do the same thing.

> Finally,  given the rate of support for newer C++ standards in LLVM/GCC
seems to be accelerating and not slowing down (AFAICT), keeping a time period
this long will just put you farther and farther behind over time.
> 
> It may be better to simply express it in terms of releases, and say
"we support the past 2/3 major gcc releases, the past 2/3 major clang
releases, and the past 2 major msvc releases"
I'd prefer this to imposing a fixed wait period of 3 years. We could add a
deprecation warning for compiler version X when (X+1) is released, and switch to
(X+1) when (X+2) is out. I can see this breaking down if some LTS distro
continues to ship version X, but that's in issue in the 3/1.5 scheme as
well, and we can make specific exceptions as needed.

I see the 3/1.5 scheme as basically a more conservative version of this, so
I'd be OK with that too.

vedant
> 
> 
> 
> On Fri, May 11, 2018 at 8:58 AM, Andrew Kelley via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> I second this proposal, and I make a motion to lengthen 3/1.5 to 6/5.
> 
> On Fri, May 11, 2018 at 9:37 AM, Keane, Erich via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> Hi All-
> As we all know, the C++14 discussion is flaring up again.  Chandler brought
up that he would like a concrete plan to switch.  In my opinion, this is
insufficient, as it will result in us simply having this discussion AGAIN next
release.  Instead, I would prefer us to have a concrete Policy on our host
compilers.  That way, changes like this are unsurprising to our users, and
advance our codebase sufficiently.  I believe the arguments for/against
upgrading have been made repeatedly, so I won't repeat them here.  My
proposal is thus:
> 
> Starting with the Clang 7.0 release, we will officially support any major
release of our host compilers (MSVC, GCC, Clang, ?ICC?) released in the past 3*
years from our previous branch date to give trunk-developers time to transition
(so for 7.0, 3 years before January 3, 2018).  This will be enforced via the
CMake CheckCompilerVersion script (ala https://reviews.llvm.org/D46723
<https://reviews.llvm.org/D46723>).  ADDITIONALLY, a CMake warning will be
issued for any major release less than 1.5* years old to give our users
sufficient time to transition/upgrade their compilers.  Finally, our dependent
C++ version will be the best released standard officially supported by the
collection of compilers (for example, we'd support -C++20 if all compilers
had std=c++20 or eqiv, but NOT std=c++2a).
> 
> The 3-years/1.5 years would result in our minimum GCC/Clang becoming:
GCC5.1/Clang3.6.  We would WARN on anything older than GCC7.1/Clang3.8
> 
> /End Proposal
> 
> 
> *: To Be Bikeshed
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180511/f33bba01/attachment.html>

On Sat, May 12, 2018 at 5:10 AM Vedant Kumar via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
> @Erich, thanks for putting this together :).
> On May 11, 2018, at 9:54 AM, Daniel Berlin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I'd be opposed to 6/5, given where it would leave us.
> It's simply hard to see a compelling reason to leave things that long.
> In particular, given this is about what it takes to produce a binary
release of clang/llvm from trunk (and not what it takes to use one), i'd
like to see some evidence/argument that using 3/1.5 would actually have a
material affect on the number of contributions, etc.
> (I have doubts it would have any affect on the abliity of new developers
to start contributing, etc).

> + 1.
> All of the clang/llvm based tools i have around (cquery, rtags, you name
it) all download and ship binary releases of clang/llvm (and FWIW, they
ship and use 1-2 year old releases).
> It's also unclear to me it makes sense to try to make sure any user can
compile the latest version - for example, researchers using it almost never
keep up with trunk, even with our current policy that supports things for
longer.  They stick with the version that existed when they started.

> So it's unclear that we are doing a thing users actually want in
practice
anyway :)

> Yep, I have the same doubts. Anecdotally, I've got a few hobby projects
I
haven't rebased since.. 3.4? I don't have a list on hand, but I've
definitely seen papers from groups that do the same thing.

> Finally,  given the rate of support for newer C++ standards in LLVM/GCC
seems to be accelerating and not slowing down (AFAICT), keeping a time
period this long will just put you farther and farther behind over time.

> It may be better to simply express it in terms of releases, and say
"we
support the past 2/3 major gcc releases, the past 2/3 major clang releases,
and the past 2 major msvc releases"


I'm a little hesitant about this because:

- This assumes that the release cadence of these projects won't change.
Tying up the LLVM HEAD to the release rate of other projects seem dangerous.

- Using time instead seems much easier to predict and easier to plan around
(by construction I think). ;)

> I'd prefer this to imposing a fixed wait period of 3 years. We could
add
a deprecation warning for compiler version X when (X+1) is released, and
switch to (X+1) when (X+2) is out. I can see this breaking down if some LTS
distro continues to ship version X, but that's in issue in the 3/1.5 scheme
as well, and we can make specific exceptions as needed.


I'm a little confused here.

I suspect the policy applies to a particular state of HEAD/trunk/master,
which means it mostly affects those who work directly on the features being
delivered in HEAD/trunk/master. For released versions, those would have
already sailed (frozen in time) and therefore we can't back-apply the
policy.

Given that, I don't see how the following won't work:

- Applying the policy to HEAD/trunk/master will/should affect the users
that *don't* track HEAD/trunk/master anyway.

- Giving a recommendation of the requirements in terms of a bootstrapping
path seems more manageable, no? i.e. if LLVM version N can build LLVM HEAD,
then if the LTS distributions can build LLVM N then they should be able to
get to LLVM HEAD.

If we're optimising for users of released versions, then there's a path
for
those packaging the release(s) to use the bootstrapping approach to get to
a specific version, if the "host compiler" for their distribution
can't
build the new release directly.

Now there's a real problem with the system ABI compatibility unless LLVM
decides that it will rely/require that it will only fully support working
with libc++ with the tools. With the libraries, that's a much harder
problem to solve, but I suspect the bootstrapping approach might also make
it feasible.

I understand that bootstrapping LLVM with itself introduces a lot of
potential for complexity and/or subtle bugginess along the way. But the
risk of that seems much less now, compared to tying the project's progress
to what other projects prioritise (e.g. OS distributions, other compilers,
etc.).
> I see the 3/1.5 scheme as basically a more conservative version of this,
so I'd be OK with that too.


Just to be clear, 3/1.5 works for me too. :)

Cheers

-- 
Dean

On 11 May 2018, at 17:54, Daniel Berlin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> It's also unclear to me it makes sense to try to make sure any user can
compile the latest version - for example, researchers using it almost never keep
up with trunk, even with our current policy that supports things for longer. 
They stick with the version that existed when they started.
This may be true (though we always follow trunk), but it isn’t something that we
should encourage.  I’ve seen (from the reviewer side) papers rejected because
people worked based on an old version of LLVM.  This has two problems for
research:

1. Their baseline is out of date.  Okay, so you showed a speedup versus a
version of LLVM from 2-3 years ago, but has trunk improved since then?  Has
trunk implemented something that makes your optimisation worse, or a better
analysis that makes it better?  In short, are your benchmarks showing anything
useful relative to the state of the art?  When reviewing, I take benchmarking
against any version of LLVM more than six months old as an indication that their
work has been superseded by something in trunk and they want to try to confuse
reviewers.

2. Their research is likely to be lower impact, because upstreaming something
based on an old release requires a lot of refactoring before you get to the
relevant state.  Again, not following trunk implies that they don’t want people
to be able to use their work or perform apples-to-apples comparisons, so it gets
a lower score when reviewing.

I know of a couple of accepted CGO papers that showed some interesting
improvements, but when the authors came to engage with the community to upstream
their results, they found that they didn’t show any speedup in the version
rebased on trunk.  This wouldn’t have happened if they’d been following trunk. 
I’d prefer that we encouraged people to do good research, rather than focus on
supporting people doing bad research.

David

On Sat, May 12, 2018 at 2:38 AM, David Chisnall <David.Chisnall at
cl.cam.ac.uk
> wrote:
> On 11 May 2018, at 17:54, Daniel Berlin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > It's also unclear to me it makes sense to try to make sure any
user can
> compile the latest version - for example, researchers using it almost never
> keep up with trunk, even with our current policy that supports things for
> longer.  They stick with the version that existed when they started.
>
> This may be true (though we always follow trunk), but it isn’t something
> that we should encourage.

Sure, but this is very orthogonal to this discussion, which is precisely my
point - our host compilation policy, which has allowed trunk compilation
with fairly old compilers for years, has had no visible effect on what
researchers do.

--Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180512/066dd972/attachment.html>

I want to chime in with an alternate perspective.  Not a counter 
argument per se, but something to consider.

When we first started evaluating LLVM, we were stuck with an absolutely 
ancient version of gcc.  We could use a newer compiler for LLVM, but if 
there was an ABI breakage - like the one moving to C++11 - between our 
build compiler and our choice of compiler for LLVM, we were pretty much 
dead in the water.  I honestly don't know if we'd had a problem at the 
time, that we'd have ever moved past it.  Or to say it differently, our 
initial evaluation period had a fixed number of person months 
allocated.  If we'd spent those person months fighting build systems, 
it's very likely we'd have not reached a point sufficient to justify a 
"go" on the rest of the project.

Just to highlight, the primary issue to be concerned about is ABI 
compatibility between whatever version of the compiler is required to 
build LLVM and the "build compiler" used for everything else. Having
the
compiler itself out of sync is (usually) not an issue, but having an ABI 
break - intentional or otherwise - is a show stopper.

Philip

On 05/11/2018 09:54 AM, Daniel Berlin via llvm-dev
wrote:
> I'd be opposed to 6/5, given where it would leave us.
> It's simply hard to see a compelling reason to leave things that long.
>
> In particular, given this is about what it takes to produce a binary 
> release of clang/llvm from trunk (and not what it takes to use one), 
> i'd like to see some evidence/argument that using 3/1.5 would actually 
> have a material affect on the number of contributions, etc.
> (I have doubts it would have any affect on the abliity of new 
> developers to start contributing, etc).
> All of the clang/llvm based tools i have around (cquery, rtags, you 
> name it) all download and ship binary releases of clang/llvm (and 
> FWIW, they ship and use 1-2 year old releases).
> It's also unclear to me it makes sense to try to make sure any user 
> can compile the latest version - for example, researchers using it 
> almost never keep up with trunk, even with our current policy that 
> supports things for longer.  They stick with the version that existed 
> when they started.
> So it's unclear that we are doing a thing users actually want in 
> practice anyway :)
>
> Finally,  given the rate of support for newer C++ standards in 
> LLVM/GCC seems to be accelerating and not slowing down (AFAICT), 
> keeping a time period this long will just put you farther and farther 
> behind over time.
>
> It may be better to simply express it in terms of releases, and say 
> "we support the past 2/3 major gcc releases, the past 2/3 major clang 
> releases, and the past 2 major msvc releases"
>
>
>
>
>
> On Fri, May 11, 2018 at 8:58 AM, Andrew Kelley via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>
>     I second this proposal, and I make a motion to lengthen 3/1.5 to 6/5.
>
>     On Fri, May 11, 2018 at 9:37 AM, Keane, Erich via llvm-dev
>     <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>
>         Hi All-
>         As we all know, the C++14 discussion is flaring up again. 
>         Chandler brought up that he would like a concrete plan to
>         switch.  In my opinion, this is insufficient, as it will
>         result in us simply having this discussion AGAIN next release.
>         Instead, I would prefer us to have a concrete Policy on our
>         host compilers.  That way, changes like this are unsurprising
>         to our users, and advance our codebase sufficiently.  I
>         believe the arguments for/against upgrading have been made
>         repeatedly, so I won't repeat them here.  My proposal is thus:
>
>         Starting with the Clang 7.0 release, we will officially
>         support any major release of our host compilers (MSVC, GCC,
>         Clang, ?ICC?) released in the past 3* years from our previous
>         branch date to give trunk-developers time to transition (so
>         for 7.0, 3 years before January 3, 2018).  This will be
>         enforced via the CMake CheckCompilerVersion script (ala
>         https://reviews.llvm.org/D46723
>         <https://reviews.llvm.org/D46723>). ADDITIONALLY, a CMake
>         warning will be issued for any major release less than 1.5*
>         years old to give our users sufficient time to
>         transition/upgrade their compilers.  Finally, our dependent
>         C++ version will be the best released standard officially
>         supported by the collection of compilers (for example, we'd
>         support -C++20 if all compilers had std=c++20 or eqiv, but NOT
>         std=c++2a).
>
>         The 3-years/1.5 years would result in our minimum GCC/Clang
>         becoming: GCC5.1/Clang3.6.  We would WARN on anything older
>         than GCC7.1/Clang3.8
>
>         /End Proposal
>
>
>         *: To Be Bikeshed
>         _______________________________________________
>         LLVM Developers mailing list
>         llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>         http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>         <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>
>
>
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>     <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180512/eea2c9a8/attachment.html>