llvm dev - Nov 2013 - [LLVMdev] [cfe-dev] RFC: A proposal to move toward using C++11 features in LLVM & Clang / bounding support for old host compilers

Renato Golin <renato.golin at linaro.org> writes:
> I think there are two main problems here:
> 1. People use trunk on side-projects because releases mean very little
> in LLVM world
> 2. We start adding new stuff as soon as a release is branched, and
> that reduces warnings up to a few days
>
> We're all talking about problem 2, when in fact, I think the problem
> is in 1.
I've been saying that for years.
> I personally don't think it's feasible to base a product/project on
a
> moving target. You already have your code that is moving, having its
> base moving too is asking for trouble. This is the same as upgrading
> the GCC from 4.8 to 4.9 when it comes out on a major distro just
> because it's shiny.
>
> Why do we do it, then? Because LLVM releases mean very little, and the
> only meaningful branch is trunk. This, for me, is the real problem.
Totally agree.
> However, Tim's position is important to consider: we've been
talking
> about this for a long while, but because no one actually set the date,
> packagers and side projects just ignored as a problem 'to deal later,
> when it really happens'. So, as much as I understand the work that
> needs to be done by the side projects, there has been a lot of warning
> for almost a year (if not more).
Except that there hasn't been warning at all.  The only "warning"
is
that "we're going to upgrade to some new toolchain."  We need to
know
*exactly* which toolchain that is so we can test all of our software
against it.  "4.7.x" doesn't help because every point release has
new
bugs, new warnings, new errors, etc.
> If we end up with a decent release policy (for backports, bugfixes,
> critical performance improvements), side projects and distributions
> would be able to use that as a base and let trunk be wild and crazy,
> as it's meant to be.
I would love to see that happen.

BUT, right now working off a release is painful because of the
integration cost of upgrading to a new release.  APIs totally change,
new bugs have to be tracked down, performance regressions fixed, etc.
Integrating our changes is a lot of work but it's not the only hard
part.  Dealing with all of the upstream changes at once is a massive
undertaking.

Now that we at least have git mirrors I can manage our custom changes
mostly mechanically (it would be easier if git were the official tool).
But I need manual intervention for handle everything else.  That's very
costly.

When I've explained this in the past, the answer was always, "work off
trunk so the changes are gradual."  I have sympathy for that view.  But
that view is based on the assumption that LLVM doesn't have at least API
backward compatibility.  LLVM is a mature product and I don't think it
has that luxury anymore.  In addition to backports to releases, we need
a process to deprecate APIs if we're going to tell people to work off
releases.

                       -David

On 6 November 2013 11:02, <dag at cray.com> wrote:
> Except that there hasn't been warning at all.  The only
"warning" is
> that "we're going to upgrade to some new toolchain."  We need
to know
> *exactly* which toolchain that is so we can test all of our software
> against it.  "4.7.x" doesn't help because every point release
has new
> bugs, new warnings, new errors, etc.
>
That's what I meant. ;)


BUT, right now working off a release is painful because of
the
> integration cost of upgrading to a new release.  APIs totally change,
> new bugs have to be tracked down, performance regressions fixed, etc.
> Integrating our changes is a lot of work but it's not the only hard
> part.  Dealing with all of the upstream changes at once is a massive
> undertaking.
>
So, here we have an even more serious problem. I can only see these options:

1. Ignore releases, everyone works on trunk
  - Changes are gradual
  1.a No API changes
    - No code breakage, we can use compilers as old as Stallman
    - Every platform is covered
    - The toolchain doesn't evolve
  1.b. API changes between each release
    - API changes under our feet, massive work all the time
    - Many API changes can happen at the same time
    - Warning time can be as little as one day
    - The toolchain evolves with time

2. Use stable releases, only core developers work on trunk
  - Changes are big, infrequent
  2.a No API changes
    - No code breakage, we can use compilers as old as Stallman
    - Every platform is covered
    - The toolchain doesn't evolve
  2.b API changes between each release
    - API changes only after *at least* six months notice
    - Massive work to move code, but only twice a year
    - The toolchain evolves with time

In a nutshell, if we want the toolchain to evolve with time, we'll need to
have API changes, and we'll need to update surrounding software to cope
with that. The only difference is if we want to be doing this as our
day-to-day job, or twice a year (or even less than that if you stay on a
release for a while longer). I personally think that it's more manageable
to do that on releases than on trunk (been there, done that, and it sucks).

All projects that depend on GCC do that on a release basis (including
kernels, distros, etc) and they have a massive undertaking on each release
of the toolchain. That includes new errors, new libraries, new options. In
your case it's a little worse, because it'll also change the compiler
you'll have to use on *your* project, and that's not something GCC users
are used to, but it has to happen one day or another.

Given that this is a special case (warnings have been only hints on the
mailing list, it'll change your compiler, we may want to move to releases),
I think we should wait until everyone is on the same page regarding all
these issues. It's already too late to have a BOF at the US meeting, but we
could gather around during the hacking session today (or via IRC) and try
to come up with a plan.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20131106/ddaac210/attachment.html>

On Wed, Nov 06, 2013 at 11:23:35AM -0800, Renato Golin
wrote:
> On 6 November 2013 11:02, <dag at cray.com> wrote:
> 
> > Except that there hasn't been warning at all.  The only
"warning" is
> > that "we're going to upgrade to some new toolchain."  We
need to know
> > *exactly* which toolchain that is so we can test all of our software
> > against it.  "4.7.x" doesn't help because every point
release has new
> > bugs, new warnings, new errors, etc.
> >
> 
> That's what I meant. ;)
> 
> 
> BUT, right now working off a release is painful because of the
> > integration cost of upgrading to a new release.  APIs totally change,
> > new bugs have to be tracked down, performance regressions fixed, etc.
> > Integrating our changes is a lot of work but it's not the only
hard
> > part.  Dealing with all of the upstream changes at once is a massive
> > undertaking.
> >
> 
> So, here we have an even more serious problem. I can only see these
options:
> 
> 1. Ignore releases, everyone works on trunk
>   - Changes are gradual
>   1.a No API changes
>     - No code breakage, we can use compilers as old as Stallman
>     - Every platform is covered
>     - The toolchain doesn't evolve
>   1.b. API changes between each release
>     - API changes under our feet, massive work all the time
>     - Many API changes can happen at the same time
>     - Warning time can be as little as one day
>     - The toolchain evolves with time
> 
> 2. Use stable releases, only core developers work on trunk
>   - Changes are big, infrequent
>   2.a No API changes
>     - No code breakage, we can use compilers as old as Stallman
>     - Every platform is covered
>     - The toolchain doesn't evolve
>   2.b API changes between each release
>     - API changes only after *at least* six months notice
>     - Massive work to move code, but only twice a year
>     - The toolchain evolves with time
> 
> In a nutshell, if we want the toolchain to evolve with time, we'll need
to
> have API changes, and we'll need to update surrounding software to cope
> with that. The only difference is if we want to be doing this as our
> day-to-day job, or twice a year (or even less than that if you stay on a
> release for a while longer). I personally think that it's more
manageable
> to do that on releases than on trunk (been there, done that, and it sucks).
> 
> All projects that depend on GCC do that on a release basis (including
> kernels, distros, etc) and they have a massive undertaking on each release
> of the toolchain. That includes new errors, new libraries, new options. In
> your case it's a little worse, because it'll also change the
compiler
> you'll have to use on *your* project, and that's not something GCC
users
> are used to, but it has to happen one day or another.
> 
> Given that this is a special case (warnings have been only hints on the
> mailing list, it'll change your compiler, we may want to move to
releases),
> I think we should wait until everyone is on the same page regarding all
> these issues. It's already too late to have a BOF at the US meeting,
but we
> could gather around during the hacking session today (or via IRC) and try
> to come up with a plan.
> 
I would be interested in discussing this at the hacking session.

-Tom

On 6 Nov 2013, at 19:23, Renato Golin <renato.golin at linaro.org> wrote:
> BUT, right now working off a release is painful because of the
> integration cost of upgrading to a new release.  APIs totally change,
> new bugs have to be tracked down, performance regressions fixed, etc.
> Integrating our changes is a lot of work but it's not the only hard
> part.  Dealing with all of the upstream changes at once is a massive
> undertaking.
I'm actually in two minds about this.  The attributes churn over the last
year was horrendous, so I've now gone back to only tracking releases for
some out-of-tree projects.  APIs were introduced with little thought, replaced,
renamed, and finally stabilised, and no documentation appeared.  It's easier
to suffer this pain once every six months than once every svn up.

And this is VERY bad for the LLVM community, because it means that people who
aren't LLVM developers won't be testing trunk.  They'll wait for a
new release, then start testing if their code is broken.  We saw this with 3.3
when there were a flurry of questions about API changes straight after the
release.  None of these should have come as a surprise to people if they'd
been following trunk.
> So, here we have an even more serious problem. I can only see these
options:
> 
> 1. Ignore releases, everyone works on trunk
>   - Changes are gradual
>   1.a No API changes
>     - No code breakage, we can use compilers as old as Stallman
>     - Every platform is covered
>     - The toolchain doesn't evolve
>   1.b. API changes between each release
>     - API changes under our feet, massive work all the time
>     - Many API changes can happen at the same time
>     - Warning time can be as little as one day
>     - The toolchain evolves with time
There is a middle ground, which is used by most libraries that are not LLVM: you
have a deprecation policy so that APIs are deprecated in one release and then
removed in the next (or the one after that).

This does impose a burden on LLVM developers, because it means that you have to
reimplement the old APIs in terms of the new ones.  In other projects, I've
found that this forces you to think very hard about the API changes, and after a
period of initial pain the result is much cleaner APIs.

For trivial changes, where someone decides to change the spelling of a method
because the old one didn't meet some convention (I've lost count of the
number of times this has happened), it means that now you get a compiler warning
saying 'getfoo() is deprecated, use getFoo() instead', rather than a
'getfoo() doesn't exist, go and read the docs (if they exist) to find
out what it is called this week'.
> 2. Use stable releases, only core developers work on trunk
>   - Changes are big, infrequent
>   2.a No API changes
>     - No code breakage, we can use compilers as old as Stallman
>     - Every platform is covered
>     - The toolchain doesn't evolve
>   2.b API changes between each release
>     - API changes only after *at least* six months notice
>     - Massive work to move code, but only twice a year
>     - The toolchain evolves with time
> 
> In a nutshell, if we want the toolchain to evolve with time, we'll need
to have API changes, and we'll need to update surrounding software to cope
with that. The only difference is if we want to be doing this as our day-to-day
job, or twice a year (or even less than that if you stay on a release for a
while longer). I personally think that it's more manageable to do that on
releases than on trunk (been there, done that, and it sucks).
> 
> All projects that depend on GCC do that on a release basis (including
kernels, distros, etc) and they have a massive undertaking on each release of
the toolchain. That includes new errors, new libraries, new options. In your
case it's a little worse, because it'll also change the compiler
you'll have to use on *your* project, and that's not something GCC users
are used to, but it has to happen one day or another.
I don't think GCC is the right comparison, because GCC is a compiler.  LLVM
is (intended to be) a set of reusable libraries for building compilers.

Maintaining approximately stable APIs and providing a deprecation strategy and
well-documented upgrade mechanisms (i.e. If you are using this API, you need to
stop and start using this one.  Here is where the new one is documented, and
here are some examples of how it's used.) is the difference between LLVM
being a set of reusable libraries and LLVM being a bunch of common code
that's been factored out of a few programs.

David

Renato Golin <renato.golin at linaro.org> writes:
> In a nutshell, if we want the toolchain to evolve with time, we'll
> need to have API changes, and we'll need to update surrounding
> software to cope with that. The only difference is if we want to be
> doing this as our day-to-day job, or twice a year (or even less than
> that if you stay on a release for a while longer). I personally think
> that it's more manageable to do that on releases than on trunk (been
> there, done that, and it sucks).
Except that the core developers recommend working off trunk.  If you
want to contribute code back, you pretty much have to work off trunk.
It is a major pain to port your changes against release x.y forward to
trunk for review and commit.

I want to re-stress that toolchain changes are fine.  We just need
enough notice before they're required on ToT.  I would like to see a
three month notice at the very least.  Four would be better.  We can't
just drop everything we're doing to go test a new toolchain.  We have to
schedule it.

                          -David