[llvm-dev] LLVM Releases: Upstream vs. Downstream / Distros

Quentin Colombet via llvm-dev
2016-05-13T20:01:29Z

> On May 12, 2016, at 7:56 AM, Kristof Beyls <kristof.beyls at arm.com>
wrote:
>
> FWIW, for our ARM Compiler product, we follow top-of-trunk, not the
releases.
> Next to picking up new functionality quicker, it also allows us to detect
regressions
> in LLVM against our in-house testing quickly, not 6 months later. We find
that when
> we find a regression within 24 to 48 hours of the commit introducing it,
it's much
> cheaper to get it fixed.
>
> In my opinion, it would be better overall for the LLVM project if
top-of-trunk is
> tested as much as possible, if testing resources are so scarce that a
choice
> has to be made between testing top-of-trunk or testing a release branch.

+1 to everything Kristof said.

I will also add to the discussion that I believe it does not make sense to try
to align releases with open source in general because different people have
different goals. For instance, when we (the open source community) branch for a
release, we try to make the branch as stable as possible and only pull-in fixes.
Other people may have different goal, like it is possible a new feature was
expected for that release and that we need to pull that in. And we are back to
the problem we were discussing in the other thread: helping release manager for
general commits.

Cheers,
-Quentin

>
> Thanks,
>
> Kristof
>
>> On 12 May 2016, at 02:10, Robinson, Paul <paul.robinson at sony.com
<mailto:paul.robinson at sony.com>> wrote:
>>
>>> -----Original Message-----
>>> From: Renato Golin [mailto:renato.golin at linaro.org
<mailto:renato.golin at linaro.org>]
>>> Sent: Wednesday, May 11, 2016 11:11 AM
>>> To: Hans Wennborg
>>> Cc: LLVM Dev; Clang Dev; Quentin Colombet; Tom Stellard; Robinson,
Paul;
>>> Jim Grosbach; Kristof Beyls; Frédéric Richez; Reid Kleckner; Philip
>>> Reames; Matthias Braun; Bernhard Rosenkränzer; Sylvestre Ledru;
Matthias
>>> Klose; Stephen Hines; Jeff Law; Ed Maste; Behan Webster
>>> Subject: Re: LLVM Releases: Upstream vs. Downstream / Distros
>>>
>>> On 11 May 2016 at 17:16, Hans Wennborg <hans at chromium.org
<mailto:hans at chromium.org>> wrote:
>>>> This is a long email :-) I've made some comments inline,
but I'll
>>>> summarize my thoughts here:
>>>
>>> Thanks Hans!
>>>
>>> I'll respond them inline, below.
>>>
>>>
>>>> - I think we should use the bug tracker to capture issues that
affect
>>>> releases. It would be cool if a commit hook could update
bugzilla
>>>> entries that refer to it.
>>>
>>> That seems like a simple hook.
>>>
>>>
>>>> At least for the major releases, I think we're doing pretty
well on
>>>> timing in terms of predictability: since 3.6, we have release
every
>>>> six months: first week of March and first week of September (+-
a few
>>>> days). Branching has been similarly predictive: mid-January and
>>>> mid-July.
>>>
>>> Indeed, we got a lot better more recently (last 2y), and mostly
thanks
>>> to you. :)
>>
>> Absolutely.  It's enough of a track record to allow reasonable
planning.
>>
>>>
>>> We used to vary 3 months +-, and now we're down to a few days.
>>> Whatever we decide, I think we should make it official but putting
it
>>> out somewhere, so people can rely on that.
>>
>> A public commitment to the future release schedule would be that much
>> more justification for planning to participate.
>>
>>>
>>> Right now, even if you're extra awesome, there's nothing
telling the
>>> distros and LLVM-based products that it will be so if someone else
>>> takes over the responsibility, so they can't adapt.
>>>
>>> That's what I meant by "quasi-chaotic".
>>>
>>>
>>>> If there are many downstream releases for which shifting this
schedule
>>>> would be useful, I suppose we could do that, but it seems
unlikely
>>>> that there would be agreement on this, and changing the
schedule is
>>>> disruptive for those who depend on it.
>>>
>>> That's the catch. If we want them to participate, the process
has to
>>> have some meaning to them. The fact that not many people do, is
clear
>>> to me what it means.
>>>
>>> We also need to know better how many other releases already depend
on
>>> the upstream process (not just Chromium, for obvious reasons), to
be
>>> able to do an informed choice of dates and frequency.
>>>
>>> The more well positioned and frequent we are, the more people will
>>> help, but there's a point where the curve bends down, and the
cost is
>>> just too great. We need to find the inflection point, and that will
>>> require some initial investigations and guesses, and a lot of fine
>>> tuning later. But if we're all on the same page, I think we can
do
>>> that, even if it takes time.
>>>
>>> I'm particularly concerned with Android, because they not only
have
>>> their own tree with heavily modified LLVM components (ex.
>>> Compiler-RT), but they also build differently and so their process
are
>>> completely alien to ours. One of the key reasons why these things
>>> happened is because:
>>>
>>> * They couldn't rely on our releases, as fixing bugs and
back-porting
>>> wasn't a thing back then
>>> * They already had their own release schedule, so aligning with
ours
>>> brought no extra benefit
>>> * We always expected people to work off trunk, and everyone had to
>>> create their own process
>>>
>>> I don't want to change how people work, just to add one more
valid way
>>> of working, which is most stable for upstream releases. :)
>>>
>>>
>>>
>>>> The only reasonable way I see of aligning upstream releases
with
>>>> downstream schedules would be to release much more often. This
works
>>>> well in Chromium where there's a 6-week staged release
schedule. This
>>>> would mean there's always a branch going for the next
release, and
>>>> important bug fixes would get merged to that.
>>>
>>> Full validation every 6 weeks is just not possible. But a multiple
of
>>> that, say every 3~4 months, could be much easier to work around.
>>>
>>>
>>>
>>>> In Chromium we drive
>>>> this from the bug tracker -- it would be very hard to scan each
commit
>>>> for things to cherry-pick. This kind of process has a high cost
>>>> though, there has to be good infrastructure for it (buildbots
on the
>>>> branch for all targets, for example), developers have to be
aware, and
>>>> even then it's a lot of work for those doing the releases.
I'm not
>>>> sure we'd want to take this on. I'm also not sure it
would be suitable
>>>> for a compiler, where we want the releases to have long
life-time.
>>>
>>> This works because you have a closed system. As you say, Chromium
is
>>> mostly final product, not a tool to develop other products, and the
>>> validation is a lot simpler.
>>>
>>> With Clang, we'd want to involve external releases into it, and
it
>>> simply wouldn't scale.
>>>
>>>
>>>
>>>> For the major releases, I've tried to do this. We could
certainly
>>>> formalize it by posting it on the web page though.
>>>
>>> I think that'd be the first step, yes. But I wanted to start
with a
>>> good number. 2 times a year? Would 3 times improve things that much
>>> for the outsiders? Or just moving the dates would be enough for
most
>>> people?
>>>
>>> That's why I copied so many outsiders, so they can chime in and
let us
>>> know what would be good for *them*.
>>
>> Data point: At Sony we ship our stuff every 6 months and that isn't
>> going to change.  Our policy has been to base our releases on upstream
>> releases, and given our current lead times, the current upstream
release
>> schedule is actually not bad, especially if it has stopped drifting.
>> Having four releases pretty consistently follow the current schedule is
>> extremely positive, thanks!
>>
>> It would help our internal planning to publish the schedule for future
>> releases.  You never know, we might even be able to fork an internal
>> branch in time to help with the release testing, although that is not
>> in any way a lightweight process (so no promises).
>>
>> There has been some talk about moving toward releasing from upstream
>> trunk, or more precisely to streamlining our internal testing process
>> in order to allow us to seriously contemplate releasing from upstream
>> trunk.  I can't argue with the streamlining part, for sure.
Whether
>> the rest of it works out must remain to be seen.
>> --paulr
>>
>>>
>>>
>>>> Most importantly, those folks should get involved :-)
>>>
>>> Indeed!
>>>
>>>
>>>> In practice, we kind of have this for at least some of the
targets.
>>>> Maybe we should write this down somewhere instead of me asking
for
>>>> (the same) volunteers each time the release process starts?
>>>
>>> I give consent to mark me as the ARM/AArch64 release tester for the
>>> foreseeable future. :)
>>>
>>> I can also help Sylvestre, Doko, Ed, Jeff, Bero etc. to test on
their
>>> system running on ARM/AArch64 hardware.
>>>
>>> cheers,
>>> --renato
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160513/fe58e44e/attachment.html>