llvm dev - May 2016 - [RFC] Helping release management

Hi Hans,

Since you are actively doing this kind of things, your feedbacks is particularly
valuable.
Thanks!
> On May 2, 2016, at 3:45 PM, Hans Wennborg <hans at chromium.org>
wrote:
> 
> On Mon, May 2, 2016 at 1:35 PM, Quentin Colombet via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> I am sending this proposal to get feedbacks on how we could make the
tagging of bug fixes and regressions more obvious. The idea is to provide easily
accessible information to help deciding what to cherry-pick in a release branch.
>> 
>> * Context *
>> 
>> People shipping compilers based on LLVM may not completely align with
the official releases of LLVM. Thus, the stabilization of each custom release
may happen at different period of time. Because of that, release managers have
to come up with their own strategy to decide which commits should be
cherry-picked during the stabilization of their release branch.
> 
> (Unrelated to your proposal, I'm curious how common it is to base
> releases of LLVM-based tools off the upstream release branches vs.
> other revisions.)
> 
>> For the official LLVM releases, people (committers, code owners, etc.)
notice LLVM release managers that a given commit is worth pulling into the
release. I would like to put in place something more systematic and that plays
nicely with scripting and such that would extend this mechanism.
>> 
>> * Proposal *
>> 
>> 1. Use [Fix] for commit related to bug fixes.
> 
> I think we're mostly pretty good at referencing PR's in commit
> messages already. That's also easy to grep for, so maybe that's
good
> enough?
When a PR is available, that is certainly good enough.
I do not want to make filling a PR mandatory for each bug we fix though. Having
a PR is great, but I can see why we may not want to create one each time we fix
something.

If we do want to go into that direction though, I believe we would need to
provide more support to make that easier. (E.g. filing via command line, getting
a number back and feeding this number to a commit.)
> 
>> 2. Add a description of the problem in the commit message to help
answer the following questions:
>> - What is fixed?
>> - Which targets are impacted?
>> - What is required to trigger the bug? (I.e., how often the end users
may encounter it.)
>> - When was the bug introduced?
> 
> This sounds like the kind of information that should be in a great
> commit message anyways.
> 
> But I'm also thinking that maybe we could be better at using our bug
> tracker? Whether a bug is a feature request, something that was always
> broken, or a regression (and from what version), sounds like a perfect
> fit for a bug tracker. Someone doing a release could then query the
> Bugzilla to see e.g. what regression bugs were fixed in a certain time
> span.
Sounds great to me.

Would that kind of workflow ease your job for tracking what should be pulled
into the release branch after you’ve branched?
What would be the best workflow for such task for you?
> 
>> #1 At the very least, I would like that each bug fix has a tag on the
first line of the commit (i.e., what ends up in the subject line of the related
email.) Something like [Fix] would do.
>> Thanks to that tag, it would be possible to easily filter bug fixes in
email and other cherry-picking helper tools, I believe.
> 
> If we really do want to make a guideline about this, I propose we
> standardize on suffixing the first line of the commit with (PRnnn).
That would work for me.

Anywhere in the commit message would work as well (like Mehdi and Hal said),
though I tend to prefer the first line (but Mehdi does not like it :)).

Cheers,
-Quentin
> 
> Cheers,
> Hans

On Mon, May 2, 2016 at 4:03 PM, Quentin Colombet <qcolombet at apple.com>
wrote:
>>> 1. Use [Fix] for commit related to bug fixes.
>>
>> I think we're mostly pretty good at referencing PR's in commit
>> messages already. That's also easy to grep for, so maybe that's
good
>> enough?
>
> When a PR is available, that is certainly good enough.
> I do not want to make filling a PR mandatory for each bug we fix though.
Having a PR is great, but I can see why we may not want to create one each time
we fix something.
At least for me, if I run into a problem that takes some work to
reduce, debug, etc., I find filing a PR worth it, if nothing else to
keep track of the work for my own sake.

For bugs which one just stumbles across in the code and which can be
fixed with a quick patch, I agree that filing a bug doesn't make
sense.

I suspect the kind of fixes that are important for a release branch
are mostly of the former kind, though.
>>> 2. Add a description of the problem in the commit message to help
answer the following questions:
>>> - What is fixed?
>>> - Which targets are impacted?
>>> - What is required to trigger the bug? (I.e., how often the end
users may encounter it.)
>>> - When was the bug introduced?
>>
>> This sounds like the kind of information that should be in a great
>> commit message anyways.
>>
>> But I'm also thinking that maybe we could be better at using our
bug
>> tracker? Whether a bug is a feature request, something that was always
>> broken, or a regression (and from what version), sounds like a perfect
>> fit for a bug tracker. Someone doing a release could then query the
>> Bugzilla to see e.g. what regression bugs were fixed in a certain time
>> span.
>
> Sounds great to me.
>
> Would that kind of workflow ease your job for tracking what should be
pulled into the release branch after you’ve branched?
> What would be the best workflow for such task for you?
My ideal workflow for discovering commits suitable for merging is when
folks cc me on the commit message, because then I don't have to do any
work :-)

But besides that, I do rely on the bug tracker quite a lot. It's hard
to keep up with the commits list, but keeping up with filed and fixed
bugs is doable, and a good source of information. It's usually easy to
tell from a PR whether it's serious or not, and most people are good
about indicating what version they used, if it started failing
recently, what target they're using, etc.

Cheers,
Hans

I'm with Hans that we should use bugzilla/phab better. It's far easier
to
update bugs than format commit messages to conform to free text parseable
content.

I don't think we should restrict the commit message more than it is, nor
that we should reprimand / block / revert patches because of downstream
release processes. I also don't think we should change everyone's
process
to fit to everyone's else's process. That doesn't scale.

I personally want to see all downstream releases participating on the
upstream release process, and all their release managers actively engaged
in the public process. Anything else will mean additional cost to us all,
and go against the open source nature of the project.

My tuppence.
Renato
On 3 May 2016 12:34 a.m., "Hans Wennborg via llvm-dev" <
llvm-dev at lists.llvm.org> wrote:

On Mon, May 2, 2016 at 4:03 PM, Quentin Colombet <qcolombet at apple.com>
wrote:
>>> 1. Use [Fix] for commit related to bug fixes.
>>
>> I think we're mostly pretty good at referencing PR's in commit
>> messages already. That's also easy to grep for, so maybe that's
good
>> enough?
>
> When a PR is available, that is certainly good enough.
> I do not want to make filling a PR mandatory for each bug we fix though.
Having a PR is great, but I can see why we may not want to create one each
time we fix something.

At least for me, if I run into a problem that takes some work to
reduce, debug, etc., I find filing a PR worth it, if nothing else to
keep track of the work for my own sake.

For bugs which one just stumbles across in the code and which can be
fixed with a quick patch, I agree that filing a bug doesn't make
sense.

I suspect the kind of fixes that are important for a release branch
are mostly of the former kind, though.
>>> 2. Add a description of the problem in the commit message to help
answer the following questions:
>>> - What is fixed?
>>> - Which targets are impacted?
>>> - What is required to trigger the bug? (I.e., how often the end
users
may encounter it.)
>>> - When was the bug introduced?
>>
>> This sounds like the kind of information that should be in a great
>> commit message anyways.
>>
>> But I'm also thinking that maybe we could be better at using our
bug
>> tracker? Whether a bug is a feature request, something that was always
>> broken, or a regression (and from what version), sounds like a perfect
>> fit for a bug tracker. Someone doing a release could then query the
>> Bugzilla to see e.g. what regression bugs were fixed in a certain time
>> span.
>
> Sounds great to me.
>
> Would that kind of workflow ease your job for tracking what should be
pulled into the release branch after you’ve branched?
> What would be the best workflow for such task for you?
My ideal workflow for discovering commits suitable for merging is when
folks cc me on the commit message, because then I don't have to do any
work :-)

But besides that, I do rely on the bug tracker quite a lot. It's hard
to keep up with the commits list, but keeping up with filed and fixed
bugs is doable, and a good source of information. It's usually easy to
tell from a PR whether it's serious or not, and most people are good
about indicating what version they used, if it started failing
recently, what target they're using, etc.

Cheers,
Hans
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160503/c4f7659a/attachment.html>

> On May 2, 2016, at 4:03 PM, Quentin Colombet via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi Hans,
> 
> Since you are actively doing this kind of things, your feedbacks is
particularly valuable.
> Thanks!
> 
>> On May 2, 2016, at 3:45 PM, Hans Wennborg <hans at chromium.org>
wrote:
>> 
>> On Mon, May 2, 2016 at 1:35 PM, Quentin Colombet via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>>> I am sending this proposal to get feedbacks on how we could make
the tagging of bug fixes and regressions more obvious. The idea is to provide
easily accessible information to help deciding what to cherry-pick in a release
branch.
>>> 
>>> * Context *
>>> 
>>> People shipping compilers based on LLVM may not completely align
with the official releases of LLVM. Thus, the stabilization of each custom
release may happen at different period of time. Because of that, release
managers have to come up with their own strategy to decide which commits should
be cherry-picked during the stabilization of their release branch.
>> 
>> (Unrelated to your proposal, I'm curious how common it is to base
>> releases of LLVM-based tools off the upstream release branches vs.
>> other revisions.)
>> 
>>> For the official LLVM releases, people (committers, code owners,
etc.) notice LLVM release managers that a given commit is worth pulling into the
release. I would like to put in place something more systematic and that plays
nicely with scripting and such that would extend this mechanism.
>>> 
>>> * Proposal *
>>> 
>>> 1. Use [Fix] for commit related to bug fixes.
>> 
>> I think we're mostly pretty good at referencing PR's in commit
>> messages already. That's also easy to grep for, so maybe that's
good
>> enough?
> 
> When a PR is available, that is certainly good enough.
> I do not want to make filling a PR mandatory for each bug we fix though.
Having a PR is great, but I can see why we may not want to create one each time
we fix something.
Why not?
> 
> If we do want to go into that direction though, I believe we would need to
provide more support to make that easier. (E.g. filing via command line, getting
a number back and feeding this number to a commit.)
Making our bug tracking system and related tools better seems independently
good.
> 
>> 
>>> 2. Add a description of the problem in the commit message to help
answer the following questions:
>>> - What is fixed?
>>> - Which targets are impacted?
>>> - What is required to trigger the bug? (I.e., how often the end
users may encounter it.)
>>> - When was the bug introduced?
>> 
>> This sounds like the kind of information that should be in a great
>> commit message anyways.
>> 
>> But I'm also thinking that maybe we could be better at using our
bug
>> tracker? Whether a bug is a feature request, something that was always
>> broken, or a regression (and from what version), sounds like a perfect
>> fit for a bug tracker. Someone doing a release could then query the
>> Bugzilla to see e.g. what regression bugs were fixed in a certain time
>> span.
> 
> Sounds great to me.
> 
> Would that kind of workflow ease your job for tracking what should be
pulled into the release branch after you’ve branched?
> What would be the best workflow for such task for you?
> 
>> 
>>> #1 At the very least, I would like that each bug fix has a tag on
the first line of the commit (i.e., what ends up in the subject line of the
related email.) Something like [Fix] would do.
>>> Thanks to that tag, it would be possible to easily filter bug fixes
in email and other cherry-picking helper tools, I believe.
>> 
>> If we really do want to make a guideline about this, I propose we
>> standardize on suffixing the first line of the commit with (PRnnn).
> 
> That would work for me.
> 
> Anywhere in the commit message would work as well (like Mehdi and Hal
said), though I tend to prefer the first line (but Mehdi does not like it :)).
> 
> Cheers,
> -Quentin
> 
>> 
>> Cheers,
>> Hans
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160504/14c36178/attachment-0001.html>

> On May 4, 2016, at 4:41 PM, Jim Grosbach <grosbach at apple.com>
wrote:
> 
>> 
>> On May 2, 2016, at 4:03 PM, Quentin Colombet via llvm-dev <llvm-dev
at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> Hi Hans,
>> 
>> Since you are actively doing this kind of things, your feedbacks is
particularly valuable.
>> Thanks!
>> 
>>> On May 2, 2016, at 3:45 PM, Hans Wennborg <hans at chromium.org
<mailto:hans at chromium.org>> wrote:
>>> 
>>> On Mon, May 2, 2016 at 1:35 PM, Quentin Colombet via llvm-dev
>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>>>> I am sending this proposal to get feedbacks on how we could
make the tagging of bug fixes and regressions more obvious. The idea is to
provide easily accessible information to help deciding what to cherry-pick in a
release branch.
>>>> 
>>>> * Context *
>>>> 
>>>> People shipping compilers based on LLVM may not completely
align with the official releases of LLVM. Thus, the stabilization of each custom
release may happen at different period of time. Because of that, release
managers have to come up with their own strategy to decide which commits should
be cherry-picked during the stabilization of their release branch.
>>> 
>>> (Unrelated to your proposal, I'm curious how common it is to
base
>>> releases of LLVM-based tools off the upstream release branches vs.
>>> other revisions.)
>>> 
>>>> For the official LLVM releases, people (committers, code
owners, etc.) notice LLVM release managers that a given commit is worth pulling
into the release. I would like to put in place something more systematic and
that plays nicely with scripting and such that would extend this mechanism.
>>>> 
>>>> * Proposal *
>>>> 
>>>> 1. Use [Fix] for commit related to bug fixes.
>>> 
>>> I think we're mostly pretty good at referencing PR's in
commit
>>> messages already. That's also easy to grep for, so maybe
that's good
>>> enough?
>> 
>> When a PR is available, that is certainly good enough.
>> I do not want to make filling a PR mandatory for each bug we fix
though. Having a PR is great, but I can see why we may not want to create one
each time we fix something.
> 
> Why not?
I should have phrase it differently :).
I do not want to make filing a PR *manually* mandatory for each bug we fix.

Typically, what I had in mind was things like typos/thinko, that are bugs, that
we notice a few minutes after we made the “main” commit. I do not want we have
to file a PR that is going to repeat what we are going to say in the commit
message.

Ultimately, yes, we should have a PR for that, but it should not be in the way
of our productivity, which leads me to your second comment
:).
> 
>> 
>> If we do want to go into that direction though, I believe we would need
to provide more support to make that easier. (E.g. filing via command line,
getting a number back and feeding this number to a commit.)
> 
> Making our bug tracking system and related tools better seems independently
good.
Agreed. Having better tools could be a way to have a PR for every fixes we do,
without actually having to file one. Like I know I am fixing a bug I introduced
in my last commit, please awesome_command_line_tool™ file a PR alongside with
the commit message.
> 
>> 
>>> 
>>>> 2. Add a description of the problem in the commit message to
help answer the following questions:
>>>> - What is fixed?
>>>> - Which targets are impacted?
>>>> - What is required to trigger the bug? (I.e., how often the end
users may encounter it.)
>>>> - When was the bug introduced?
>>> 
>>> This sounds like the kind of information that should be in a great
>>> commit message anyways.
>>> 
>>> But I'm also thinking that maybe we could be better at using
our bug
>>> tracker? Whether a bug is a feature request, something that was
always
>>> broken, or a regression (and from what version), sounds like a
perfect
>>> fit for a bug tracker. Someone doing a release could then query the
>>> Bugzilla to see e.g. what regression bugs were fixed in a certain
time
>>> span.
>> 
>> Sounds great to me.
>> 
>> Would that kind of workflow ease your job for tracking what should be
pulled into the release branch after you’ve branched?
>> What would be the best workflow for such task for you?
>> 
>>> 
>>>> #1 At the very least, I would like that each bug fix has a tag
on the first line of the commit (i.e., what ends up in the subject line of the
related email.) Something like [Fix] would do.
>>>> Thanks to that tag, it would be possible to easily filter bug
fixes in email and other cherry-picking helper tools, I believe.
>>> 
>>> If we really do want to make a guideline about this, I propose we
>>> standardize on suffixing the first line of the commit with (PRnnn).
>> 
>> That would work for me.
>> 
>> Anywhere in the commit message would work as well (like Mehdi and Hal
said), though I tend to prefer the first line (but Mehdi does not like it :)).
>> 
>> Cheers,
>> -Quentin
>> 
>>> 
>>> Cheers,
>>> Hans
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160504/14db3233/attachment.html>