llvm dev - Nov 2021 - RFC: Code Review Process

Hi,

# Proposal

The LLVM Foundation Board of Directors is seeking comment on the current state
of Code Review
within the LLVM Project and its sub-projects.   Phabricator is no longer
actively maintained
and we would like to move away from a self-hosted solution, so our goal is to
determine if
GitHub Pull Requests are a good alternative to our current code review tool:
Phabricator.

Specifically we are looking for feedback on:
- What features or properties make Github Pull Requests better than Phabricator?
- What features or properties  make Phabricator better than GitHub Pull
Requests?
- What new workflows or process improvements will be possible with GitHub Pull
Requests?
- Which workflows aren’t possible with GitHub Pull Requests?
- Any other information that you think will help the Board of Directors make the
best decision.

# Where to Direct Feedback

Please provide feedback on the Infrastructure Working Group ticket[1].  This
will make
it easier to collect and consolidate the responses.   At the end of the comment
period
the Infrastructure Working Group will collect the feedback for further analysis
and summarization.

# Timeline

The timeline for this RFC will be as follows:

- RFC posted on llvm-dev for public review and comment
- 30 days after the date of posting, public comment closes.
- IWG will have 14 days from closure of public comments to review and summarize
public
   comments into a pros and cons list to be present to LLVM Foundation Board
- Foundation Board will have 30 days to make a final decision about using GitHub
Pull Requests
   and then communicate a migration plan to the community.

Thank you,
LLVM Foundation Board of Directors

[1] https://github.com/llvm/llvm-iwg/issues/73

On Tue, 5 Oct 2021 at 17:06, Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> - Any other information that you think will help the Board of Directors
> make the best decision.
- Foundation Board will have 30 days to make a final decision about
using
> GitHub Pull Requests and then communicate a migration plan to the
community.

Hi Tom,

Please help me here, I think I'm severely misunderstanding what this
means...

I'm reading it that the "Board of Directors" will make a decision
and
communicate to the community, apparently through some undisclosed internal
process.

For example:
 * What about people that are on holidays during the 30 days comment period?
 * What if the points are not made clear after 30 days?
 * How do we know the IWG will correctly summarise the comments to the
board?
 * How does the board guarantee it will take all facts in consideration
without bias?
 * What kind of recourse would the community have if the decision alienates
a large part of the developers?

Please understand that I'm not assuming malice *at all*. We're all
humans,
and in the effort to make some change happen we quite often let unconscious
bias be the merits of our decisions.

For context...

Since its inception[1], the foundation has always steered away from
technical decisions, always referring to the llvm-dev list for those.
Previous long running contentious issues (Github, monorepo, CoC) were all
decided by the community, in the llvm-dev list, and executed by the
foundation.

Recent discussions about the mailing list, irc, discord, discourse have
emphasised that, even with an infrastructure working group, the views of
the community are still too hard to predict and make it work for the
majority. Neither the board of directors, nor the IWG are wide and diverse
enough to make decisions that take most people's views into consideration.
That is why we still rely on the dev list for large technical discussions
and decisions.

Code review and bug tracking are very much technical decisions. Not code
directly, but how we all work. And there are a lot of us. Giving feedback
and having no insight into the decision making process will certainly
divide the community even more, if we're forced to accept whatever outcome.

I can't see how this "solves" the problem of never-ending
discussions,
other than further fragmenting the community.

cheers,
--renato

[1] http://blog.llvm.org/2014/04/the-llvm-foundation.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211005/2e925010/attachment.html>

On Tue, Oct 5, 2021 at 9:05 AM Tom Stellard via cfe-dev
<cfe-dev at lists.llvm.org> wrote:
>
> Hi,
>
> # Proposal
>
> The LLVM Foundation Board of Directors is seeking comment on the current
state of Code Review
> within the LLVM Project and its sub-projects.   Phabricator is no longer
actively maintained
> and we would like to move away from a self-hosted solution, so our goal is
to determine if
> GitHub Pull Requests are a good alternative to our current code review
tool: Phabricator.
>
> Specifically we are looking for feedback on:
> - What features or properties make Github Pull Requests better than
Phabricator?
I appreciate the ability to have bots help participate in code review,
though perhaps this isn't actually specific to github.

In particular, I think rustc's use of bors for merging is pretty
awesome; people literally cannot merge unless the bot has run all unit
tests on all platforms.  As is, anyone can commit without running any
tests.

I frequently break the build for non-linux platforms. That LLVM has so
many reverts for breakage on platforms tested post submit is
embarrassing.  People chasing such breakages is a waste of their time,
IMO.  We should have more pre-submit testing; I think the board should
focus on that particular problem first, BEFORE pursuing changing code
review platforms.

It seems an obvious risk to me that phab is no longer maintained, but
as others have noted, it hasn't all come crashing down yet.
> - What features or properties  make Phabricator better than GitHub Pull
Requests?
> - What new workflows or process improvements will be possible with GitHub
Pull Requests?
> - Which workflows aren’t possible with GitHub Pull Requests?
> - Any other information that you think will help the Board of Directors
make the best decision.
>
> # Where to Direct Feedback
>
> Please provide feedback on the Infrastructure Working Group ticket[1]. 
This will make
> it easier to collect and consolidate the responses.   At the end of the
comment period
> the Infrastructure Working Group will collect the feedback for further
analysis and summarization.
>
> # Timeline
>
> The timeline for this RFC will be as follows:
>
> - RFC posted on llvm-dev for public review and comment
> - 30 days after the date of posting, public comment closes.
> - IWG will have 14 days from closure of public comments to review and
summarize public
>    comments into a pros and cons list to be present to LLVM Foundation
Board
> - Foundation Board will have 30 days to make a final decision about using
GitHub Pull Requests
>    and then communicate a migration plan to the community.
>
> Thank you,
> LLVM Foundation Board of Directors
>
> [1] https://github.com/llvm/llvm-iwg/issues/73
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev


-- 
Thanks,
~Nick Desaulniers

Hi Tom,

   Thank you for stepping up and encouraging discussion but also a timeline!

   I've been using GitHub Pull Requests and Phabricator for years. I 
have been waiting for this switch since years. In my opinion Phabricator 
has been holding us off for too long. I do not want to go into too much 
details on features, web interface etc. I think both systems are good 
for implementing a reasonable code review process. I want to point out 
two major features which will introduce drastic improvements. Firstly, 
preflight builds will reduce the 
land,revert,reland,revert,reapply,revert style of development. Secondly, 
GitHub is the choice for many (young?) developers who will have much 
easier time to start contributing. Believe it or not the people I have 
to introduce to LLVM development have never heard of Phabricator before 
and struggle quite a bit...

Best, Vassil


On 10/5/21 7:05 PM, Tom Stellard via cfe-dev wrote:
> Hi,
>
> # Proposal
>
> The LLVM Foundation Board of Directors is seeking comment on the 
> current state of Code Review
> within the LLVM Project and its sub-projects.   Phabricator is no 
> longer actively maintained
> and we would like to move away from a self-hosted solution, so our 
> goal is to determine if
> GitHub Pull Requests are a good alternative to our current code review 
> tool: Phabricator.
>
> Specifically we are looking for feedback on:
> - What features or properties make Github Pull Requests better than 
> Phabricator?
> - What features or properties  make Phabricator better than GitHub 
> Pull Requests?
> - What new workflows or process improvements will be possible with 
> GitHub Pull Requests?
> - Which workflows aren’t possible with GitHub Pull Requests?
> - Any other information that you think will help the Board of 
> Directors make the best decision.
>
> # Where to Direct Feedback
>
> Please provide feedback on the Infrastructure Working Group 
> ticket[1].  This will make
> it easier to collect and consolidate the responses.   At the end of 
> the comment period
> the Infrastructure Working Group will collect the feedback for further 
> analysis and summarization.
>
> # Timeline
>
> The timeline for this RFC will be as follows:
>
> - RFC posted on llvm-dev for public review and comment
> - 30 days after the date of posting, public comment closes.
> - IWG will have 14 days from closure of public comments to review and 
> summarize public
>   comments into a pros and cons list to be present to LLVM Foundation 
> Board
> - Foundation Board will have 30 days to make a final decision about 
> using GitHub Pull Requests
>   and then communicate a migration plan to the community.
>
> Thank you,
> LLVM Foundation Board of Directors
>
> [1] https://github.com/llvm/llvm-iwg/issues/73
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

On 10/5/21 9:05 AM, Tom Stellard wrote:
> Hi,
> 
> # Proposal
> 
> The LLVM Foundation Board of Directors is seeking comment on the current
state of Code Review
> within the LLVM Project and its sub-projects.   Phabricator is no longer
actively maintained
> and we would like to move away from a self-hosted solution, so our goal is
to determine if
> GitHub Pull Requests are a good alternative to our current code review
tool: Phabricator.
> 
> Specifically we are looking for feedback on:
> - What features or properties make Github Pull Requests better than
Phabricator?
> - What features or properties  make Phabricator better than GitHub Pull
Requests?
> - What new workflows or process improvements will be possible with GitHub
Pull Requests?
> - Which workflows aren’t possible with GitHub Pull Requests?
> - Any other information that you think will help the Board of Directors
make the best decision.
> 
> # Where to Direct Feedback
> 
> Please provide feedback on the Infrastructure Working Group ticket[1]. 
This will make
> it easier to collect and consolidate the responses.   At the end of the
comment period
> the Infrastructure Working Group will collect the feedback for further
analysis and summarization.
> 
> # Timeline
> 
> The timeline for this RFC will be as follows:
> 
> - RFC posted on llvm-dev for public review and comment
> - 30 days after the date of posting, public comment closes.
Hi,

Just a reminder, please add your comments to the github issue[1] by
November 4.

-Tom

> - IWG will have 14 days from closure of public comments to review and
summarize public
>    comments into a pros and cons list to be present to LLVM Foundation
Board
> - Foundation Board will have 30 days to make a final decision about using
GitHub Pull Requests
>    and then communicate a migration plan to the community.
> 
> Thank you,
> LLVM Foundation Board of Directors
> 
> [1] https://github.com/llvm/llvm-iwg/issues/73