llvm dev - Feb 2016 - RFC: Move the test-suite LLVM project to GitHub?

> On Feb 25, 2016, at 3:33 AM, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Kristof, Chandler,
> 
> I think most of the responses seem favourable of the move, the
> concerns being which Git repo we'll use (GitHub, GitLab, BitBucket),
> but they're essentially identical on the git side.
> 
> Infrastructure decisions will need to be taken into account, but that
> doesn't interfere with the "how we commit" discussion in any
way.
> 
> 
> On 25 February 2016 at 08:55, Kristof Beyls via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> I'm afraid I've never committed to any github project before,
so I am a bit
>> confused on how committing would work in practice.
> 
> GitHub (and others) work by fork&pull-request.
GitHub offers  a simple *raw* git hosting. All the fork&pull-request is
sugar in the web interface. You don't *have to* use it.

> Simply put, you fork
> the original repo, do your changes, push to your repo and make a
> "pull-request" to the original repo. The added work is that one
of the
> original repo owners will have to "accept" your pull-request, and
> GitHub then merges your changes.
> 
> One of the design decisions of working this way is to have less
> committers on the main repo, and everyone else works on their own.
> This is great for collaborative development, but not very similar of
> the way we work.
We don't *have to* work this way on GitHub, you can allow
"everyone" to commit (by asking to be added to the committer list just
the same way we do it with the current svn servers).
I.e. it is possible to move to GitHub and *not* change our current workflow.
> 
> I believe all those services use something similar to git-new-workdir,
> so even if we have 100 forks of the test-suite, we won't have 100 x
> 1GB of used space. But if we move into a "less owners" scenario,
we
> will penalise them with pull-requests all the time. Having said that,
> if we have enough, and if they, too, follow the fork & pull-request
> model, I think it could work.
> 
> 
>> Obviously, I hope for the mechanism to commit to be simple - e.g. just
do a
>> git push from the command line.
> 
> This is a little more complex. SVN is very conservative on history,
> and that saves us from destroying the origin. Git, on the other hand,
> allows anyone with write access to completely wipe out the repo. I
> don't think anyone would want to, but accidents do happen in git.
This is why git has "hooks": for example
https://gist.github.com/stefansundin/d465f1e331fc5c632088

You can prevent from rewriting the history on the server side, which eliminate
risk of "accidents". (You can have a whitelist of people allowed to
rewrite the history...).

> 
> As was said earlier, "git doesn't destroy data" in the sense
that
> anyone with a good tree can re-make the origin, or even for a limited
> time (while commits are still in cache), you can cherry-pick them back
> and remake the tree. But buildbots will NOT be happy with broken
> trees. With new committers coming all the time, it's hard to control
> how much people know about git and GitHub (et al), so accidents are
> bound to happen more often than they should.
> 
> The other problem is about release validation. We'll need to do
> separate "releases" of the test-suite for each LLVM release,
since we
> can't run old tests with new releases and vice versa.
Not sure how this part relates to git/github/svn?

-- 
Mehdi

> 
> Another way to work with git is like the Linux kernel, where different
> maintainers have different trees and they all merge from each other. I
> personally think that would be a nightmare for us.
> 
> If anyone with more git experience than me can come up with a safe way
> to have 100s of committers pushing to master, I'd be happy to know.
> 
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 25 February 2016 at 11:54, Mehdi Amini <mehdi.amini at apple.com>
wrote:
> GitHub offers  a simple *raw* git hosting. All the fork&pull-request is
sugar in the web interface. You don't *have to* use it.
I mentioned this below. I was referring how GitHub *expects* to be used.

> This is why git has "hooks": for example
https://gist.github.com/stefansundin/d465f1e331fc5c632088
>
> You can prevent from rewriting the history on the server side, which
eliminate risk of "accidents". (You can have a whitelist of people
allowed to rewrite the history...).
Right, this solves most of the problems.

> Not sure how this part relates to git/github/svn?
It relates to having a separate repository. The administration cost of
adding new committers, tags, branches is linearly proportional to the
number of repositories, with the addition complexity of them being
decoupled.

Specifically for the releases, we need to make sure that the commit on
the test-suite that gets tagged for a release is the one supposed to
be, not just the tip of the tree. If we allow non-linear history, this
gets more complex.

Luckily, the test-suite is low traffic enough that this won't be a
problem often.

cheers,
--renato

Sent from my iPhone
> On Feb 25, 2016, at 4:05 AM, Renato Golin <renato.golin at
linaro.org> wrote:
> 
>> On 25 February 2016 at 11:54, Mehdi Amini <mehdi.amini at
apple.com> wrote:
>> GitHub offers  a simple *raw* git hosting. All the
fork&pull-request is sugar in the web interface. You don't *have to* use
it.
> 
> I mentioned this below. I was referring how GitHub *expects* to be used.
> 
> 
>> This is why git has "hooks": for example
https://gist.github.com/stefansundin/d465f1e331fc5c632088
>> 
>> You can prevent from rewriting the history on the server side, which
eliminate risk of "accidents". (You can have a whitelist of people
allowed to rewrite the history...).
> 
> Right, this solves most of the problems.
> 
> 
>> Not sure how this part relates to git/github/svn?
> 
> It relates to having a separate repository.
Oh yes, makes sense!
(Sorry it's a bit late...)


-- 
Mehdi

> The administration cost of
> adding new committers, tags, branches is linearly proportional to the
> number of repositories, with the addition complexity of them being
> decoupled.
> 
> Specifically for the releases, we need to make sure that the commit on
> the test-suite that gets tagged for a release is the one supposed to
> be, not just the tip of the tree. If we allow non-linear history, this
> gets more complex.
> 
> Luckily, the test-suite is low traffic enough that this won't be a
> problem often.
> 
> cheers,
> --renato

Mehdi Amini via llvm-dev <llvm-dev <at> lists.llvm.org> writes:
> > This is a little more complex. SVN is very conservative on history,
> > and that saves us from destroying the origin. Git, on the other hand,
> > allows anyone with write access to completely wipe out the repo. I
> > don't think anyone would want to, but accidents do happen in git.
> 
> This is why git has "hooks": for example
https://gist.github.com/stefansundin/d465f1e331fc5c632088
> 
> You can prevent from rewriting the history on the server side, which
eliminate risk of "accidents". (You
> can have a whitelist of people allowed to rewrite the history...).
One downside of hooks is that you're relying on everyone who has commit
access to manually set them up on every machine they might commit from.
On the server side, GitHub has a relatively recent feature called
protected branches that you can enable
(https://github.com/blog/2051-protected-branches-and-required-status-checks)
to prevent force pushing
to important permanent branches like master or releases. Force pushing
to temporary personal branches is usually fine, but it's cleaner to
store personal branches within your personal fork rather than inside
the official main repo anyway.

They also have an additional option on top of the protected branch
feature that only allows pull requests to be merged if they've passed
certain status checks from external CI services like Travis CI, AppVeyor,
etc. It's also possible to make buildbot send status webhooks to github
for this, but this only helps if the buildbot is configured to do
pre-merge testing the way Rust does rather than post-commit testing
like LLVM's buildbots do now. I've found the required status checks
are picky about not accepting green statuses that were run against an
out-of-date merge commit, so the required status feature isn't really
suitable for a rapidly changing master from my experience the last time
I tried it.

-Tony

Am 25.02.2016 um 13:08 schrieb Tony Kelman via llvm-dev:
> Mehdi Amini via llvm-dev <llvm-dev <at> lists.llvm.org> writes:
>
>>> This is a little more complex. SVN is very conservative on history,
>>> and that saves us from destroying the origin. Git, on the other
hand,
>>> allows anyone with write access to completely wipe out the repo. I
>>> don't think anyone would want to, but accidents do happen in
git.
>>
>> This is why git has "hooks": for example
> https://gist.github.com/stefansundin/d465f1e331fc5c632088
>>
>> You can prevent from rewriting the history on the server side, which
> eliminate risk of "accidents". (You
>> can have a whitelist of people allowed to rewrite the history...).
>
> One downside of hooks is that you're relying on everyone who has commit
> access to manually set them up on every machine they might commit from.
If the repo is on GitHub, the hook runs on GitHub, so individual 
committers don't have to do anything specific.
> On the server side, GitHub has a relatively recent feature called
> protected branches that you can enable
>
(https://github.com/blog/2051-protected-branches-and-required-status-checks)
> to prevent force pushing
> to important permanent branches like master or releases.
Oh, I wasn't aware that they added this.
That fixes my biggest reservation about GitHub.

> On Feb 25, 2016, at 4:05 AM, Renato Golin <renato.golin at
linaro.org> wrote:
> 
> On 25 February 2016 at 11:54, Mehdi Amini <mehdi.amini at apple.com>
wrote:
>> GitHub offers  a simple *raw* git hosting. All the
fork&pull-request is sugar in the web interface. You don't *have to* use
it.
> 
> I mentioned this below. I was referring how GitHub *expects* to be used.
I'd vote to change one thing at a time and not start switching to a full
"github-style" model for now. We can continue in the current style on
github as well.(Though as a side note, I don't think we are far away from
the github model as phabricator revisions feel pretty similar to pull requests).
> 
> 
>> This is why git has "hooks": for example
https://gist.github.com/stefansundin/d465f1e331fc5c632088
>> 
>> You can prevent from rewriting the history on the server side, which
eliminate risk of "accidents". (You can have a whitelist of people
allowed to rewrite the history...).
> 
> Right, this solves most of the problems.
So far my experience has been that git is no worse (and often slightly better)
in terms of reliability than svn.
> 
> 
>> Not sure how this part relates to git/github/svn?
> 
> It relates to having a separate repository. The administration cost of
> adding new committers, tags, branches is linearly proportional to the
> number of repositories, with the addition complexity of them being
> decoupled.
> 
> Specifically for the releases, we need to make sure that the commit on
> the test-suite that gets tagged for a release is the one supposed to
> be, not just the tip of the tree. If we allow non-linear history, this
> gets more complex.
> 
> Luckily, the test-suite is low traffic enough that this won't be a
> problem often.
That is an interesting point, will Chris start adding people to github as well?

- Matthias

> On Feb 25, 2016, at 4:05 AM, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On 25 February 2016 at 11:54, Mehdi Amini <mehdi.amini at apple.com>
wrote:
>> GitHub offers  a simple *raw* git hosting. All the
fork&pull-request is sugar in the web interface. You don't *have to* use
it.
> 
> I mentioned this below. I was referring how GitHub *expects* to be used.
I fundamentally disagree with this (and yes, I am a frequent github user,
username=lattner).

GitHub supports LLVM style development as it is today (well, but obviously using
git instead of svn).  A project can maintain a list of committers, and they have
direct push access to the repository.  This workflow obviously supports the
existing “send patch to -commits” or phabricator paths, and people with commit
access can apply patches for people without commit access.

The vastly superior thing about Github is that it gives you the *ability* to
also support pull request.  PR’s are totally awesome from a community outreach
support, because it drastically lowers that barrier for minor contributions to
the project from people who do not have commit access.  The workflow is
seriously a huge step up from what we do on llvm.org.

In any case, while I’m very positive about moving the program testsuite to
github, I consider it obvious that anyone with direct commit access to an
llvm.org repo would also have direct commit access to the program testsuite, and
we’d use the same policies for what kinds of changes are allowed.  PR’s would
just be a great new workflow to enable new kinds of people to make contributions
in an easier way.

-Chris