> On May 31, 2016, at 1:16 PM, Chris Lattner via llvm-dev <llvm-dev at lists.llvm.org> wrote: > >> On May 31, 2016, at 12:31 PM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> There has been some discussion on IRC about SVN hosting and the perils >> of doing it ourselves. The consensus on the current discussion was >> that moving to a Git-only solution would have some disvantages, but >> many advantages. Furthermore, not hosting our own repos would save us >> a lot of headaches, admin costs and timed out connections. > > Personally, I’m hugely in favor of moving llvm’s source hosting to github at some point, despite the fact that I continue to dislike git as a tool and consider monotonicly increasing version numbers to be hugely beneficial.Getting a monotonically increasing revision number seems doable in git with some server-side scripting using git notes or named tags (yet to be seen is how to achieve it *reliably* with github hosting). However the challenge is more about sharing this number across repositories (i.e. having clang and llvm in sync). I can imagine some tooling for that, but with a github hosting it may still be fragile. Ideally, I'd prefer the cross-repository to be handled with an extra layer, in a way similar as described in: https://gerrit-review.googlesource.com/Documentation/user-submodules.htm (somehow conceptually similar to Android manifests XML files). It would be easy to have tooling/scripts for llvm that would easily say "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all llvm subproject under this root", or "checkout this specific revision for all these" (with a monotonic number for the revision). (+1 to all the rest of what you wrote) -- Mehdi> The killer feature to me is the community aspects of github, allowing people to get involved in the project more easily and make “drive by” contributions through the pull request model. Github also has a very scriptable interface, allowing integration of external bug trackers etc into the workflow (which is good, because its bugtracker is anemic). > >> 4. We currently host our own SVN/Git, ViewVC and Klaus, Phabricator, >> etc. Not only this incurs in additional admin cost, but it also gets >> outdated, locally modified, and it needs to be backed up, etc. GitHub >> gives all that for us for free. > > Yes, it would be great to get out of this business. > >> 5. We can still use Bugzilla (and lock GitHub's own bug system), but >> we can also use GitHub's system to manage releases (it's actually >> quite good for that). > > If we made this change, I think we should only change one thing at a time: change source hosting, but not phabricator or the bug tracker. We could then discuss moving off phabricator to the github PR model, etc. > >> 6. GitHub has automated testing of merge requests, meaning we can have >> pre-commit tests enabled on a set of fast bots, triggered by GitHub's >> own validation hooks. > > This works pretty well. The major problem is with tests that are flakey. > > -Chris > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
On 31 May 2016 at 21:28, Mehdi Amini <mehdi.amini at apple.com> wrote:> Ideally, I'd prefer the cross-repository to be handled with an extra layer, in a way similar as described in: https://gerrit-review.googlesource.com/Documentation/user-submodules.htm (somehow conceptually similar to Android manifests XML files). > It would be easy to have tooling/scripts for llvm that would easily say "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all llvm subproject under this root", or "checkout this specific revision for all these" (with a monotonic number for the revision).At Linaro, we already have a set of scripts that do that. We're now moving to git worktree, and I think it's going to simplify our work considerably. But honestly, I'd rather not force anyone to use any set of scripts, and let people work directly with git, so I'd be more in favour of having a server-side solution, if at all possible. cheers, --renato
Strong +1 to move to an external hosted git sooner rather than later!
1) I personally had very good experiences with git submodules. They are
certainly harder to get used to as you have to learn a bunch of extra magic on
top of the already magical git: i.e. "git clone --recurse-submodules",
then learn how to have your submodules point to different commits locally
sometimes, etc.
I have had very good experience in another project that used to do llvm/clang
style "just checkout those two project at the same date" and I found
submodules more stable and robust and technically superior solution at the cost
of a higher bar learning curve for new contributors.
So in this context I would recommend to change one thing at a time and only
switch svn->git in step 1 and leave the switch to submodules as a 2nd step
(or not do it at all if that is community consensus).
2) As far as the "increasing revision" numbers go: In my opinion this
is about:
- We really should stay with a linear history and not introduce merge
commits.
- As long as we do not move to submodules we need a solution to enforce
"CommitDate"s (or also "AuthorDate"s) to be the time a
commit was pushed to the server so our current workflow of checking out
llvm/clang at the same time still works.
I believe that those two points to be solvable with some server side
scripting. With those two properties in place actual increasing numeric revision
numbers bring that much value to the table and I would assume we can go without
them.
- Matthias
> On May 31, 2016, at 1:28 PM, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>
>>
>> On May 31, 2016, at 1:16 PM, Chris Lattner via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>>> On May 31, 2016, at 12:31 PM, Renato Golin via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
>>> There has been some discussion on IRC about SVN hosting and the
perils
>>> of doing it ourselves. The consensus on the current discussion was
>>> that moving to a Git-only solution would have some disvantages, but
>>> many advantages. Furthermore, not hosting our own repos would save
us
>>> a lot of headaches, admin costs and timed out connections.
>>
>> Personally, I’m hugely in favor of moving llvm’s source hosting to
github at some point, despite the fact that I continue to dislike git as a tool
and consider monotonicly increasing version numbers to be hugely beneficial.
>
> Getting a monotonically increasing revision number seems doable in git with
some server-side scripting using git notes or named tags (yet to be seen is how
to achieve it *reliably* with github hosting).
> However the challenge is more about sharing this number across repositories
(i.e. having clang and llvm in sync). I can imagine some tooling for that, but
with a github hosting it may still be fragile.
>
> Ideally, I'd prefer the cross-repository to be handled with an extra
layer, in a way similar as described in:
https://gerrit-review.googlesource.com/Documentation/user-submodules.htm
<https://gerrit-review.googlesource.com/Documentation/user-submodules.htm>
(somehow conceptually similar to Android manifests XML files).
> It would be easy to have tooling/scripts for llvm that would easily say
"checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or
"update all llvm subproject under this root", or "checkout this
specific revision for all these" (with a monotonic number for the
revision).
>
> (+1 to all the rest of what you wrote)
>
> --
> Mehdi
>
>
>> The killer feature to me is the community aspects of github, allowing
people to get involved in the project more easily and make “drive by”
contributions through the pull request model. Github also has a very scriptable
interface, allowing integration of external bug trackers etc into the workflow
(which is good, because its bugtracker is anemic).
>>
>>> 4. We currently host our own SVN/Git, ViewVC and Klaus,
Phabricator,
>>> etc. Not only this incurs in additional admin cost, but it also
gets
>>> outdated, locally modified, and it needs to be backed up, etc.
GitHub
>>> gives all that for us for free.
>>
>> Yes, it would be great to get out of this business.
>>
>>> 5. We can still use Bugzilla (and lock GitHub's own bug
system), but
>>> we can also use GitHub's system to manage releases (it's
actually
>>> quite good for that).
>>
>> If we made this change, I think we should only change one thing at a
time: change source hosting, but not phabricator or the bug tracker. We could
then discuss moving off phabricator to the github PR model, etc.
>>
>>> 6. GitHub has automated testing of merge requests, meaning we can
have
>>> pre-commit tests enabled on a set of fast bots, triggered by
GitHub's
>>> own validation hooks.
>>
>> This works pretty well. The major problem is with tests that are
flakey.
>>
>> -Chris
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160531/99ce524f/attachment.html>
> On May 31, 2016, at 1:31 PM, Renato Golin <renato.golin at linaro.org> wrote: > > On 31 May 2016 at 21:28, Mehdi Amini <mehdi.amini at apple.com> wrote: >> Ideally, I'd prefer the cross-repository to be handled with an extra layer, in a way similar as described in: https://gerrit-review.googlesource.com/Documentation/user-submodules.htm (somehow conceptually similar to Android manifests XML files). >> It would be easy to have tooling/scripts for llvm that would easily say "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all llvm subproject under this root", or "checkout this specific revision for all these" (with a monotonic number for the revision). > > At Linaro, we already have a set of scripts that do that. We're now > moving to git worktree, and I think it's going to simplify our work > considerably. But honestly, I'd rather not force anyone to use any set > of scripts, and let people work directly with git, so I'd be more in > favour of having a server-side solution, if at all possible.Apparently I wasn't very clear: llvm and clang (and the others projects) would be simple decoupled, individual git repositories. You would be able to check them out however you want and commit to them individually. There would be an extra "integration repository" on top that would only provide the service that tells "r12345 is llvm:36c941c clang:eaf492b compiler-rt:6d77ea5". This repository should be managed transparently by some server-side integration. The provided scripting I was referring to would just be a convenience that is using this extra layer of metadata ("integration repository") to be able checkout the other individual repositories together at the right "rev-lock" revision. This is not on your way if you don't want to use it, but it provides this "single increase monotonic revision number across multiple repository" that is convenient for some people. Makes sense? -- Medhi
Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> writes:>> Personally, I’m hugely in favor of moving llvm’s source hosting to >> github at some point, despite the fact that I continue to dislike >> git as a tool and consider monotonicly increasing version numbers to >> be hugely beneficial. > > Getting a monotonically increasing revision number seems doable in git > with some server-side scripting using git notes or named tags (yet to > be seen is how to achieve it *reliably* with github hosting). > However the challenge is more about sharing this number across > repositories (i.e. having clang and llvm in sync). I can imagine some > tooling for that, but with a github hosting it may still be fragile.What exactly is the concrete benefit of monotonically increasing revision numbers? It's completely foreign to git's architecture. Putting this requirement on git is going to severely limit how the history is allowed to look. Maybe that's what people want, I don't know. We certainly haven't missed them since moving to git. -David
> On Jun 2, 2016, at 11:01 AM, dag at cray.com wrote: > > Mehdi Amini via llvm-dev <llvm-dev at lists.llvm.org> writes: > >>> Personally, I’m hugely in favor of moving llvm’s source hosting to >>> github at some point, despite the fact that I continue to dislike >>> git as a tool and consider monotonicly increasing version numbers to >>> be hugely beneficial. >> >> Getting a monotonically increasing revision number seems doable in git >> with some server-side scripting using git notes or named tags (yet to >> be seen is how to achieve it *reliably* with github hosting). >> However the challenge is more about sharing this number across >> repositories (i.e. having clang and llvm in sync). I can imagine some >> tooling for that, but with a github hosting it may still be fragile. > > What exactly is the concrete benefit of monotonically increasing > revision numbers? It's completely foreign to git's architecture.I'm notified that bug is fixed in r12345, my binary says "clang -v" is r23456, if I observe the bug I know there is a problem immediately. Another use case is that it provides a cross-repository reference. I'm sure that if you lookup the previous discussion about moving to git (two years ago, and four years ago I believe), the threads contain plenty of arguments from people attached to it. I don't want to play the "devil's advocate" here since I'm not that bothered by loosing this.> > Putting this requirement on git is going to severely limit how the > history is allowed to look. Maybe that's what people want, I don't > know. We certainly haven't missed them since moving to git.I'm personally fine with arbitrary history in git, but some people (I suspect many here) are attached to the linear history. So my impression is that such a move has more chances of being accepted by the community if we keep the same workflow (linear history + monotonic revision numbers), at least at first. -- Mehdi
On Jun 2, 2016, at 11:01 AM, dag at cray.com wrote:> > What exactly is the concrete benefit of monotonically increasing > revision numbers? It's completely foreign to git's architecture. > > Putting this requirement on git is going to severely limit how the > history is allowed to look. Maybe that's what people want, I don't > know. We certainly haven't missed them since moving to git.I think we *want* to severely limit how history is allowed to look. We don’t want the entire development history of your long-lived branch coming to mainline. When a patch is accepted to an llvm.org repository, it is committed as an atomic unit. It is expected to be reviewed (pre or post), include testcases, and uphold the standards of the project *in that unit*. There is no value in providing additional details. If there were, then the patch should be committed as a linear series of independent pieces, each of which lives up to the project’s standards. -Chris