> On May 31, 2016, at 1:31 PM, Renato Golin <renato.golin at linaro.org> wrote: > > On 31 May 2016 at 21:28, Mehdi Amini <mehdi.amini at apple.com> wrote: >> Ideally, I'd prefer the cross-repository to be handled with an extra layer, in a way similar as described in: https://gerrit-review.googlesource.com/Documentation/user-submodules.htm (somehow conceptually similar to Android manifests XML files). >> It would be easy to have tooling/scripts for llvm that would easily say "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all llvm subproject under this root", or "checkout this specific revision for all these" (with a monotonic number for the revision). > > At Linaro, we already have a set of scripts that do that. We're now > moving to git worktree, and I think it's going to simplify our work > considerably. But honestly, I'd rather not force anyone to use any set > of scripts, and let people work directly with git, so I'd be more in > favour of having a server-side solution, if at all possible.Apparently I wasn't very clear: llvm and clang (and the others projects) would be simple decoupled, individual git repositories. You would be able to check them out however you want and commit to them individually. There would be an extra "integration repository" on top that would only provide the service that tells "r12345 is llvm:36c941c clang:eaf492b compiler-rt:6d77ea5". This repository should be managed transparently by some server-side integration. The provided scripting I was referring to would just be a convenience that is using this extra layer of metadata ("integration repository") to be able checkout the other individual repositories together at the right "rev-lock" revision. This is not on your way if you don't want to use it, but it provides this "single increase monotonic revision number across multiple repository" that is convenient for some people. Makes sense? -- Medhi
On 31 May 2016 at 13:45, Mehdi Amini via lldb-dev <lldb-dev at lists.llvm.org> wrote:> Apparently I wasn't very clear: llvm and clang (and the others projects) would be simple decoupled, individual git repositories. You would be able to check them out however you want and commit to them individually. > There would be an extra "integration repository" on top that would only provide the service that tells "r12345 is llvm:36c941c clang:eaf492b compiler-rt:6d77ea5". This repository should be managed transparently by some server-side integration.This actually sounds like a really good idea even if a full move to git gets blocked for some reason. It seems like it could be a fairly common requirement: I don't suppose you know of an existing script that could do it? If not, I may take a stab. Tim.
> On May 31, 2016, at 2:07 PM, Tim Northover <t.p.northover at gmail.com> wrote: > > On 31 May 2016 at 13:45, Mehdi Amini via lldb-dev > <lldb-dev at lists.llvm.org> wrote: >> Apparently I wasn't very clear: llvm and clang (and the others projects) would be simple decoupled, individual git repositories. You would be able to check them out however you want and commit to them individually. >> There would be an extra "integration repository" on top that would only provide the service that tells "r12345 is llvm:36c941c clang:eaf492b compiler-rt:6d77ea5". This repository should be managed transparently by some server-side integration. > > This actually sounds like a really good idea even if a full move to > git gets blocked for some reason. It seems like it could be a fairly > common requirement: I don't suppose you know of an existing script > that could do it? If not, I may take a stab.I don't know of a script that would update this automatically, but the Gerrit feature seems pretty close: https://gerrit-review.googlesource.com/Documentation/user-submodules.html ; and it is possible that some Android tooling exists around the repo tool: https://source.android.com/source/developing.html I think it is fairly easy to setup tools that do that, it is harder to do it reliably (i.e. handle concurrent push in clang and llvm for instance), or what if the server-side script fails the update because of a network issue? Maybe it is not very important and one commit won't be part of the "global numbering"? Alternatively the push can be "failed" in such cases? -- Mehdi
On 5/31/2016 4:46 PM, Mehdi Amini via cfe-dev wrote:> Apparently I wasn't very clear: llvm and clang (and the others projects) would be simple decoupled, individual git repositories. You would be able to check them out however you want and commit to them individually. > There would be an extra "integration repository" on top that would only provide the service that tells "r12345 is llvm:36c941c clang:eaf492b compiler-rt:6d77ea5". This repository should be managed transparently by some server-side integration. > The provided scripting I was referring to would just be a convenience that is using this extra layer of metadata ("integration repository") to be able checkout the other individual repositories together at the right "rev-lock" revision. > This is not on your way if you don't want to use it, but it provides this "single increase monotonic revision number across multiple repository" that is convenient for some people. > > Makes sense?Yes, makes sense; we have been doing exactly this for the last few months. We created our own integration repo (to host our own build integration scripts) and cloned the llvm and clang repos from (I assume) https://github.com/llvm-mirror as sub-modules within it. We're just using the native git command line to manage things and, so far, so good. We're still working on getting a full continuous integration process in place (right now we manually pull periodically), but expect to have that soon. The CI process is just to inform us of conflicts and allow us to resolve them proactively; we don't release product based on trunk. Tom.
On 2016-05-31 22:45, Mehdi Amini via llvm-dev wrote:>> On May 31, 2016, at 1:31 PM, Renato Golin <renato.golin at linaro.org> >> wrote: >> >> On 31 May 2016 at 21:28, Mehdi Amini <mehdi.amini at apple.com> wrote: >>> Ideally, I'd prefer the cross-repository to be handled with an extra >>> layer, in a way similar as described in: >>> https://gerrit-review.googlesource.com/Documentation/user-submodules.htm >>> (somehow conceptually similar to Android manifests XML files). >>> It would be easy to have tooling/scripts for llvm that would easily >>> say "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or >>> "update all llvm subproject under this root", or "checkout this >>> specific revision for all these" (with a monotonic number for the >>> revision). >> >> At Linaro, we already have a set of scripts that do that. We're now >> moving to git worktree, and I think it's going to simplify our work >> considerably. But honestly, I'd rather not force anyone to use any set >> of scripts, and let people work directly with git, so I'd be more in >> favour of having a server-side solution, if at all possible. > > Apparently I wasn't very clear: llvm and clang (and the others > projects) would be simple decoupled, individual git repositories. You > would be able to check them out however you want and commit to them > individually. > There would be an extra "integration repository" on top that would > only provide the service that tells "r12345 is llvm:36c941c > clang:eaf492b compiler-rt:6d77ea5". This repository should be managed > transparently by some server-side integration. > The provided scripting I was referring to would just be a convenience > that is using this extra layer of metadata ("integration repository") > to be able checkout the other individual repositories together at the > right "rev-lock" revision. > This is not on your way if you don't want to use it, but it provides > this "single increase monotonic revision number across multiple > repository" that is convenient for some people. > > Makes sense?How would you ensure that two dependent changes on different repositories get the same revision number?
On Wed, Jun 1, 2016 at 1:07 PM, Manuel Jacob via llvm-dev < llvm-dev at lists.llvm.org> wrote:> On 2016-05-31 22:45, Mehdi Amini via llvm-dev wrote: > >> On May 31, 2016, at 1:31 PM, Renato Golin <renato.golin at linaro.org> >>> wrote: >>> >>> On 31 May 2016 at 21:28, Mehdi Amini <mehdi.amini at apple.com> wrote: >>> >>>> Ideally, I'd prefer the cross-repository to be handled with an extra >>>> layer, in a way similar as described in: >>>> https://gerrit-review.googlesource.com/Documentation/user-submodules.htm >>>> (somehow conceptually similar to Android manifests XML files). >>>> It would be easy to have tooling/scripts for llvm that would easily say >>>> "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all >>>> llvm subproject under this root", or "checkout this specific revision for >>>> all these" (with a monotonic number for the revision). >>>> >>> >>> At Linaro, we already have a set of scripts that do that. We're now >>> moving to git worktree, and I think it's going to simplify our work >>> considerably. But honestly, I'd rather not force anyone to use any set >>> of scripts, and let people work directly with git, so I'd be more in >>> favour of having a server-side solution, if at all possible. >>> >> >> Apparently I wasn't very clear: llvm and clang (and the others >> projects) would be simple decoupled, individual git repositories. You >> would be able to check them out however you want and commit to them >> individually. >> There would be an extra "integration repository" on top that would >> only provide the service that tells "r12345 is llvm:36c941c >> clang:eaf492b compiler-rt:6d77ea5". This repository should be managed >> transparently by some server-side integration. >> The provided scripting I was referring to would just be a convenience >> that is using this extra layer of metadata ("integration repository") >> to be able checkout the other individual repositories together at the >> right "rev-lock" revision. >> This is not on your way if you don't want to use it, but it provides >> this "single increase monotonic revision number across multiple >> repository" that is convenient for some people. >> >> Makes sense? >> > > How would you ensure that two dependent changes on different repositories > get the same revision number?That is not the case today and isn't (in my opinion) a problem we need to solve with this migration. If there's a small (1-2 revision) window in which things are broken, that's annoying but not a showstopper and not a regression. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160601/4e3178d4/attachment.html>
> On Jun 1, 2016, at 1:07 PM, Manuel Jacob <me at manueljacob.de> wrote: > > On 2016-05-31 22:45, Mehdi Amini via llvm-dev wrote: >>> On May 31, 2016, at 1:31 PM, Renato Golin <renato.golin at linaro.org> wrote: >>>> On 31 May 2016 at 21:28, Mehdi Amini <mehdi.amini at apple.com> wrote: >>>> Ideally, I'd prefer the cross-repository to be handled with an extra layer, in a way similar as described in: https://gerrit-review.googlesource.com/Documentation/user-submodules.htm (somehow conceptually similar to Android manifests XML files). >>>> It would be easy to have tooling/scripts for llvm that would easily say "checkout llvm+clang+compiler-rt+libcxx+clang-extra here", or "update all llvm subproject under this root", or "checkout this specific revision for all these" (with a monotonic number for the revision). >>> At Linaro, we already have a set of scripts that do that. We're now >>> moving to git worktree, and I think it's going to simplify our work >>> considerably. But honestly, I'd rather not force anyone to use any set >>> of scripts, and let people work directly with git, so I'd be more in >>> favour of having a server-side solution, if at all possible. >> Apparently I wasn't very clear: llvm and clang (and the others >> projects) would be simple decoupled, individual git repositories. You >> would be able to check them out however you want and commit to them >> individually. >> There would be an extra "integration repository" on top that would >> only provide the service that tells "r12345 is llvm:36c941c >> clang:eaf492b compiler-rt:6d77ea5". This repository should be managed >> transparently by some server-side integration. >> The provided scripting I was referring to would just be a convenience >> that is using this extra layer of metadata ("integration repository") >> to be able checkout the other individual repositories together at the >> right "rev-lock" revision. >> This is not on your way if you don't want to use it, but it provides >> this "single increase monotonic revision number across multiple >> repository" that is convenient for some people. >> Makes sense? > > How would you ensure that two dependent changes on different repositories get the same revision number?I don't plan to try to address that. I plan to reproduce the behavior we have right now when someone commit an API change in LLVM and separately commit the fix in clang: the fix gets a different revision number. I'm not sure how to be robust against that other than putting all the projects in the same repo and asking developers to build them all before push. If you have ideas/suggestions, please share! -- Mehdi
Tim Northover via cfe-dev <cfe-dev at lists.llvm.org> writes:> On 31 May 2016 at 13:45, Mehdi Amini via lldb-dev > <lldb-dev at lists.llvm.org> wrote: >> Apparently I wasn't very clear: llvm and clang (and the others >> projects) would be simple decoupled, individual git >> repositories. You would be able to check them out however you want >> and commit to them individually. >> There would be an extra "integration repository" on top that would >> only provide the service that tells "r12345 is llvm:36c941c >> clang:eaf492b compiler-rt:6d77ea5". This repository should be >> managed transparently by some server-side integration. > > This actually sounds like a really good idea even if a full move to > git gets blocked for some reason. It seems like it could be a fairly > common requirement: I don't suppose you know of an existing script > that could do it? If not, I may take a stab.How do you get monotonically increasing number with a history graph? There are multiple ways to linearize the history. You can simply disallow merges I guess but that seems not much better than just sticking with SVN. GitHub's pull request model kind of breaks down if you can't do merges. -David