llvm dev - Jul 2016 - [RFC] One or many git repositories?

> On Jul 20, 2016, at 5:56 PM, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On 21 July 2016 at 01:39, Justin Lebar <jlebar at google.com> wrote:
>> This is true if you s/checkout/clone/.  With a single repo, you must
>> clone (download) everything (*), but after you've done so you can
use
>> sparse checkouts to check out (create a working copy of) only llvm and
>> clang.  So you should only notice the fact that there exist things
>> other than llvm and clang when you first clone (download) llvm.
> 
> So, we use that to a certain extent.
> 
> Linaro's GCC validation uses the full checkout, then do a shallow
> checkout that only has the updates.
> 
> Our LLVM scripts, OTOH, clone all repos and use worktree for *all*
> branches, and we only branch on the repos that we choose, for each
> "working dir".
> 
> Our scripts probably would need certain modifications... but it should be
fine.
> 
> But I'm not, by far, the most problematic user.
> 
> The real problem, and why people accepted sub-modules, is that a lot
> of downstream people only use one or another projects. Mostly LLVM or
> Clang or libc++.
> 
> Checking out all of it is bad, but having them officially interlinked,
> it seems, is worse. IIUC, the problem is that the projects are now
> built independently on their projects, but more and more CMake changes
> are creeping in, making it harder and harder to separate their
> projects from the rest of LLVM. This means they'll now depend on a
> much larger body of sources that will need to be compiled together,
> and will probably mean they'll abandon LLVM in favour of something
> lighter.
> 
> I honestly don't know how big is that problem, I don't have it
myself,
> but I "can imagine" compiling LLVM and Clang without need would
be
> pretty bad.

You seem to imply that all the projects in the single repo would be built by
default, while it is not part of the proposal.
Actually I’d expect an opt-in mechanism, so that: `mkdir build-llvm &&
cd build-llvm && cmake ../llvm` only builds LLVM.

— 
Mehdi

On Wed, Jul 20, 2016 at 6:02 PM Mehdi Amini via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
> > On Jul 20, 2016, at 5:56 PM, Renato Golin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > On 21 July 2016 at 01:39, Justin Lebar <jlebar at google.com>
wrote:
> >> This is true if you s/checkout/clone/.  With a single repo, you
must
> >> clone (download) everything (*), but after you've done so you
can use
> >> sparse checkouts to check out (create a working copy of) only llvm
and
> >> clang.  So you should only notice the fact that there exist things
> >> other than llvm and clang when you first clone (download) llvm.
> >
> > So, we use that to a certain extent.
> >
> > Linaro's GCC validation uses the full checkout, then do a shallow
> > checkout that only has the updates.
> >
> > Our LLVM scripts, OTOH, clone all repos and use worktree for *all*
> > branches, and we only branch on the repos that we choose, for each
> > "working dir".
> >
> > Our scripts probably would need certain modifications... but it should
> be fine.
> >
> > But I'm not, by far, the most problematic user.
> >
> > The real problem, and why people accepted sub-modules, is that a lot
> > of downstream people only use one or another projects. Mostly LLVM or
> > Clang or libc++.
> >
> > Checking out all of it is bad, but having them officially interlinked,
> > it seems, is worse. IIUC, the problem is that the projects are now
> > built independently on their projects, but more and more CMake changes
> > are creeping in, making it harder and harder to separate their
> > projects from the rest of LLVM. This means they'll now depend on a
> > much larger body of sources that will need to be compiled together,
> > and will probably mean they'll abandon LLVM in favour of something
> > lighter.
> >
> > I honestly don't know how big is that problem, I don't have it
myself,
> > but I "can imagine" compiling LLVM and Clang without need
would be
> > pretty bad.
>
>
> You seem to imply that all the projects in the single repo would be built
> by default, while it is not part of the proposal.
> Actually I’d expect an opt-in mechanism, so that: `mkdir build-llvm
&& cd
> build-llvm && cmake ../llvm` only builds LLVM.
>
If we end up with a single repository, I agree and think at least some
level of opt-in for building subprojects is essential.

I would expect at *most* to automatically enable building the set of
subprojects we currently suggest by default in the getting started docs.
Any more than that wouldn't make sense, and I could even imagine defaulting
*fewer* projects at the build system level.

>
> —
> Mehdi
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160721/24164825/attachment.html>

On 21 July 2016 at 02:05, Chandler Carruth <chandlerc at google.com>
wrote:
> If we end up with a single repository, I agree and think at least some
level
> of opt-in for building subprojects is essential.
We were originally trying to avoid too many moves at the same time.

There is already some CMake efforts to help build the different
repositories, but it's not linked to any proposal.

I think doing so would complicate both build system and version
control migrations...

--renato