llvm dev - Jul 2016 - [RFC] One or many git repositories?

> On 29 Jul 2016, at 19:19, David Chisnall via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On 29 Jul 2016, at 05:11, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>> 
>> What I meant by “different problem" is that “downstream users” for
instance don’t need to commit, that makes their problem/workflow quite different
from an upstream developer (for instance it is fairly easy to maintain a
read-only view of the existing individual git repo currently on llvm.org).
> 
> I’m not convinced by this distinction.  A lot of downstream developers need
to patch LLVM and we benefit when they upstream their changes.  We should not
make it harder for them to do this.  To give a couple of example downstream
projects, both FreeBSD and Swift have patches on LLVM / Clang in their versions
that they gradually filter upstream.  Both projects have LLVM committers among
their members.  If the workflow that we recommend for them makes upstreaming
easy then they benefit (maintaining a fork is effort) and LLVM benefits (having
people provide bug fixes makes our code better).
> 
> The workflow that we want to recommend to these people is:
> 
> - Fork the repo that you’re interested in from the LLVM GitHub organisation
> - Make your changes
> - Send pull requests for anything that you think is of interest to upstream
> 
I understand this, but why isn't "the repo you're interested
in" just the megarepo (or monorepo) where every LLVM project resides?
> This makes the barrier to entry for sending code back upstream *much* lower
than it currently is, to the benefit of all.  If the alternative is:
> 
> - Fork a read-only repo that you’re interested in from the LLVM GitHub
organisation
> - Make your changes
> - Fork a different repo from the LLVM GitHub organisation
> - Run a script to filter some of your changes into that one
> - Send a pull request from that
> - Deal with merging between the two yourself
> 
> I strongly suspect that we’ll get a lot fewer useful contributions from
downstream.  Or downstream people will just work on the monorepo and eat the
cost.
> 
It isn't -- for downstream users of any of the LLVM projects, I suspect the
answer will just be "instead of forking N repositories to get the benefit
from these N projects, just fork the megarepo".

If I was a downstream user, this sounds like a simpler proposition *even if
I'm only interested in one part of the overall LLVM project*.
> If someone is working on a downstream LLVM project and becoming familiar
with our codebase, then we want them to be subtly nudging their workflow so that
they eventually become LLVM contributors without noticing!
> 
Indeed. The best way I think, all things considered, is that we have a single
megarepo where everything LLVM is in there. That way in case anybody wants to
make any changes to any part of it, it's a simpler process _especially
compared to the status quo_.

Cheers

On 29 Jul 2016, at 12:35, Dean Michael Berris <dean.berris at gmail.com>
wrote:
> 
> I understand this, but why isn't "the repo you're interested
in" just the megarepo (or monorepo) where every LLVM project resides?
Your assumption is a downstream user of LLVM.  As previously pointed out, we
have downstream users of libc++ and the sanitizer runtimes who compile with gcc.
For a downstream user of LLVM, the cost of getting everything else is in the
noise.  For a downstream user of libc++ who may want to contribute upstream, the
overhead is huge.

David

> On 29 Jul 2016, at 21:58, David Chisnall <david.chisnall at
cl.cam.ac.uk> wrote:
> 
> On 29 Jul 2016, at 12:35, Dean Michael Berris <dean.berris at
gmail.com> wrote:
>> 
>> I understand this, but why isn't "the repo you're
interested in" just the megarepo (or monorepo) where every LLVM project
resides?
> 
> Your assumption is a downstream user of LLVM.  As previously pointed out,
we have downstream users of libc++ and the sanitizer runtimes who compile with
gcc.  For a downstream user of LLVM, the cost of getting everything else is in
the noise.  For a downstream user of libc++ who may want to contribute upstream,
the overhead is huge.
> 
Even then, are we seriously ignoring the fact that even if you did clone the
whole repository including everything, that you can still build just the libc++
and sanitiser runtimes if you wanted to? Why is this "noise" of any
importance to the users who get what they want and then some?

I know some people use only numbered releases of LLVM and the projects. They can
keep using those as long as LLVM provides them.

Is it really impossible to just build non-LLVM dependent versions of libc++ or
the sanitiser runtimes if they reside in one git megarepo?