llvm dev - Jul 2016 - [RFC] One or many git repositories?

On 29 July 2016 at 15:26, Robinson, Paul via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> I believe David Chisnall up-thread cited a difference in checkout times
> on the order of a handful of seconds versus a couple of minutes.  While
> naively it might seem not a big deal, over time and depending on what you
> are trying to do yes it can be a big burden.
TL;DR: This thread is dead. Let's move on.

I think the biggest fallacy in this thread is that changing process is cheap.

It is certainly cheap for me to do "git foo" instead of "git
bar" from
now on. It's moderately expensive to change my buildbot
configurations, Zorg's builders and re-test everything for public CI.
It's a lot more expensive to change how distributions build their
hundreds of thousands of packages over multiple LTS releases, or how
downstream users like Sony, Apple or ARM re-factor their entire build
systems (which very likely link to a lot of non-LLVM stuff), and then
some.

None of that is impossible, most of that is a "one off". Most of the
companies and big projects "could" afford to do that.

But there are two big points that people like me, Paul and David have
been unsuccessfully trying to make obvious:

1. Not every LLVM user is as big as FreeBSD, Sony or Apple. There are
a lot of very interesting projects (hobbyists, academia, professional)
using Clang, LLVM, libc++, etc. that don't have the staff to do that
move. Being a hobbyist myself, I know too well that, when a library
radically changes the way they behave (like boost did every new
release about 10 years ago), I will stop using it.

2. Changes in complex systems have unwanted larger consequences. Build
systems are some of the most complex systems in existence because
they're mostly irrational and patched together with duct tape and
paper clips. What may be very simple for some build systems, could be
impossible for others, and that's not the other's team's fault.

So, if you have a complex build system yourself, and you spent some
time and have figured out that it would be easy, you *cannot* assume
it should be easy for everyone with an less or equally complex build
systems.

If you find it simple to change your own workflow towards this or that
solution, you *cannot* assume everyone else should feel the same. This
also doesn't diminish their intelligence or competence. Intelligent
and competent people work in very different ways, and it's actually
because of that fact that we can do such complex software works in a
multitude of systems. If we were all equal, we wouldn't need to
discuss anything. :)

Mehdi said very early, and repeated many time, on some of the threads,
something to the effect of: "Saying how hard or easy it is for you is
an invalid argument, we need more concrete facts".

I absolutely agree with that statement, but interpreting how easy or
hard concrete facts would be fall on the same fallacy, so it doesn't
bring us closer to consensus, it brings us closer to dissent.

That is why I think this thread has already surpassed it's usefulness
(for a long time), and we need a concrete write up on the proposal. (I
hear it's in progress, let's wait for it).
>From now on, I'd propose the discussion to be *just* about this
specific proposal, preferably over a Phabricator review on the
document. People that have strong opinions about it should wait for
the survey.

Just to reiterate, the survey is to collect opinions in a formal and
non-passionate manner. It will not be a "majority vote", and we're
not
locked between these two solutions as they're absolutely drawn out in
the documents, nor we are forced to take any decision if the community
is clearly split. The last think I want is to destroy part of the
community while trying to make it better.

But this long thread is not doing any good either.

cheers,
--renato

> On Jul 29, 2016, at 7:52 AM, Renato Golin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> On 29 July 2016 at 15:26, Robinson, Paul via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> I believe David Chisnall up-thread cited a difference in checkout times
>> on the order of a handful of seconds versus a couple of minutes.  While
>> naively it might seem not a big deal, over time and depending on what
you
>> are trying to do yes it can be a big burden.
> 
> TL;DR: This thread is dead. Let's move on.
> 
> I think the biggest fallacy in this thread is that changing process is
cheap.
> 
> It is certainly cheap for me to do "git foo" instead of "git
bar" from
> now on. It's moderately expensive to change my buildbot
> configurations, Zorg's builders and re-test everything for public CI.
> It's a lot more expensive to change how distributions build their
> hundreds of thousands of packages over multiple LTS releases, or how
> downstream users like Sony, Apple or ARM re-factor their entire build
> systems (which very likely link to a lot of non-LLVM stuff), and then
> some.
> 
> None of that is impossible, most of that is a "one off". Most of
the
> companies and big projects "could" afford to do that.
> 
> But there are two big points that people like me, Paul and David have
> been unsuccessfully trying to make obvious:
> 
> 1. Not every LLVM user is as big as FreeBSD, Sony or Apple. There are
> a lot of very interesting projects (hobbyists, academia, professional)
> using Clang, LLVM, libc++, etc. that don't have the staff to do that
> move. Being a hobbyist myself, I know too well that, when a library
> radically changes the way they behave (like boost did every new
> release about 10 years ago), I will stop using it.
> 
> 2. Changes in complex systems have unwanted larger consequences. Build
> systems are some of the most complex systems in existence because
> they're mostly irrational and patched together with duct tape and
> paper clips. What may be very simple for some build systems, could be
> impossible for others, and that's not the other's team's fault.
> 
> So, if you have a complex build system yourself, and you spent some
> time and have figured out that it would be easy, you *cannot* assume
> it should be easy for everyone with an less or equally complex build
> systems.
> 
> If you find it simple to change your own workflow towards this or that
> solution, you *cannot* assume everyone else should feel the same. This
> also doesn't diminish their intelligence or competence. Intelligent
> and competent people work in very different ways, and it's actually
> because of that fact that we can do such complex software works in a
> multitude of systems. If we were all equal, we wouldn't need to
> discuss anything. :)
> 
> Mehdi said very early, and repeated many time, on some of the threads,
> something to the effect of: "Saying how hard or easy it is for you is
> an invalid argument, we need more concrete facts".
> 
> I absolutely agree with that statement, but interpreting how easy or
> hard concrete facts would be fall on the same fallacy, so it doesn't
> bring us closer to consensus, it brings us closer to dissent.
> 
> That is why I think this thread has already surpassed it's usefulness
> (for a long time), and we need a concrete write up on the proposal. (I
> hear it's in progress, let's wait for it).
> 
> From now on, I'd propose the discussion to be *just* about this
> specific proposal, preferably over a Phabricator review on the
> document. People that have strong opinions about it should wait for
> the survey.
I don’t see how this thread is dead or how it would help to stop the discussion
as you’re trying to do. I’m opposed to that.

People have still been raising concerns the last two days, and hearing about
these *now* will help to build a stronger proposal *before* getting to a survey.
They also help to design a survey that will ask the *right* set of questions.

Some people can be in vacations: Chris just got back and contributes with his
opinion the last two days, Lang yesterday, others may follow, their views are
valuable.

So please keep this thread alive, and wether I agree or not with the opinions
expressed, I still want to hear more of these because they help pointing at the
real weaknesses of the proposal, and eventually adjust it.

— 
Mehdi

> 
> Just to reiterate, the survey is to collect opinions in a formal and
> non-passionate manner. It will not be a "majority vote", and
we're not
> locked between these two solutions as they're absolutely drawn out in
> the documents, nor we are forced to take any decision if the community
> is clearly split. The last think I want is to destroy part of the
> community while trying to make it better.
> 
> But this long thread is not doing any good either.
> 
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 29 July 2016 at 18:14, Mehdi Amini <mehdi.amini at apple.com>
wrote:
> I don’t see how this thread is dead or how it would help to stop the
discussion as you’re trying to do. I’m opposed to that.
I'm not trying to stop discussions, I'm trying to help them be more
effective.

What I propose that you do, and what we've done on the sub-modules
thread, is to split the discussion into sub-threads. There are too
many concerns, and a lot of irrelevant ones, being discussed in this
thread.

Writing the document will help, but there's a lot of good discussions
in this thread (in the last two days included) that not everyone is
paying attention (as they should). These discussions would be too much
for a document review, I think, so the list is still the best place we
can do this.

But a thread that is nearing 200 emails is less helpful than small
ones. It also encourages people to digress more than a few focused
discussions.
>From what I could pay attention, the current issues are:
1. What goes where? Where in the spectrum between today and a single
mono-repo do we stand, and why? This will affect the success of this
proposal, so needs to be ironed out before we do the survey.

2. What combination of mono-repo and shallow/sparse checkout do we
stand for the different scenarios? This one depends on the one above,
but it's more about projects' infrastructure and how can they adapt to
whatever new repo layout we choose.

> So please keep this thread alive, and wether I agree or not with the
opinions expressed, I still want to hear more of these because they help
pointing at the real weaknesses of the proposal, and eventually adjust it.
That's great! Can you start a few new threads on specific subjects,
summarising what has been discussed so far *only* for those topics?

thanks!
--renato