llvm dev - Jul 2020 - RFC: Adding a staging branch (temporarily) to facilitate upstreaming

On 6/30/20 2:07 PM, Chris Lattner via llvm-dev wrote:
>
>
>> On Jun 30, 2020, at 2:02 PM, Duncan Exon Smith <dexonsmith at
apple.com
>> <mailto:dexonsmith at apple.com>> wrote:
>>
>>
>>
>>> On 2020-Jun-30, at 13:28, Chris Lattner via llvm-dev 
>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>>>
>>>> On Jun 29, 2020, at 10:15 PM, Chandler Carruth via llvm-dev 
>>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>>>>
>>>> IMO, a pull request isn't as clear given that they
haven't been
>>>> used for contributions before. This is not a time to be
innovative
>>>> IMO. A branch as a staging location has been used many times
over
>>>> the history of the project though and seems nicely unambiguous
in
>>>> that regard.
>>>
>>> I don’t have a opinion on this either way, but can git/GitHub 
>>> maintain forks within the same organization?  You could have 
>>> llvm/llvm-project and llvm/llvm-project-apple-staging or something 
>>> like that?
>>
>> I don't think GitHub allows you fork your own repo so I think it 
>> would be disconnected from a GitHub point of view. That has a few 
>> downsides, although I'm not sure how important they are.
>>
>> Regardless, if a separate repo is preferred, then a better name from 
>> our perspective would be "llvm-project-staging" (dropping the
>> "-apple" suffix). We could push a "staging/apple"
branch there.
>
> Ok, I’m not very concerned either way, it was just a thought.  I’m 
> very happy to see this upstreaming work happen, thanks!
>
> -Chris
I have a mild preference for the separate llvm-project-staging approach, 
but am not opposed to an in tree branch either.  The main argument I see 
for the separate repo is that the bar can be lower because the 
consequences for being "wrong" about the code being fully merged
quickly
are lower.

Or another thought, maybe we should even use the incubator flow here?  
Nothing says an incubator has to be long lived.  If we spun up an 
"incubator-staging-apple" repo, wouldn't that demonstrate the same
benefits?

Philip


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200701/f4ca022a/attachment.html>

The consequences of pushing a new branch name and then deleting it later
should be low. And if we really wanted to exclude it from the default pull
set, it can be placed in, "refs/staging/apple" instead of
"refs/heads/staging/apple". Nothing outside refs/heads/ and refs/tags/
gets
pulled by default. This *should* be the obvious answer. However, we've had
cases in the past that when people have accidentally created a branch or
opened a pull request (which creates a new branch in "refs/pull/NNN"),
which triggered phabricator to start sending a bunch of effectively-spam
emails. I believe that may still be a problem.

Unless someone can confirm that this *won't* happen, it's probably *not*
a
good idea to push 30,000 "new" old commits into any branch in the
repo.

I'd also concur with the other comments that it really feels quite silly to
do have to do anything technical here at all, versus posting an email to
llvm-dev along the lines of "Apple is contributing the code added in
github.com/apple/llvm-project, commit hash
a1fde6dadf210c937c88509ab775610213e2cfc5, and all prior commits it depends
on, under the Apache2 license with LLVM project exceptions, as listed in
https://github.com/apple/llvm-project/blob/a1fde6dadf210c937c88509ab775610213e2cfc5/LICENSE.txt".
That seems like it'd be sufficient -- and even more explicit as a statement
of intent than creating a branch! (But, of course, IANAL).

Or -- how about both sending such a message *and* creating a phab review
for `git diff origin/master...swift/apple/master` (three dots diff gets the
diff only from the last merge-point). Maybe that covers all the bases
sufficiently?

On Wed, Jul 1, 2020 at 1:19 PM Philip Reames via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
> On 6/30/20 2:07 PM, Chris Lattner via llvm-dev wrote:
>
>
>
> On Jun 30, 2020, at 2:02 PM, Duncan Exon Smith <dexonsmith at
apple.com>
> wrote:
>
>
>
> On 2020-Jun-30, at 13:28, Chris Lattner via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> On Jun 29, 2020, at 10:15 PM, Chandler Carruth via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>
> IMO, a pull request isn't as clear given that they haven't been
used for
> contributions before. This is not a time to be innovative IMO. A branch as
> a staging location has been used many times over the history of the project
> though and seems nicely unambiguous in that regard.
>
>
> I don’t have a opinion on this either way, but can git/GitHub maintain
> forks within the same organization?  You could have llvm/llvm-project and
> llvm/llvm-project-apple-staging or something like that?
>
>
> I don't think GitHub allows you fork your own repo so I think it would
be
> disconnected from a GitHub point of view. That has a few downsides,
> although I'm not sure how important they are.
>
> Regardless, if a separate repo is preferred, then a better name from our
> perspective would be "llvm-project-staging" (dropping the
"-apple" suffix).
> We could push a "staging/apple" branch there.
>
>
> Ok, I’m not very concerned either way, it was just a thought.  I’m very
> happy to see this upstreaming work happen, thanks!
>
> -Chris
>
> I have a mild preference for the separate llvm-project-staging approach,
> but am not opposed to an in tree branch either.  The main argument I see
> for the separate repo is that the bar can be lower because the consequences
> for being "wrong" about the code being fully merged quickly are
lower.
>
> Or another thought, maybe we should even use the incubator flow here?
> Nothing says an incubator has to be long lived.  If we spun up an
> "incubator-staging-apple" repo, wouldn't that demonstrate the
same benefits?
>
> Philip
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200701/832575e7/attachment.html>

On Wed, Jul 1, 2020 at 10:20 AM Philip Reames via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
> On 6/30/20 2:07 PM, Chris Lattner via llvm-dev wrote:
>
>
>
> On Jun 30, 2020, at 2:02 PM, Duncan Exon Smith <dexonsmith at
apple.com>
> wrote:
>
>
>
> On 2020-Jun-30, at 13:28, Chris Lattner via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> On Jun 29, 2020, at 10:15 PM, Chandler Carruth via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>
> IMO, a pull request isn't as clear given that they haven't been
used for
> contributions before. This is not a time to be innovative IMO. A branch as
> a staging location has been used many times over the history of the project
> though and seems nicely unambiguous in that regard.
>
>
> I don’t have a opinion on this either way, but can git/GitHub maintain
> forks within the same organization?  You could have llvm/llvm-project and
> llvm/llvm-project-apple-staging or something like that?
>
>
> I don't think GitHub allows you fork your own repo so I think it would
be
> disconnected from a GitHub point of view. That has a few downsides,
> although I'm not sure how important they are.
>
> Regardless, if a separate repo is preferred, then a better name from our
> perspective would be "llvm-project-staging" (dropping the
"-apple" suffix).
> We could push a "staging/apple" branch there.
>
>
> Ok, I’m not very concerned either way, it was just a thought.  I’m very
> happy to see this upstreaming work happen, thanks!
>
> -Chris
>
> I have a mild preference for the separate llvm-project-staging approach,
> but am not opposed to an in tree branch either.  The main argument I see
> for the separate repo is that the bar can be lower because the consequences
> for being "wrong" about the code being fully merged quickly are
lower.
>
> Or another thought, maybe we should even use the incubator flow here?
> Nothing says an incubator has to be long lived.  If we spun up an
> "incubator-staging-apple" repo, wouldn't that demonstrate the
same benefits?
>
I am not convinced the "incubator" proposal is suited for this purpose
as
this is not about a new project but about a patch on top of LLVM itself. My
understanding of the incubator is to build new project/subprojects, but not
to diverge from LLVM itself (if it isn't clear in the current proposal we
should discuss it and clarify it, either way we end-up).

-- 
Mehdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200701/8423fde8/attachment.html>

On Wed, Jul 1, 2020 at 1:27 PM Mehdi AMINI via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>
> On Wed, Jul 1, 2020 at 10:20 AM Philip Reames via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>> On 6/30/20 2:07 PM, Chris Lattner via llvm-dev wrote:
>>
>>
>>
>> On Jun 30, 2020, at 2:02 PM, Duncan Exon Smith <dexonsmith at
apple.com>
>> wrote:
>>
>>
>>
>> On 2020-Jun-30, at 13:28, Chris Lattner via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> On Jun 29, 2020, at 10:15 PM, Chandler Carruth via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>
>> IMO, a pull request isn't as clear given that they haven't been
used for
>> contributions before. This is not a time to be innovative IMO. A branch
as
>> a staging location has been used many times over the history of the
project
>> though and seems nicely unambiguous in that regard.
>>
>>
>> I don’t have a opinion on this either way, but can git/GitHub maintain
>> forks within the same organization?  You could have llvm/llvm-project
and
>> llvm/llvm-project-apple-staging or something like that?
>>
>>
>> I don't think GitHub allows you fork your own repo so I think it
would be
>> disconnected from a GitHub point of view. That has a few downsides,
>> although I'm not sure how important they are.
>>
>> Regardless, if a separate repo is preferred, then a better name from
our
>> perspective would be "llvm-project-staging" (dropping the
"-apple" suffix).
>> We could push a "staging/apple" branch there.
>>
>>
>> Ok, I’m not very concerned either way, it was just a thought.  I’m very
>> happy to see this upstreaming work happen, thanks!
>>
>> -Chris
>>
>> I have a mild preference for the separate llvm-project-staging
approach,
>> but am not opposed to an in tree branch either.  The main argument I
see
>> for the separate repo is that the bar can be lower because the
consequences
>> for being "wrong" about the code being fully merged quickly
are lower.
>>
>> Or another thought, maybe we should even use the incubator flow here?
>> Nothing says an incubator has to be long lived.  If we spun up an
>> "incubator-staging-apple" repo, wouldn't that demonstrate
the same benefits?
>>
> I am not convinced the "incubator" proposal is suited for this
purpose as
> this is not about a new project but about a patch on top of LLVM itself. My
> understanding of the incubator is to build new project/subprojects, but not
> to diverge from LLVM itself (if it isn't clear in the current proposal
we
> should discuss it and clarify it, either way we end-up).
>
The "incubator" does not need to be (and shouldn't be) the only
way to
create a new repo in the LLVM org. If there is a pragmatic need for a new
utilitarian repo that fits outside of that process, then that seems like
something that the community can decide to do at any time without further
consequence.

>
> --
> Mehdi
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200701/7902b413/attachment.html>

On Wed, Jul 1, 2020 at 4:19 PM James Y Knight <jyknight at google.com>
wrote:
> The consequences of pushing a new branch name and then deleting it later
> should be low. And if we really wanted to exclude it from the default pull
> set, it can be placed in, "refs/staging/apple" instead of
> "refs/heads/staging/apple". Nothing outside refs/heads/ and
refs/tags/ gets
> pulled by default. This *should* be the obvious answer. However, we've
> had cases in the past that when people have accidentally created a branch
> or opened a pull request (which creates a new branch in
"refs/pull/NNN"),
> which triggered phabricator to start sending a bunch of effectively-spam
> emails. I believe that may still be a problem.
>
> Unless someone can confirm that this *won't* happen, it's probably
*not* a
> good idea to push 30,000 "new" old commits into any branch in the
repo.
>
Well, I can now confirm that this in fact does still happen. The creation
of https://github.com/llvm/llvm-project/pull/232 has triggered phabricator
to send out emails notifying me of my commits that were in there, as if
they were actually committed to a real branch. I assume the same
occurred for others. This is really unfortunate behavior, and I do hope we
can figure out how to configure phabricator to stop doing this.

But...now it's been done. So I guess that's that! The apple branch is in
github.com/llvm/llvm-project (in branch refs/pull/232/head; you can
checkout via `git fetch origin refs/pull/232/head && git checkout
FETCH_HEAD`).

I'd also concur with the other comments that it really feels quite silly
to
> do have to do anything technical here at all, versus posting an email to
> llvm-dev along the lines of "Apple is contributing the code added in
> github.com/apple/llvm-project, commit hash
> a1fde6dadf210c937c88509ab775610213e2cfc5, and all prior commits it depends
> on, under the Apache2 license with LLVM project exceptions, as listed in
>
https://github.com/apple/llvm-project/blob/a1fde6dadf210c937c88509ab775610213e2cfc5/LICENSE.txt".
> That seems like it'd be sufficient -- and even more explicit as a
statement
> of intent than creating a branch! (But, of course, IANAL).
>
> Or -- how about both sending such a message *and* creating a phab review
> for `git diff origin/master...swift/apple/master` (three dots diff gets the
> diff only from the last merge-point). Maybe that covers all the bases
> sufficiently?
>
> On Wed, Jul 1, 2020 at 1:19 PM Philip Reames via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>> On 6/30/20 2:07 PM, Chris Lattner via llvm-dev wrote:
>>
>>
>>
>> On Jun 30, 2020, at 2:02 PM, Duncan Exon Smith <dexonsmith at
apple.com>
>> wrote:
>>
>>
>>
>> On 2020-Jun-30, at 13:28, Chris Lattner via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> On Jun 29, 2020, at 10:15 PM, Chandler Carruth via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>
>> IMO, a pull request isn't as clear given that they haven't been
used for
>> contributions before. This is not a time to be innovative IMO. A branch
as
>> a staging location has been used many times over the history of the
project
>> though and seems nicely unambiguous in that regard.
>>
>>
>> I don’t have a opinion on this either way, but can git/GitHub maintain
>> forks within the same organization?  You could have llvm/llvm-project
and
>> llvm/llvm-project-apple-staging or something like that?
>>
>>
>> I don't think GitHub allows you fork your own repo so I think it
would be
>> disconnected from a GitHub point of view. That has a few downsides,
>> although I'm not sure how important they are.
>>
>> Regardless, if a separate repo is preferred, then a better name from
our
>> perspective would be "llvm-project-staging" (dropping the
"-apple" suffix).
>> We could push a "staging/apple" branch there.
>>
>>
>> Ok, I’m not very concerned either way, it was just a thought.  I’m very
>> happy to see this upstreaming work happen, thanks!
>>
>> -Chris
>>
>> I have a mild preference for the separate llvm-project-staging
approach,
>> but am not opposed to an in tree branch either.  The main argument I
see
>> for the separate repo is that the bar can be lower because the
consequences
>> for being "wrong" about the code being fully merged quickly
are lower.
>>
>> Or another thought, maybe we should even use the incubator flow here?
>> Nothing says an incubator has to be long lived.  If we spun up an
>> "incubator-staging-apple" repo, wouldn't that demonstrate
the same benefits?
>>
>> Philip
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200714/226d0f4c/attachment.html>