llvm dev - Oct 2019 - [cfe-dev] GitHub Migration Schedule and Plans

On 10/10/2019 11:40 AM, Mehdi AMINI wrote:
> 
> 
> On Thu, Oct 10, 2019 at 10:59 AM Tom Stellard <tstellar at redhat.com
<mailto:tstellar at redhat.com>> wrote:
> 
>     On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
>     >
>     >
>     > On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev
<cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
<mailto:cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>>> wrote:
>     >
>     >     Hi,
>     >
>     >     We're less than 2 weeks away from the developer meeting,
so I wanted to
>     >     give an update on the GitHub migration and what's
(hopefully) going to
>     >     happen during the developer meeting.
>     >
>     >     Everyone who has added their information to the
github-usernames.txt
>     >     file in SVN before today should have received an invite to
become a collaborator
>     >     on the llvm-project repository.  If you did not receive an
invite and think
>     >     you should have, please contact me off-list.  I will continue
to monitor the
>     >     file for new updates and periodically send out new batches of
invites.
>     >
>     >     There is still some ongoing work to get the buildbots ready
and the mailing lists
>     >     ready, but we are optimistic that the work will be done in
time.
>     >
>     >     The team at GitHub has finished implementing the "Require
Linear History"
>     >     branch protection that we requested.  The feature is in beta
and currently
>     >     enabled in the llvm-project repository.  This means that we
will have the
>     >     option to commit directly via git, in addition to using the
git-llvm script.
>     >     A patch that updates git-llvm to push to git instead of svn
can be found here:
>     >     https://reviews.llvm.org/D67772.  You should be able to test
it out on your
>     >     own fork of the llvm-project repository.
>     >
>     >     The current plan is to begin the final migration steps on the
evening (PDT)
>     >     of October 21.  Here is what will happen:
>     >
>     >     1. Make SVN read-only.
>     >     2. Turn-off the SVN->git update process.
>     >     3. Commit the new git-llvm script directly to github.
>     >     4. Grant all contributors write access to the repository.
>     >
>     >
>     > Is the repo configured to forbid contributors to create new
branches? I'm worried about the "jungle" it can become quickly if
we leave open the possibility to create branches "at will" in the
repo, I rather leave this to maintainers.
>     >
> 
>     I haven't been able to find a way to restrict branch creation for
committers,
>     I'm not sure if this is even possible.
> 
> 
> I think you can just go to the branch settings, add a new branch protection
rule, match on everything but master, and check "Restrict who can push to
matching branches".
> 
I tried this, and the branch protections only come into effect after a branch
has been creating, so this doesn't prevent new branches.  It's actually
worse
than doing nothing, because once the branch is created the branch protection
prevents you from deleting it.

-Tom
>  
> 
>     We could try to enforce this rule in the git-llvm script, but this
would
>     mean making use of the script mandatory, which was our original plan,
but
>     that was based on the assumption that the "Require Linear
History"
>     protection would not be ready in time.
> 
>     Generally, would it be better if we kept use of the script mandatory so
that
>     we can handle this and other potential restrictions in the future?
> 
>     - Tom
>     > 
>     >
>     >     5. Email lists announcing that the migration is complete.
>     >
>     >     Once the migration is complete, if you run into any issues,
please file
>     >     a bug, and mark it as a blocker for the github metabug
PR39393.
>     >
>     >     If you have any questions or think I am missing something,
please
>     >     let me know.
>     >
>     >
>     > This is fantastic! Thank you so much for doing this work Tom :)
>     >
>     > --
>     > Mehdi
>     >
>     > 
>

On Thu, Oct 10, 2019 at 12:29 PM Tom Stellard via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 10/10/2019 11:40 AM, Mehdi AMINI wrote:
> >
> >
> > On Thu, Oct 10, 2019 at 10:59 AM Tom Stellard <tstellar at
redhat.com
> <mailto:tstellar at redhat.com>> wrote:
> >
> >     On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
> >     >
> >     >
> >     > On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev <
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
<mailto:
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>>
wrote:
> >     >
> >     >     Hi,
> >     >
> >     >     We're less than 2 weeks away from the developer
meeting, so I
> wanted to
> >     >     give an update on the GitHub migration and what's
(hopefully)
> going to
> >     >     happen during the developer meeting.
> >     >
> >     >     Everyone who has added their information to the
> github-usernames.txt
> >     >     file in SVN before today should have received an invite
to
> become a collaborator
> >     >     on the llvm-project repository.  If you did not receive
an
> invite and think
> >     >     you should have, please contact me off-list.  I will
continue
> to monitor the
> >     >     file for new updates and periodically send out new
batches of
> invites.
> >     >
> >     >     There is still some ongoing work to get the buildbots
ready
> and the mailing lists
> >     >     ready, but we are optimistic that the work will be done
in
> time.
> >     >
> >     >     The team at GitHub has finished implementing the
"Require
> Linear History"
> >     >     branch protection that we requested.  The feature is in
beta
> and currently
> >     >     enabled in the llvm-project repository.  This means that
we
> will have the
> >     >     option to commit directly via git, in addition to using
the
> git-llvm script.
> >     >     A patch that updates git-llvm to push to git instead of
svn
> can be found here:
> >     >     https://reviews.llvm.org/D67772.  You should be able to
test
> it out on your
> >     >     own fork of the llvm-project repository.
> >     >
> >     >     The current plan is to begin the final migration steps on
the
> evening (PDT)
> >     >     of October 21.  Here is what will happen:
> >     >
> >     >     1. Make SVN read-only.
> >     >     2. Turn-off the SVN->git update process.
> >     >     3. Commit the new git-llvm script directly to github.
> >     >     4. Grant all contributors write access to the repository.
> >     >
> >     >
> >     > Is the repo configured to forbid contributors to create new
> branches? I'm worried about the "jungle" it can become
quickly if we leave
> open the possibility to create branches "at will" in the repo, I
rather
> leave this to maintainers.
> >     >
> >
> >     I haven't been able to find a way to restrict branch creation
for
> committers,
> >     I'm not sure if this is even possible.
> >
> >
> > I think you can just go to the branch settings, add a new branch
> protection rule, match on everything but master, and check "Restrict
who
> can push to matching branches".
> >
>
> I tried this, and the branch protections only come into effect after a
> branch
> has been creating, so this doesn't prevent new branches.  It's
actually
> worse
> than doing nothing, because once the branch is created the branch
> protection
> prevents you from deleting it.
>
> -Tom
>
FWIW, we're interested in periodically (weekly) tagging well-tested/stable
revisions, but via a branch instead of just a tag so we can include which
cherrypicks (e.g. reverts or fixes) are needed. We do this with the current
svn repo so we'd just be porting existing functionality to github.

(Also, I'm not sure this announcement thread with all the *-dev lists is
the best place to discuss branching policy, but I wanted to get this bit in
since y'all brought it up :) )

>
> >
> >
> >     We could try to enforce this rule in the git-llvm script, but this
> would
> >     mean making use of the script mandatory, which was our original
> plan, but
> >     that was based on the assumption that the "Require Linear
History"
> >     protection would not be ready in time.
> >
> >     Generally, would it be better if we kept use of the script
mandatory
> so that
> >     we can handle this and other potential restrictions in the future?
> >
> >     - Tom
> >     >
> >     >
> >     >     5. Email lists announcing that the migration is complete.
> >     >
> >     >     Once the migration is complete, if you run into any
issues,
> please file
> >     >     a bug, and mark it as a blocker for the github metabug
PR39393.
> >     >
> >     >     If you have any questions or think I am missing
something,
> please
> >     >     let me know.
> >     >
> >     >
> >     > This is fantastic! Thank you so much for doing this work Tom
:)
> >     >
> >     > --
> >     > Mehdi
> >     >
> >     >
> >
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191010/e767fda3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4849 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191010/e767fda3/attachment.bin>

On Thu, Oct 10, 2019 at 1:14 PM Jordan Rupprecht <rupprecht at google.com>
wrote:
>
>
> On Thu, Oct 10, 2019 at 12:29 PM Tom Stellard via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On 10/10/2019 11:40 AM, Mehdi AMINI wrote:
>> >
>> >
>> > On Thu, Oct 10, 2019 at 10:59 AM Tom Stellard <tstellar at
redhat.com
>> <mailto:tstellar at redhat.com>> wrote:
>> >
>> >     On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
>> >     >
>> >     >
>> >     > On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev
<
>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
<mailto:
>> cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>>> wrote:
>> >     >
>> >     >     Hi,
>> >     >
>> >     >     We're less than 2 weeks away from the developer
meeting, so I
>> wanted to
>> >     >     give an update on the GitHub migration and what's
(hopefully)
>> going to
>> >     >     happen during the developer meeting.
>> >     >
>> >     >     Everyone who has added their information to the
>> github-usernames.txt
>> >     >     file in SVN before today should have received an
invite to
>> become a collaborator
>> >     >     on the llvm-project repository.  If you did not
receive an
>> invite and think
>> >     >     you should have, please contact me off-list.  I will
continue
>> to monitor the
>> >     >     file for new updates and periodically send out new
batches of
>> invites.
>> >     >
>> >     >     There is still some ongoing work to get the buildbots
ready
>> and the mailing lists
>> >     >     ready, but we are optimistic that the work will be
done in
>> time.
>> >     >
>> >     >     The team at GitHub has finished implementing the
"Require
>> Linear History"
>> >     >     branch protection that we requested.  The feature is
in beta
>> and currently
>> >     >     enabled in the llvm-project repository.  This means
that we
>> will have the
>> >     >     option to commit directly via git, in addition to
using the
>> git-llvm script.
>> >     >     A patch that updates git-llvm to push to git instead
of svn
>> can be found here:
>> >     >     https://reviews.llvm.org/D67772.  You should be able
to test
>> it out on your
>> >     >     own fork of the llvm-project repository.
>> >     >
>> >     >     The current plan is to begin the final migration
steps on the
>> evening (PDT)
>> >     >     of October 21.  Here is what will happen:
>> >     >
>> >     >     1. Make SVN read-only.
>> >     >     2. Turn-off the SVN->git update process.
>> >     >     3. Commit the new git-llvm script directly to github.
>> >     >     4. Grant all contributors write access to the
repository.
>> >     >
>> >     >
>> >     > Is the repo configured to forbid contributors to create
new
>> branches? I'm worried about the "jungle" it can become
quickly if we leave
>> open the possibility to create branches "at will" in the
repo, I rather
>> leave this to maintainers.
>> >     >
>> >
>> >     I haven't been able to find a way to restrict branch
creation for
>> committers,
>> >     I'm not sure if this is even possible.
>> >
>> >
>> > I think you can just go to the branch settings, add a new branch
>> protection rule, match on everything but master, and check
"Restrict who
>> can push to matching branches".
>> >
>>
>> I tried this, and the branch protections only come into effect after a
>> branch
>> has been creating, so this doesn't prevent new branches.  It's
actually
>> worse
>> than doing nothing, because once the branch is created the branch
>> protection
>> prevents you from deleting it.
>>
>
Ah, interesting :)
Since for now `git-llvm` is necessary to push to the repo, I guess creating
a branch won't even really be possible? So when adding the support for
branch creation to `git-llvm` we could print a disclaimer about it or
something?

>
>> -Tom
>>
>
> FWIW, we're interested in periodically (weekly) tagging
well-tested/stable
> revisions, but via a branch instead of just a tag so we can include which
> cherrypicks (e.g. reverts or fixes) are needed. We do this with the current
> svn repo so we'd just be porting existing functionality to github.
>
I would likely include this in the general category of "project
maintainer"
created branch?
The main difference with SVN, is that on GitHub a fork is "cheap" and
everyone can have their own. Also compared to SVN you don't really know
ahead of time when you type `git clone` or `git pull` that you may pull a
lot of noise.
I'm not against branches for the sake of it, I just have some bad memories
from a repo where every developer was pushing their work in progress
branches all the time and they never got deleted: to the point where some
git CLI auto-completion was unusable (there were >10k branches and probably
as many tags).

>
> (Also, I'm not sure this announcement thread with all the *-dev lists
is
> the best place to discuss branching policy, but I wanted to get this bit in
> since y'all brought it up :) )
>
(we discussed branches in past GitHub migration thread, we just never had a
dedicated email thread to user-branches, as it didn't seem like there was a
strong need for it)

>
>
>>
>> >
>> >
>> >     We could try to enforce this rule in the git-llvm script, but
this
>> would
>> >     mean making use of the script mandatory, which was our
original
>> plan, but
>> >     that was based on the assumption that the "Require Linear
History"
>> >     protection would not be ready in time.
>> >
>> >     Generally, would it be better if we kept use of the script
>> mandatory so that
>> >     we can handle this and other potential restrictions in the
future?
>> >
>> >     - Tom
>> >     >
>> >     >
>> >     >     5. Email lists announcing that the migration is
complete.
>> >     >
>> >     >     Once the migration is complete, if you run into any
issues,
>> please file
>> >     >     a bug, and mark it as a blocker for the github
metabug
>> PR39393.
>> >     >
>> >     >     If you have any questions or think I am missing
something,
>> please
>> >     >     let me know.
>> >     >
>> >     >
>> >     > This is fantastic! Thank you so much for doing this work
Tom :)
>> >     >
>> >     > --
>> >     > Mehdi
>> >     >
>> >     >
>> >
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191010/0f175932/attachment.html>

On Thu, Oct 10, 2019 at 4:15 PM Jordan Rupprecht via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>
> On Thu, Oct 10, 2019 at 12:29 PM Tom Stellard via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On 10/10/2019 11:40 AM, Mehdi AMINI wrote:
>> >
>> >
>> > On Thu, Oct 10, 2019 at 10:59 AM Tom Stellard <tstellar at
redhat.com
>> <mailto:tstellar at redhat.com>> wrote:
>> >
>> >     On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
>> >     >
>> >     >
>> >     > On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev
<
>> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
<mailto:
>> cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>>> wrote:
>> >     >
>> >     >     Hi,
>> >     >
>> >     >     We're less than 2 weeks away from the developer
meeting, so I
>> wanted to
>> >     >     give an update on the GitHub migration and what's
(hopefully)
>> going to
>> >     >     happen during the developer meeting.
>> >     >
>> >     >     Everyone who has added their information to the
>> github-usernames.txt
>> >     >     file in SVN before today should have received an
invite to
>> become a collaborator
>> >     >     on the llvm-project repository.  If you did not
receive an
>> invite and think
>> >     >     you should have, please contact me off-list.  I will
continue
>> to monitor the
>> >     >     file for new updates and periodically send out new
batches of
>> invites.
>> >     >
>> >     >     There is still some ongoing work to get the buildbots
ready
>> and the mailing lists
>> >     >     ready, but we are optimistic that the work will be
done in
>> time.
>> >     >
>> >     >     The team at GitHub has finished implementing the
"Require
>> Linear History"
>> >     >     branch protection that we requested.  The feature is
in beta
>> and currently
>> >     >     enabled in the llvm-project repository.  This means
that we
>> will have the
>> >     >     option to commit directly via git, in addition to
using the
>> git-llvm script.
>> >     >     A patch that updates git-llvm to push to git instead
of svn
>> can be found here:
>> >     >     https://reviews.llvm.org/D67772.  You should be able
to test
>> it out on your
>> >     >     own fork of the llvm-project repository.
>> >     >
>> >     >     The current plan is to begin the final migration
steps on the
>> evening (PDT)
>> >     >     of October 21.  Here is what will happen:
>> >     >
>> >     >     1. Make SVN read-only.
>> >     >     2. Turn-off the SVN->git update process.
>> >     >     3. Commit the new git-llvm script directly to github.
>> >     >     4. Grant all contributors write access to the
repository.
>> >     >
>> >     >
>> >     > Is the repo configured to forbid contributors to create
new
>> branches? I'm worried about the "jungle" it can become
quickly if we leave
>> open the possibility to create branches "at will" in the
repo, I rather
>> leave this to maintainers.
>> >     >
>> >
>> >     I haven't been able to find a way to restrict branch
creation for
>> committers,
>> >     I'm not sure if this is even possible.
>> >
>> >
>> > I think you can just go to the branch settings, add a new branch
>> protection rule, match on everything but master, and check
"Restrict who
>> can push to matching branches".
>> >
>>
>> I tried this, and the branch protections only come into effect after a
>> branch
>> has been creating, so this doesn't prevent new branches.  It's
actually
>> worse
>> than doing nothing, because once the branch is created the branch
>> protection
>> prevents you from deleting it.
>>
>> -Tom
>>
>
> FWIW, we're interested in periodically (weekly) tagging
well-tested/stable
> revisions, but via a branch instead of just a tag so we can include which
> cherrypicks (e.g. reverts or fixes) are needed. We do this with the current
> svn repo so we'd just be porting existing functionality to github.
>
> (Also, I'm not sure this announcement thread with all the *-dev lists
is
> the best place to discuss branching policy, but I wanted to get this bit in
> since y'all brought it up :) )
>
IMO -- we should generally discourage random developer branches, and
instead encourage people to fork the repo and do their work there. But I'd
concur that it's not so big a concern as to require it to be prohibited via
tooling, since they're easily removable. I can imagine some cases where a
temporary dev branch in the main repo might be appropriate to use, and it'd
be nice not to prohibit such a valid use.

With regards to these sorts of "vendor release branches", though --
I'm
strongly against putting those in the main repo. That usage would certainly
be best handled with a separate Google "fork" of the llvm-project
repository -- and similarly for anyone else who wants to do it. Putting
such branches into the main llvm repo means that everyone pulling it pulls
down all the google release branches by default, which is really
unnecessary clutter (check out
https://github.com/llvm/llvm-project-legacy-branches if you'd like to see
what such clutter starts to look like...note that github doesn't even
display all of them in its branch/tag selector popup because there's too
many.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191015/2388bed2/attachment.html>