llvm dev - Oct 2019 - [cfe-dev] GitHub Migration Schedule and Plans

On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
> 
> 
> On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev <cfe-dev at
lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
> 
>     Hi,
> 
>     We're less than 2 weeks away from the developer meeting, so I
wanted to
>     give an update on the GitHub migration and what's (hopefully) going
to
>     happen during the developer meeting.
> 
>     Everyone who has added their information to the github-usernames.txt
>     file in SVN before today should have received an invite to become a
collaborator
>     on the llvm-project repository.  If you did not receive an invite and
think
>     you should have, please contact me off-list.  I will continue to
monitor the
>     file for new updates and periodically send out new batches of invites.
> 
>     There is still some ongoing work to get the buildbots ready and the
mailing lists
>     ready, but we are optimistic that the work will be done in time.
> 
>     The team at GitHub has finished implementing the "Require Linear
History"
>     branch protection that we requested.  The feature is in beta and
currently
>     enabled in the llvm-project repository.  This means that we will have
the
>     option to commit directly via git, in addition to using the git-llvm
script.
>     A patch that updates git-llvm to push to git instead of svn can be
found here:
>     https://reviews.llvm.org/D67772.  You should be able to test it out on
your
>     own fork of the llvm-project repository.
> 
>     The current plan is to begin the final migration steps on the evening
(PDT)
>     of October 21.  Here is what will happen:
> 
>     1. Make SVN read-only.
>     2. Turn-off the SVN->git update process.
>     3. Commit the new git-llvm script directly to github.
>     4. Grant all contributors write access to the repository.
> 
> 
> Is the repo configured to forbid contributors to create new branches?
I'm worried about the "jungle" it can become quickly if we leave
open the possibility to create branches "at will" in the repo, I
rather leave this to maintainers.
> 
I haven't been able to find a way to restrict branch creation for
committers,
I'm not sure if this is even possible.

We could try to enforce this rule in the git-llvm script, but this would
mean making use of the script mandatory, which was our original plan, but
that was based on the assumption that the "Require Linear History"
protection would not be ready in time.

Generally, would it be better if we kept use of the script mandatory so that 
we can handle this and other potential restrictions in the future?

- Tom
>  
> 
>     5. Email lists announcing that the migration is complete.
> 
>     Once the migration is complete, if you run into any issues, please file
>     a bug, and mark it as a blocker for the github metabug PR39393.
> 
>     If you have any questions or think I am missing something, please
>     let me know.
> 
> 
> This is fantastic! Thank you so much for doing this work Tom :)
> 
> -- 
> Mehdi
> 
>

On Thu, Oct 10, 2019 at 10:59 AM Tom Stellard <tstellar at redhat.com>
wrote:
> On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
> >
> >
> > On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev <
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
wrote:
> >
> >     Hi,
> >
> >     We're less than 2 weeks away from the developer meeting, so I
wanted
> to
> >     give an update on the GitHub migration and what's (hopefully)
going
> to
> >     happen during the developer meeting.
> >
> >     Everyone who has added their information to the
github-usernames.txt
> >     file in SVN before today should have received an invite to become
a
> collaborator
> >     on the llvm-project repository.  If you did not receive an invite
> and think
> >     you should have, please contact me off-list.  I will continue to
> monitor the
> >     file for new updates and periodically send out new batches of
> invites.
> >
> >     There is still some ongoing work to get the buildbots ready and
the
> mailing lists
> >     ready, but we are optimistic that the work will be done in time.
> >
> >     The team at GitHub has finished implementing the "Require
Linear
> History"
> >     branch protection that we requested.  The feature is in beta and
> currently
> >     enabled in the llvm-project repository.  This means that we will
> have the
> >     option to commit directly via git, in addition to using the
git-llvm
> script.
> >     A patch that updates git-llvm to push to git instead of svn can be
> found here:
> >     https://reviews.llvm.org/D67772.  You should be able to test it
out
> on your
> >     own fork of the llvm-project repository.
> >
> >     The current plan is to begin the final migration steps on the
> evening (PDT)
> >     of October 21.  Here is what will happen:
> >
> >     1. Make SVN read-only.
> >     2. Turn-off the SVN->git update process.
> >     3. Commit the new git-llvm script directly to github.
> >     4. Grant all contributors write access to the repository.
> >
> >
> > Is the repo configured to forbid contributors to create new branches?
> I'm worried about the "jungle" it can become quickly if we
leave open the
> possibility to create branches "at will" in the repo, I rather
leave this
> to maintainers.
> >
>
> I haven't been able to find a way to restrict branch creation for
> committers,
> I'm not sure if this is even possible.
>
>
I think you can just go to the branch settings, add a new branch protection
rule, match on everything but master, and check "Restrict who can push to
matching branches".


> We could try to enforce this rule in the git-llvm script, but this would
> mean making use of the script mandatory, which was our original plan, but
> that was based on the assumption that the "Require Linear
History"
> protection would not be ready in time.
>
> Generally, would it be better if we kept use of the script mandatory so
> that
> we can handle this and other potential restrictions in the future?
>
> - Tom
> >
> >
> >     5. Email lists announcing that the migration is complete.
> >
> >     Once the migration is complete, if you run into any issues, please
> file
> >     a bug, and mark it as a blocker for the github metabug PR39393.
> >
> >     If you have any questions or think I am missing something, please
> >     let me know.
> >
> >
> > This is fantastic! Thank you so much for doing this work Tom :)
> >
> > --
> > Mehdi
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191010/2bcc40dd/attachment.html>

On Thu, 10 Oct 2019, Tom Stellard via cfe-dev wrote:
> On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
>> 
>> 
>> On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev <cfe-dev at
lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
>>
>>     Hi,
>>
>>     We're less than 2 weeks away from the developer meeting, so I
wanted to
>>     give an update on the GitHub migration and what's (hopefully)
going to
>>     happen during the developer meeting.
>>
>>     Everyone who has added their information to the
github-usernames.txt
>>     file in SVN before today should have received an invite to become a
collaborator
>>     on the llvm-project repository.  If you did not receive an invite
and think
>>     you should have, please contact me off-list.  I will continue to
monitor the
>>     file for new updates and periodically send out new batches of
invites.
>>
>>     There is still some ongoing work to get the buildbots ready and the
mailing lists
>>     ready, but we are optimistic that the work will be done in time.
>>
>>     The team at GitHub has finished implementing the "Require
Linear History"
>>     branch protection that we requested.  The feature is in beta and
currently
>>     enabled in the llvm-project repository.  This means that we will
have the
>>     option to commit directly via git, in addition to using the
git-llvm script.
>>     A patch that updates git-llvm to push to git instead of svn can be
found here:
>>     https://reviews.llvm.org/D67772.  You should be able to test it out
on your
>>     own fork of the llvm-project repository.
>>
>>     The current plan is to begin the final migration steps on the
evening (PDT)
>>     of October 21.  Here is what will happen:
>>
>>     1. Make SVN read-only.
>>     2. Turn-off the SVN->git update process.
>>     3. Commit the new git-llvm script directly to github.
>>     4. Grant all contributors write access to the repository.
>> 
>> 
>> Is the repo configured to forbid contributors to create new branches?
I'm worried about the "jungle" it can become quickly if we leave
open the possibility to create branches "at will" in the repo, I
rather leave this to maintainers.
>> 
>
> I haven't been able to find a way to restrict branch creation for
committers,
> I'm not sure if this is even possible.
>
> We could try to enforce this rule in the git-llvm script, but this would
> mean making use of the script mandatory, which was our original plan, but
> that was based on the assumption that the "Require Linear
History"
> protection would not be ready in time.
>
> Generally, would it be better if we kept use of the script mandatory so
that
> we can handle this and other potential restrictions in the future?
Personally, I'd prefer to avoid requiring a custom tool for pushing 
commits, if possible.

As for creating undesired branches; contrary to accidentally pushing merge 
commits on the master branch, any accidentally created branch should be 
possible to remove without any permanent traces left behind (especially as 
it doesn't affect the master branch). So for that purpose alone, I'd
vote
for not requiring git-llvm for pushing.

Btw, out of curiousity, what mechanism does GitHub offer for verifying 
that the push actually comes from git-llvm? (I'm sorry if this part was 
discussed elsewhere, but I don't remember seeing it discussed lately.)

// Martin

On Thu, Oct 10, 2019 at 12:02 PM Martin Storsjö <martin at martin.st>
wrote:
> On Thu, 10 Oct 2019, Tom Stellard via cfe-dev wrote:
>
> > On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
> >>
> >>
> >> On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev <
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
wrote:
> >>
> >>     Hi,
> >>
> >>     We're less than 2 weeks away from the developer meeting,
so I
> wanted to
> >>     give an update on the GitHub migration and what's
(hopefully) going
> to
> >>     happen during the developer meeting.
> >>
> >>     Everyone who has added their information to the
github-usernames.txt
> >>     file in SVN before today should have received an invite to
become a
> collaborator
> >>     on the llvm-project repository.  If you did not receive an
invite
> and think
> >>     you should have, please contact me off-list.  I will continue
to
> monitor the
> >>     file for new updates and periodically send out new batches of
> invites.
> >>
> >>     There is still some ongoing work to get the buildbots ready
and the
> mailing lists
> >>     ready, but we are optimistic that the work will be done in
time.
> >>
> >>     The team at GitHub has finished implementing the "Require
Linear
> History"
> >>     branch protection that we requested.  The feature is in beta
and
> currently
> >>     enabled in the llvm-project repository.  This means that we
will
> have the
> >>     option to commit directly via git, in addition to using the
> git-llvm script.
> >>     A patch that updates git-llvm to push to git instead of svn
can be
> found here:
> >>     https://reviews.llvm.org/D67772.  You should be able to test
it
> out on your
> >>     own fork of the llvm-project repository.
> >>
> >>     The current plan is to begin the final migration steps on the
> evening (PDT)
> >>     of October 21.  Here is what will happen:
> >>
> >>     1. Make SVN read-only.
> >>     2. Turn-off the SVN->git update process.
> >>     3. Commit the new git-llvm script directly to github.
> >>     4. Grant all contributors write access to the repository.
> >>
> >>
> >> Is the repo configured to forbid contributors to create new
branches?
> I'm worried about the "jungle" it can become quickly if we
leave open the
> possibility to create branches "at will" in the repo, I rather
leave this
> to maintainers.
> >>
> >
> > I haven't been able to find a way to restrict branch creation for
> committers,
> > I'm not sure if this is even possible.
> >
> > We could try to enforce this rule in the git-llvm script, but this
would
> > mean making use of the script mandatory, which was our original plan,
but
> > that was based on the assumption that the "Require Linear
History"
> > protection would not be ready in time.
> >
> > Generally, would it be better if we kept use of the script mandatory
so
> that
> > we can handle this and other potential restrictions in the future?
>
> Personally, I'd prefer to avoid requiring a custom tool for pushing
> commits, if possible.
>
> As for creating undesired branches; contrary to accidentally pushing merge
> commits on the master branch, any accidentally created branch should be
> possible to remove without any permanent traces left behind (especially as
> it doesn't affect the master branch). So for that purpose alone,
I'd vote
> for not requiring git-llvm for pushing.
>
The original reason for the tool is about linear history (contrary to
branches, this can't be undone).

>
> Btw, out of curiousity, what mechanism does GitHub offer for verifying
> that the push actually comes from git-llvm? (I'm sorry if this part was
> discussed elsewhere, but I don't remember seeing it discussed lately.)

I mentioned it on LLVM-dev a couple of times a while back (here for
example: http://lists.llvm.org/pipermail/llvm-dev/2019-February/129896.html
 )

But apparently the concrete implementation was discussed on cfe-dev@ only,
so here are the details:
http://lists.llvm.org/pipermail/cfe-dev/2019-April/062063.html

-- 
Mehdi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20191010/c613cbdb/attachment.html>

On 10/10/2019 11:40 AM, Mehdi AMINI wrote:
> 
> 
> On Thu, Oct 10, 2019 at 10:59 AM Tom Stellard <tstellar at redhat.com
<mailto:tstellar at redhat.com>> wrote:
> 
>     On 10/09/2019 11:05 PM, Mehdi AMINI wrote:
>     >
>     >
>     > On Wed, Oct 9, 2019 at 10:16 PM Tom Stellard via cfe-dev
<cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
<mailto:cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>>> wrote:
>     >
>     >     Hi,
>     >
>     >     We're less than 2 weeks away from the developer meeting,
so I wanted to
>     >     give an update on the GitHub migration and what's
(hopefully) going to
>     >     happen during the developer meeting.
>     >
>     >     Everyone who has added their information to the
github-usernames.txt
>     >     file in SVN before today should have received an invite to
become a collaborator
>     >     on the llvm-project repository.  If you did not receive an
invite and think
>     >     you should have, please contact me off-list.  I will continue
to monitor the
>     >     file for new updates and periodically send out new batches of
invites.
>     >
>     >     There is still some ongoing work to get the buildbots ready
and the mailing lists
>     >     ready, but we are optimistic that the work will be done in
time.
>     >
>     >     The team at GitHub has finished implementing the "Require
Linear History"
>     >     branch protection that we requested.  The feature is in beta
and currently
>     >     enabled in the llvm-project repository.  This means that we
will have the
>     >     option to commit directly via git, in addition to using the
git-llvm script.
>     >     A patch that updates git-llvm to push to git instead of svn
can be found here:
>     >     https://reviews.llvm.org/D67772.  You should be able to test
it out on your
>     >     own fork of the llvm-project repository.
>     >
>     >     The current plan is to begin the final migration steps on the
evening (PDT)
>     >     of October 21.  Here is what will happen:
>     >
>     >     1. Make SVN read-only.
>     >     2. Turn-off the SVN->git update process.
>     >     3. Commit the new git-llvm script directly to github.
>     >     4. Grant all contributors write access to the repository.
>     >
>     >
>     > Is the repo configured to forbid contributors to create new
branches? I'm worried about the "jungle" it can become quickly if
we leave open the possibility to create branches "at will" in the
repo, I rather leave this to maintainers.
>     >
> 
>     I haven't been able to find a way to restrict branch creation for
committers,
>     I'm not sure if this is even possible.
> 
> 
> I think you can just go to the branch settings, add a new branch protection
rule, match on everything but master, and check "Restrict who can push to
matching branches".
> 
I tried this, and the branch protections only come into effect after a branch
has been creating, so this doesn't prevent new branches.  It's actually
worse
than doing nothing, because once the branch is created the branch protection
prevents you from deleting it.

-Tom
>  
> 
>     We could try to enforce this rule in the git-llvm script, but this
would
>     mean making use of the script mandatory, which was our original plan,
but
>     that was based on the assumption that the "Require Linear
History"
>     protection would not be ready in time.
> 
>     Generally, would it be better if we kept use of the script mandatory so
that
>     we can handle this and other potential restrictions in the future?
> 
>     - Tom
>     > 
>     >
>     >     5. Email lists announcing that the migration is complete.
>     >
>     >     Once the migration is complete, if you run into any issues,
please file
>     >     a bug, and mark it as a blocker for the github metabug
PR39393.
>     >
>     >     If you have any questions or think I am missing something,
please
>     >     let me know.
>     >
>     >
>     > This is fantastic! Thank you so much for doing this work Tom :)
>     >
>     > --
>     > Mehdi
>     >
>     > 
>