So, there's been a bit of a misunderstanding about the hooks that are supported in GitHub, and after talking to the GitHub staff, I'd like to clarify what they are and how we can use them. 1. Pre-commit hooks, avoiding forced pushes / re-order GitHub doesn't support server hooks due to security concerns. But there is an alternative: https://help.github.com/articles/about-required-status-checks/ I don't know how we'd check for non-ff-merges with this, and I'd appreciate if someone with better GitHub knowledge could chime in. But they *do* stop pushes from going in, which is what we want. Maybe we would need a web-service (see 2) to get this working. How does Swift solve this? Do we really need a linear history on the projects, or just on the umbrella project? 2. Post-commit umbrella updates We can use webhooks: https://developer.github.com/webhooks/ This would hit some webpage / buildbot and make them update the llvm-projects (with sub-modules) via git. We'd be required to maintain a piece of web service somewhere, but the maintenance of that web-service will be a lot less than the current SVN/Git servers. 3. Post-commit email, for review/history We can use email Webhooks: https://help.github.com/articles/managing-notifications-for-pushes-to-a-repository/ This would be enabled on all projects except the umbrella, so we can continue to make post-commit review. I believe 2 and 3 should be reasonably easy to set up, but I'm not sure about 1. It looks like it could work, but this is really a GitHub thing more than a Git thing. Any ideas? cheers, -renato
> On Jul 19, 2016, at 1:27 PM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > So, there's been a bit of a misunderstanding about the hooks that are > supported in GitHub, and after talking to the GitHub staff, I'd like > to clarify what they are and how we can use them. > > > 1. Pre-commit hooks, avoiding forced pushes / re-order > > GitHub doesn't support server hooks due to security concerns. > > But there is an alternative: > > https://help.github.com/articles/about-required-status-checks/Does this require submitting a PR for each commit?> I don't know how we'd check for non-ff-merges with this, and I'd > appreciate if someone with better GitHub knowledge could chime in. But > they *do* stop pushes from going in, which is what we want. Maybe we > would need a web-service (see 2) to get this working. > > How does Swift solve this?It doesn't, we use a pull request model. It's encouraged to run the CI on a PR before merging but not mandatory. Committing to the repo directly (without going through a PR) is possible.> Do we really need a linear history on the > projects, or just on the umbrella project?I, for one, am in favor of maintaining a linear history in all the sub-projects.> 2. Post-commit umbrella updates > > We can use webhooks: > > https://developer.github.com/webhooks/ > > This would hit some webpage / buildbot and make them update the > llvm-projects (with sub-modules) via git. > > We'd be required to maintain a piece of web service somewhere, but the > maintenance of that web-service will be a lot less than the current > SVN/Git servers.So the web-service will maintain a canonical set of repos with linear history? If so, + 1. best, vedant> 3. Post-commit email, for review/history > > We can use email Webhooks: > > https://help.github.com/articles/managing-notifications-for-pushes-to-a-repository/ > > This would be enabled on all projects except the umbrella, so we can > continue to make post-commit review. > > > I believe 2 and 3 should be reasonably easy to set up, but I'm not > sure about 1. It looks like it could work, but this is really a GitHub > thing more than a Git thing. > > Any ideas? > > cheers, > -renato > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> On Jul 19, 2016, at 1:27 PM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > So, there's been a bit of a misunderstanding about the hooks that are > supported in GitHub, and after talking to the GitHub staff, I'd like > to clarify what they are and how we can use them. > > > 1. Pre-commit hooks, avoiding forced pushes / re-order > > GitHub doesn't support server hooks due to security concerns. > > But there is an alternative: > > https://help.github.com/articles/about-required-status-checks/ > > I don't know how we'd check for non-ff-merges with this, and I'd > appreciate if someone with better GitHub knowledge could chime in. But > they *do* stop pushes from going in, which is what we want. Maybe we > would need a web-service (see 2) to get this working. > > How does Swift solve this? Do we really need a linear history on the > projects, or just on the umbrella project?Just to repeat what I said earlier I consider linear history an important feature! About the status checks: This sounds like we have to setup some buildbot/jenkins instance that reacts when pull requests are created with a simple job that checks linear history, commit date for us and then marks the commit as good to push as the last step (or pushes it automatically). (We could extend this later to more advanced pre-commit sanity checking like checking whether stuff compiles etc...) This will be work we have to do.> > > 2. Post-commit umbrella updates > > We can use webhooks: > > https://developer.github.com/webhooks/ > > This would hit some webpage / buildbot and make them update the > llvm-projects (with sub-modules) via git. > > We'd be required to maintain a piece of web service somewhere, but the > maintenance of that web-service will be a lot less than the current > SVN/Git servers. > > > 3. Post-commit email, for review/history > > We can use email Webhooks: > > https://help.github.com/articles/managing-notifications-for-pushes-to-a-repository/ > > This would be enabled on all projects except the umbrella, so we can > continue to make post-commit review. > > > I believe 2 and 3 should be reasonably easy to set up, but I'm not > sure about 1. It looks like it could work, but this is really a GitHub > thing more than a Git thing. > > Any ideas? > > cheers, > -renato > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
On 19 July 2016 at 13:44, Vedant Kumar via llvm-dev <llvm-dev at lists.llvm.org> wrote:>> https://help.github.com/articles/about-required-status-checks/ > > Does this require submitting a PR for each commit?It shouldn't do. It looks like the workflow would be: 1. Make your changes locally. 2. Push to your own fork (alternatively your own branch on github). 3. Run a sanity-check script that then POSTs to the status API on this branch to indicate success. 4. git push to master. I'm guessing a bit with step 2 -- I assume it has to allow statuses from other forks, otherwise there would be chaos on LLVM's main repository. What I'm still not entirely clear on is how the status is authenticated, it looks like it has to be an entirely trust-based system. It doesn't look like you can check "this status was set by a server at llvm.org".>From here and IRC, it seems like the current checks we might want are:1. Linear history. 2. Monotonic timestamps. 3. No timestamps from the future (checked against a reliable internet clock, not locally). Tim. P.S. Notes on using statuses for experiments: 1. Create OAuth token here https://github.com/settings/tokens (scope should be repo:/status). 2. Put simple status json into file: { "state": "success", "target_url": "https://example.com/build/status", "description": "The build succeeded!", "context": "sanity-check" } 3. Update a commit on GitHub. E.g. (last hash is a commit, substitute your username/OAuth token for authentication): curl -H "Content-Type: application/json" -u TNorthover:<OAuth SHA> --data @simple.txt https://api.github.com/repos/TNorthover/tmp/statuses/d0679d64a1f8b70bb33d27123ff4ed23dad18012 4. Now that at least one status exists, you can enable protected branch on your project. I have https://github.com/TNorthover/tmp for now (with master protected). I'd be happy to add anyone as a developer if they want to play too (could be good for testing the fork behaviour). 5. GitHub will deny a push to master unless it has a visible commit with the successful status.
On 19 July 2016 at 22:09, Matthias Braun <mbraun at apple.com> wrote:> About the status checks: This sounds like we have to setup some buildbot/jenkins instance that reacts when pull requests are created with a simple job > that checks linear history, commit date for us and then marks the commit as good to push as the last step (or pushes it automatically).IIUC, we don't need that. If the hook returns "success" (as it it builds, or it's linear), the push is allowed. If not, it's blocked.> (We could extend this later to more advanced pre-commit sanity checking like checking whether stuff compiles etc...)Yes, and I think a lot of people would want that, but we need to setup some infrastructure to do that (Jenkins, Buildbots, etc) and that's not trivial. We'll leave it for later. cheers, --renato
> On Jul 19, 2016, at 1:27 PM, Renato Golin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > So, there's been a bit of a misunderstanding about the hooks that are > supported in GitHub, and after talking to the GitHub staff, I'd like > to clarify what they are and how we can use them. > > > 1. Pre-commit hooks, avoiding forced pushes / re-order > > GitHub doesn't support server hooks due to security concerns.Well, maybe GitHub hosting is not the right choice for our requirement then.> > But there is an alternative: > > https://help.github.com/articles/about-required-status-checks/ > > I don't know how we'd check for non-ff-merges with this, and I'd > appreciate if someone with better GitHub knowledge could chime in. But > they *do* stop pushes from going in, which is what we want. Maybe we > would need a web-service (see 2) to get this working. > > How does Swift solve this? Do we really need a linear history on the > projects, or just on the umbrella project? > > > 2. Post-commit umbrella updates > > We can use webhooks: > > https://developer.github.com/webhooks/ > > This would hit some webpage / buildbot and make them update the > llvm-projects (with sub-modules) via git. > > We'd be required to maintain a piece of web service somewhere, but the > maintenance of that web-service will be a lot less than the current > SVN/Git servers.Claiming that it "will be *a lot* less” burden that now is easy, but I don’t see any obvious fact to back this up. What is the current maintenance requirement of SVN/Git? Can someone who knows provides some fact? CC Anton just in case. — Mehdi> > > 3. Post-commit email, for review/history > > We can use email Webhooks: > > https://help.github.com/articles/managing-notifications-for-pushes-to-a-repository/ > > This would be enabled on all projects except the umbrella, so we can > continue to make post-commit review. > > > I believe 2 and 3 should be reasonably easy to set up, but I'm not > sure about 1. It looks like it could work, but this is really a GitHub > thing more than a Git thing. > > Any ideas? > > cheers, > -renato > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
On 19 July 2016 at 22:35, Mehdi Amini <mehdi.amini at apple.com> wrote:> Claiming that it "will be *a lot* less” burden that now is easy, but I don’t see any obvious fact to back this up. > What is the current maintenance requirement of SVN/Git? Can someone who knows provides some fact?I'll let Anton tell his side, and Tanya talk about the real costs, but here are some facts I know: Our ARM/AArch64 buildbots fail around 2~3 times a month with SVN errors. Sometimes it's only the fast ones, sometimes all of them (depends on how long it takes to fix). Sometimes the fix is just "wait", sometimes Anton has to actively fix it. (he also has to work, sleep, eat, etc). In the past, we were hit by web spiders that ignored completely the robots.txt file. Anton has made that better, but it can escalate if the spider realise we blocked them. There are ways to work around, but not without accidentally blocking innocent people (mostly in China). The cost of the AWS servers is ~$5k / year. It's not *only* for SVN, but also for web servers and hosting packages. Recently we turned off the deb hosting because of budget (our server and bandwidth couldn't cope with it). So, while $5k/year might not look like much, it's enough to pay a lot of students to go to the LLVM events, that couldn't otherwise go. It's also nowhere near what we would like if we were to host a robust repository with the features that GitHub can provide. Mainly bandwidth, storage, stability and support. Given the AWS costs that I've seen at Linaro, we'd have to *at least* double that money to host a dedicated machine with enough bandwidth to have repositories, binaries, videos etc. not counting paying someone to actively maintain it, if we want to compare one to one with what GitHub provides for free. I will make no attempt at estimating Anton's time, or Tanya's or anyone else's, but I believe they (and their companies/universities) would very much rather they work on actual compiler stuff. I'm sure that, if we join the human cost, it'll far outweigh the infrastructure costs, even if we double/triple our current spending. On the other hand, as Tim has shown, a web-service with a JSON file will be running some web server which is light and cheaper than a normal web-page to deliver (less content, less bandwidth, less storage, less I/O), and could serve hundreds, if not thousands of queries per second with a small AWS image. The web-hooks would be setup once and hosted by GitHub, so zero additional work from our side, as well as all the forking, branching, merging, SVN interface (which we can't easily get if we move to local Git). The level of failure in the web-services will be lower (lower load, less probability of barfing) and even if it does, it will only affect the services that use it (buildbots, LNT, bisect), not any other developer. Moreover, our side of the web-service can fail catastrophically and need a wipe and restart, and *none* of our commit history would be affected. On the other hand, if the SVN fails catastrophically today, I don't know if we have a good backup policy that will mean commits could be lost. GitHub may not provide guarantees, but they do have proper backup policies. All in all, may not look much, but running a decent and stable web service with so much at stake is *not* a simple task, and we shouldn't take it for granted. cheers, --renato
I've not read all of the github threads, so sorry if this has been brought up, but... On Tue, Jul 19, 2016 at 1:27 PM Renato Golin via llvm-dev < llvm-dev at lists.llvm.org> wrote:> 1. Pre-commit hooks, avoiding forced pushes / re-order > > GitHub doesn't support server hooks due to security concerns. >GitHub does support protected branches which prevent forced pushes. I've even played with them in the llvm GitHub project and they work as expected. It should give the exact workflow that I think LLVM devs are used to with Subversion of post-commit review. -Chandler -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160719/1d7ad314/attachment.html>
That is what I'm proposing, and Tim is helping us test. We should reach a solution quickly, and once we do, I'll update the document. Feel free to try his repo, I'll only try tomorrow. If you guys come up with a clear flow before that, let me know. Cheers, Renato On 20 Jul 2016 12:36 a.m., "Chandler Carruth" <chandlerc at google.com> wrote:> I've not read all of the github threads, so sorry if this has been brought > up, but... > > On Tue, Jul 19, 2016 at 1:27 PM Renato Golin via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> 1. Pre-commit hooks, avoiding forced pushes / re-order >> >> GitHub doesn't support server hooks due to security concerns. >> > > GitHub does support protected branches which prevent forced pushes. I've > even played with them in the llvm GitHub project and they work as expected. > It should give the exact workflow that I think LLVM devs are used to with > Subversion of post-commit review. > > -Chandler >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160720/a2103844/attachment.html>
Not to derail a large portion of this thread by just pointing out how easy what you want to do is, but ... On Tue, Jul 19, 2016 at 1:27 PM, Renato Golin via llvm-dev < llvm-dev at lists.llvm.org> wrote:> So, there's been a bit of a misunderstanding about the hooks that are > supported in GitHub, and after talking to the GitHub staff, I'd like > to clarify what they are and how we can use them. > > > 1. Pre-commit hooks, avoiding forced pushes / re-order > > GitHub doesn't support server hooks due to security concerns. > > But there is an alternative: > > https://help.github.com/articles/about-required-status-checks/ > > I don't know how we'd check for non-ff-merges with this, and I'd > appreciate if someone with better GitHub knowledge could chime in. But > they *do* stop pushes from going in, which is what we want. Maybe we > would need a web-service (see 2) to get this working. >They do stop pushes, because you can require status checks. You don't need a web service. Anyone with push access can create a status, or you can also grant oauth scope of repo:status to just grant status creation access without push access to code itself. All you have to do to create a status is post to a specific URL (and include your access tokens, blah blah blah). You can use the context field to say what service created the status, etc. There are libraries for all of it. For example, Google's CLA bot uses https://github.com/google/go-github/tree/master/github to do it's work. There are libraries for every language you can think of, AFAIK ( https://developer.github.com/libraries/) In any case, all you would have to is post a status of state failed to the ref for the pull request, and it will reflect in the pull request. In this case, what that means is "have thing" (cron job, whatever), post state pending on every new request that comes in. As it figures out whether they are linear history or not, post state success/failed. You can then mark whatever status checks you want as required for a given branch (it'll give you a dropdown of status types it finds in the past week). it will not allow push or merge until the required statuses are green. The interface also allows you to check "Require branches to be up-to-date before merging". --Dan -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160719/70388184/attachment.html>
Also note that github already blocks non-ff pushes unless you force (and you can't force a protected branch without being an admin, and you can even disable admins force-pushing), so ... On Tue, Jul 19, 2016 at 8:20 PM, Daniel Berlin <dberlin at dberlin.org> wrote:> Not to derail a large portion of this thread by just pointing out how easy > what you want to do is, but ... > > On Tue, Jul 19, 2016 at 1:27 PM, Renato Golin via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> So, there's been a bit of a misunderstanding about the hooks that are >> supported in GitHub, and after talking to the GitHub staff, I'd like >> to clarify what they are and how we can use them. >> >> >> 1. Pre-commit hooks, avoiding forced pushes / re-order >> >> GitHub doesn't support server hooks due to security concerns. >> >> But there is an alternative: >> >> https://help.github.com/articles/about-required-status-checks/ >> >> I don't know how we'd check for non-ff-merges with this, and I'd >> appreciate if someone with better GitHub knowledge could chime in. But >> they *do* stop pushes from going in, which is what we want. Maybe we >> would need a web-service (see 2) to get this working. >> > > They do stop pushes, because you can require status checks. > > You don't need a web service. Anyone with push access can create a status, > or you can also > grant oauth scope of repo:status to just grant status creation access > without push access to code itself. > > All you have to do to create a status is post to a specific URL (and > include your access tokens, blah blah blah). You can use the context field > to say what service created the status, etc. There are libraries for all > of it. > For example, Google's CLA bot uses > https://github.com/google/go-github/tree/master/github to do it's work. > There are libraries for every language you can think of, AFAIK ( > https://developer.github.com/libraries/) > > In any case, all you would have to is post a status of state failed to the > ref for the pull request, and it will reflect in the pull request. > > In this case, what that means is "have thing" (cron job, whatever), post > state pending on every new request that comes in. > As it figures out whether they are linear history or not, post state > success/failed. > > You can then mark whatever status checks you want as required for a given > branch (it'll give you a dropdown of status types it finds in the past > week). > > it will not allow push or merge until the required statuses are green. > > > The interface also allows you to check "Require branches to be up-to-date > before merging". > > --Dan > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160719/8cb738a0/attachment.html>