Perhaps it helps to know that I have access to the machines and have helped debug many of the current problems. I'm not speaking from the outside, guessing how hard things are. I also think you are assuming a lot about where services can be hosted and at which cost (labour, not hardware). So, unless you are volunteering to take care of the whole infrastructure, I suggest taking the opinion of people that are working with it a but more seriously. However, I'd still trust github over anyone of us any day, to host our repositories. Any one of us could leave the community at any time, but github, as a company, with many employees and a successful business, will probably outlast any if us in our current employments.>From that point of view, the foundation would be better betting their moneyon a stable product than on individual volunteers. While it's cool that some people volunteer, you can't base such a critical system on it. Cheers, Renato On 19 Jul 2016 11:47 p.m., "Mehdi Amini" <mehdi.amini at apple.com> wrote:> > > On Jul 19, 2016, at 3:32 PM, Renato Golin <renato.golin at linaro.org> > wrote: > > > > On 19 July 2016 at 23:16, Mehdi Amini <mehdi.amini at apple.com> wrote: > >>> In the past, we were hit by web spiders that ignored completely the > >>> robots.txt file. Anton has made that better, but it can escalate if > >>> the spider realise we blocked them. There are ways to work around, but > >>> not without accidentally blocking innocent people (mostly in China). > >> > >> That’s not relevant: this is about the WWW server, it does not have to > be related to the hosting the repos. > > > > No, this is about hosting the SVN server. The SVN view was disabled > > for months this year before we could really see what was going on. > > I don’t believe the online SVN viewer has to be on the server that hosts > the repo that everyone access: the WWW server could mirror the SVN to > provide local access to the viewer if needed (hence why I view this as > unrelated to hosting source code). > > >> Moving the SVN repo does not solve hosting videos, Debian packages, etc. > >> I suspect most of the bandwidth does not come from `svn up` or `git > pull`. > > > > They share the same bandwidth, and sometimes the same server. It is > relevant. > > Well, “they share the same bandwidth” is exactly what I mean by > “conflating the issues”. > They don’t *have to* share the same bandwidth. Hosting repos could be > setup totally separated from hosting WWW. > You need to account things properly. > > > One thing making SVN slow was the amount of Debian packages being > > downloaded form the same place. > > > > > >> Like… proper hooks? > > > > If we can work around it, and it seems we can, this is not such a big > issue. > > > > > >> You’re again conflating svn/git and hosting “binaries and videos”. I > don’t think we ever planned to host these on github? > > > > No, but they all share bandwidth. We moved videos to Youtube to > > offload the bandwidth, and moving the code to GitHub shares the same > > mindset. > > It shares the same mindset *only* if the code itself is a significant > bandwidth consumer, otherwise no it does not make sense. > > > > > > >> Possibly, I don’t know, but that’s exactly why I asked for first hand > data on the subject (i.e. Anton and/or Tanya) about hosting the git/SVN > repos themselves, instead of hand-wavy “I believe” discussions. > > > > Bear in mind that I gave you facts (bandwidth problems, turned off SVN > > services, constant breakdowns, expertise in handling traffic, backup > > solutions). > > And I consider many of the “facts” you gave to conflate other element than > hosting the repository *alone*, which makes it hard to me to see them as > relevant as-is. > > > > > I also made you aware that the human cost is not *just* Tanya and > > Anton, but also me and everyone else that maintains buildbots, > > external mirrors, etc. and it *is* larger than the hardware costs. You > > just don't see it because we're all volunteers. > > > > Branding them as "hand-wavy I believe" is *not* appropriate. > > I apologize if I hurt your feeling, but the reality is that I feel you’re > conflating multiple things together that are not directly related to > “moving the repository only”, and that does not help to be convincing. My > use of “hand-wavy”, if that’s what bothered you, means really that (I’m not > attaching any other value judgement to this expression as a non-native > speaker, maybe it is not the right choice of word from a non-native > speaker). > > — > Mehdi-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160720/55c1b006/attachment-0001.html>
> On Jul 19, 2016, at 4:00 PM, Renato Golin <renato.golin at linaro.org> wrote: > > Perhaps it helps to know that I have access to the machines and have helped debug many of the current problems. I'm not speaking from the outside, guessing how hard things are. > > I also think you are assuming a lot about where services can be hosted and at which cost (labour, not hardware). >It seems you’re assuming as much about me as you think I’m assuming about this whole thing...> So, unless you are volunteering to take care of the whole infrastructure, I suggest taking the opinion of people that are working with it a but more seriously. >Sorry, I appreciate your opinion, but it is still not “facts” or “data”. When you write “it will save *a lot* of maintenance”, I raise my eyebrows. I’ll take what you write seriously when it’ll be be phrased less qualitatively and more quantitatively, i.e. for instance I would ask less if you wrote “currently volunteers (Anton, Tanya, whoever) spends on average 5h a week to fix problem purely related to the *repositories* (not the video, not the Debian packages, etc.) and moving to an external host would save totally these 5h/week”. — Mehdi> However, I'd still trust github over anyone of us any day, to host our repositories. Any one of us could leave the community at any time, but github, as a company, with many employees and a successful business, will probably outlast any if us in our current employments. > > From that point of view, the foundation would be better betting their money on a stable product than on individual volunteers. While it's cool that some people volunteer, you can't base such a critical system on it. > > Cheers, > Renato > > > On 19 Jul 2016 11:47 p.m., "Mehdi Amini" <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: > > > On Jul 19, 2016, at 3:32 PM, Renato Golin <renato.golin at linaro.org <mailto:renato.golin at linaro.org>> wrote: > > > > On 19 July 2016 at 23:16, Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote: > >>> In the past, we were hit by web spiders that ignored completely the > >>> robots.txt file. Anton has made that better, but it can escalate if > >>> the spider realise we blocked them. There are ways to work around, but > >>> not without accidentally blocking innocent people (mostly in China). > >> > >> That’s not relevant: this is about the WWW server, it does not have to be related to the hosting the repos. > > > > No, this is about hosting the SVN server. The SVN view was disabled > > for months this year before we could really see what was going on. > > I don’t believe the online SVN viewer has to be on the server that hosts the repo that everyone access: the WWW server could mirror the SVN to provide local access to the viewer if needed (hence why I view this as unrelated to hosting source code). > > >> Moving the SVN repo does not solve hosting videos, Debian packages, etc. > >> I suspect most of the bandwidth does not come from `svn up` or `git pull`. > > > > They share the same bandwidth, and sometimes the same server. It is relevant. > > Well, “they share the same bandwidth” is exactly what I mean by “conflating the issues”. > They don’t *have to* share the same bandwidth. Hosting repos could be setup totally separated from hosting WWW. > You need to account things properly. > > > One thing making SVN slow was the amount of Debian packages being > > downloaded form the same place. > > > > > >> Like… proper hooks? > > > > If we can work around it, and it seems we can, this is not such a big issue. > > > > > >> You’re again conflating svn/git and hosting “binaries and videos”. I don’t think we ever planned to host these on github? > > > > No, but they all share bandwidth. We moved videos to Youtube to > > offload the bandwidth, and moving the code to GitHub shares the same > > mindset. > > It shares the same mindset *only* if the code itself is a significant bandwidth consumer, otherwise no it does not make sense. > > > > > > >> Possibly, I don’t know, but that’s exactly why I asked for first hand data on the subject (i.e. Anton and/or Tanya) about hosting the git/SVN repos themselves, instead of hand-wavy “I believe” discussions. > > > > Bear in mind that I gave you facts (bandwidth problems, turned off SVN > > services, constant breakdowns, expertise in handling traffic, backup > > solutions). > > And I consider many of the “facts” you gave to conflate other element than hosting the repository *alone*, which makes it hard to me to see them as relevant as-is. > > > > > I also made you aware that the human cost is not *just* Tanya and > > Anton, but also me and everyone else that maintains buildbots, > > external mirrors, etc. and it *is* larger than the hardware costs. You > > just don't see it because we're all volunteers. > > > > Branding them as "hand-wavy I believe" is *not* appropriate. > > I apologize if I hurt your feeling, but the reality is that I feel you’re conflating multiple things together that are not directly related to “moving the repository only”, and that does not help to be convincing. My use of “hand-wavy”, if that’s what bothered you, means really that (I’m not attaching any other value judgement to this expression as a non-native speaker, maybe it is not the right choice of word from a non-native speaker). > > — > Mehdi-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160719/a3fe7748/attachment.html>
You're not taking into account that we're all volunteers and could disappear overnight. You're taking the stance that this is a company with employees and continuity plans, when in reality, our hours worked are less important than the time it takes for us to fix the problems, which are measured in days, weeks or months, not hours or minutes. Everyone else's time will also be affected (and was) in addition to ours. You're also assuming that we're being compensated or at least doing this on our work hours. I don't know about Anton, but I'm mostly doing this on my spare time (it's almost 1am here, and I'm in bed already). There are things that you can't put a price on, and comparing volunteered personal time with wages and hardware cost is, indeed, a bit offensive. Cheers, Renato On 20 Jul 2016 12:06 a.m., "Mehdi Amini" <mehdi.amini at apple.com> wrote:> > On Jul 19, 2016, at 4:00 PM, Renato Golin <renato.golin at linaro.org> wrote: > > Perhaps it helps to know that I have access to the machines and have > helped debug many of the current problems. I'm not speaking from the > outside, guessing how hard things are. > > I also think you are assuming a lot about where services can be hosted and > at which cost (labour, not hardware). > > It seems you’re assuming as much about me as you think I’m assuming about > this whole thing... > > So, unless you are volunteering to take care of the whole infrastructure, > I suggest taking the opinion of people that are working with it a but more > seriously. > > Sorry, I appreciate your opinion, but it is still not “facts” or “data”. > When you write “it will save *a lot* of maintenance”, I raise my eyebrows. > I’ll take what you write seriously when it’ll be be phrased less > qualitatively and more quantitatively, i.e. for instance I would ask less > if you wrote “currently volunteers (Anton, Tanya, whoever) spends on > average 5h a week to fix problem purely related to the *repositories* (not > the video, not the Debian packages, etc.) and moving to an external host > would save totally these 5h/week”. > > — > Mehdi > > > However, I'd still trust github over anyone of us any day, to host our > repositories. Any one of us could leave the community at any time, but > github, as a company, with many employees and a successful business, will > probably outlast any if us in our current employments. > > From that point of view, the foundation would be better betting their > money on a stable product than on individual volunteers. While it's cool > that some people volunteer, you can't base such a critical system on it. > > Cheers, > Renato > > On 19 Jul 2016 11:47 p.m., "Mehdi Amini" <mehdi.amini at apple.com> wrote: > >> >> > On Jul 19, 2016, at 3:32 PM, Renato Golin <renato.golin at linaro.org> >> wrote: >> > >> > On 19 July 2016 at 23:16, Mehdi Amini <mehdi.amini at apple.com> wrote: >> >>> In the past, we were hit by web spiders that ignored completely the >> >>> robots.txt file. Anton has made that better, but it can escalate if >> >>> the spider realise we blocked them. There are ways to work around, but >> >>> not without accidentally blocking innocent people (mostly in China). >> >> >> >> That’s not relevant: this is about the WWW server, it does not have to >> be related to the hosting the repos. >> > >> > No, this is about hosting the SVN server. The SVN view was disabled >> > for months this year before we could really see what was going on. >> >> I don’t believe the online SVN viewer has to be on the server that hosts >> the repo that everyone access: the WWW server could mirror the SVN to >> provide local access to the viewer if needed (hence why I view this as >> unrelated to hosting source code). >> >> >> Moving the SVN repo does not solve hosting videos, Debian packages, >> etc. >> >> I suspect most of the bandwidth does not come from `svn up` or `git >> pull`. >> > >> > They share the same bandwidth, and sometimes the same server. It is >> relevant. >> >> Well, “they share the same bandwidth” is exactly what I mean by >> “conflating the issues”. >> They don’t *have to* share the same bandwidth. Hosting repos could be >> setup totally separated from hosting WWW. >> You need to account things properly. >> >> > One thing making SVN slow was the amount of Debian packages being >> > downloaded form the same place. >> > >> > >> >> Like… proper hooks? >> > >> > If we can work around it, and it seems we can, this is not such a big >> issue. >> > >> > >> >> You’re again conflating svn/git and hosting “binaries and videos”. I >> don’t think we ever planned to host these on github? >> > >> > No, but they all share bandwidth. We moved videos to Youtube to >> > offload the bandwidth, and moving the code to GitHub shares the same >> > mindset. >> >> It shares the same mindset *only* if the code itself is a significant >> bandwidth consumer, otherwise no it does not make sense. >> >> > >> > >> >> Possibly, I don’t know, but that’s exactly why I asked for first hand >> data on the subject (i.e. Anton and/or Tanya) about hosting the git/SVN >> repos themselves, instead of hand-wavy “I believe” discussions. >> > >> > Bear in mind that I gave you facts (bandwidth problems, turned off SVN >> > services, constant breakdowns, expertise in handling traffic, backup >> > solutions). >> >> And I consider many of the “facts” you gave to conflate other element >> than hosting the repository *alone*, which makes it hard to me to see them >> as relevant as-is. >> >> > >> > I also made you aware that the human cost is not *just* Tanya and >> > Anton, but also me and everyone else that maintains buildbots, >> > external mirrors, etc. and it *is* larger than the hardware costs. You >> > just don't see it because we're all volunteers. >> > >> > Branding them as "hand-wavy I believe" is *not* appropriate. >> >> I apologize if I hurt your feeling, but the reality is that I feel you’re >> conflating multiple things together that are not directly related to >> “moving the repository only”, and that does not help to be convincing. My >> use of “hand-wavy”, if that’s what bothered you, means really that (I’m not >> attaching any other value judgement to this expression as a non-native >> speaker, maybe it is not the right choice of word from a non-native >> speaker). >> >> — >> Mehdi > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160720/4e0108d2/attachment.html>