The cloud instance would need funding to run at the scale the llvm.org <http://llvm.org/> server runs at. It can only accommodate about 5k submissions worth of data at its current size. I assume that would be exhausted pretty fast with a bunch of machines submitting several times per day.> On Dec 1, 2014, at 9:49 AM, Tobias Grosser <tobias at grosser.es> wrote: > > On 01.12.2014 18:45, Chris Matthews wrote: >> This was part of the motivation my cloud LNT instance. >> >> The Heroku cloud which I am running on is using a postgres cluster, not a single machine. It is well tuned for large databases. > > If it is maintained and well tuned, maybe we can just point llvm.org/perf there and submit to this instance? > > Cheers, > Tobias-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141201/eb219fab/attachment.html>
Hi Chris, On 1 December 2014 at 18:31, Chris Matthews <chris.matthews at apple.com> wrote:> The cloud instance would need funding to run at the scale the llvm.org > server runs at. It can only accommodate about 5k submissions worth of data > at its current size. I assume that would be exhausted pretty fast with a > bunch of machines submitting several times per day.I haven't used Heroku but if it's working and they provide access to a postgres cluster (is this [1] what you're using?) then that sounds very handy. Given that the problem is database reliability, Docker isn't really going to solve this problem on its own. I will probably still make a container but it's going to be for a separate project of mine so that will be orthogonal to this thread. So I guess two questions come to mind: * Is Heroku is the right choice (we don't want to be locked into Heroku)? Given the progress you've made it sounds suitable. * Who should fund whatever host is used to host the LNT infrastructure. Given the commercial interest in LLVM I hope that this will be straight forward Presumably this is what the LLVM foundation [2] is for? Should we start a new thread CC'ing the foundation directors on the subject of finding a host for the LNT infrastructure? [1] https://www.heroku.com/postgres [2] http://blog.llvm.org/2014/04/the-llvm-foundation.html Thanks, Dan.
On Thu, Dec 4, 2014 at 8:00 AM, Dan Liew <dan at su-root.co.uk> wrote:> * Who should fund whatever host is used to host the LNT > infrastructure. Given the commercial interest in LLVM I hope that this > will be straight forward >FWIW, if you can use google's cloud offerings, I can likely fund it. This isn't about only being willing to fund our platform vs. some other platform, or which platform is better. It's just that we're already well set up to handle this when its on top of GCE and related services -- this is how we are currently running the phabricator instance and we're working on setting up bots here as well. But I have no idea if GCE and related bits are even useful for what LNT needs. Just let me know if they are, and I'll try to get things rolling. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141204/0b9c2bdf/attachment.html>
I am not particularly attached to Heroku. I chose Heroku because their cloud is particularly suited to LNT. They have a section in their docs about how to port a flask app (the web framework LNT is built on) to Heroku. To get LNT working on Heroku took about 10 lines of code, and 3 hours of mucking around. It is transparently HTTP load balanced, is running on a Postgres cluster, has email service etc. So Heroku made it dead easy. Since Heroku required no changes to LNT, we are certainly not locked in, it just provided an great service, that got rid of all the admin futzing I’d have to be doing to keep a server like this going. For background, original post is here: http://article.gmane.org/gmane.comp.compilers.llvm.devel/75774 <http://article.gmane.org/gmane.comp.compilers.llvm.devel/75774> I think it would be great if we could get it running on GCE too. I think the important part is to get away from maintaining “servers". That is a big time suck, and get us nothing. We also need to leverage features we just can’t get at a small scale, like real redundancy and reliability, database clusters. That is how we can derive value from using these services.> On Dec 4, 2014, at 8:00 AM, Dan Liew <dan at su-root.co.uk> wrote: > > Hi Chris, > > On 1 December 2014 at 18:31, Chris Matthews <chris.matthews at apple.com> wrote: >> The cloud instance would need funding to run at the scale the llvm.org >> server runs at. It can only accommodate about 5k submissions worth of data >> at its current size. I assume that would be exhausted pretty fast with a >> bunch of machines submitting several times per day. > > I haven't used Heroku but if it's working and they provide access to a > postgres cluster (is this [1] what you're using?) then that sounds > very handy. > > Given that the problem is database reliability, Docker isn't really > going to solve this problem on its own. I will probably still make a > container but it's going to be for a separate project of mine so that > will be orthogonal to this thread. > > So I guess two questions come to mind: > > * Is Heroku is the right choice (we don't want to be locked into > Heroku)? Given the progress you've made it sounds suitable. > * Who should fund whatever host is used to host the LNT > infrastructure. Given the commercial interest in LLVM I hope that this > will be straight forward > > Presumably this is what the LLVM foundation [2] is for? Should we > start a new thread CC'ing the foundation directors on the subject of > finding a host for the LNT infrastructure? > > [1] https://www.heroku.com/postgres > [2] http://blog.llvm.org/2014/04/the-llvm-foundation.html > > Thanks, > Dan.-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141204/7f54dd80/attachment.html>