On 27.11.2014 20:39, Chris Matthews wrote:> I can look into it if I get access.Anton, any chance we can get Renato or Chris access to llvm.org to fix this? Tobias
I realized that we never finalized the switchover to Postgres, instead the default database was still SQLite (which has grown huge) and it was shadow importing into the PostgreSQL database. I have now switch it over to only run against Postgres, which I suspect will eliminate the failures we were seeing. Please let me know if you notice any problems. It seems like the switch already gives a big improvement to the response time of the perf homepage. I'm not sure what to do w.r.t. access to the machine, I think the best solution is to try and move LNT off of llvm.org to a machine we don't need to be as careful with. FYI, I attached the LNT log in case anyone wants to look at the errors and see if we can fix the SQLite implementation to not fail on them. - Daniel On Thu, Nov 27, 2014 at 12:26 PM, Tobias Grosser <tobias at grosser.es> wrote:> On 27.11.2014 20:39, Chris Matthews wrote: > >> I can look into it if I get access. >> > > Anton, any chance we can get Renato or Chris access to llvm.org to fix > this? > > Tobias > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141127/caf9f8a7/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: lnt.log Type: application/octet-stream Size: 34149 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141127/caf9f8a7/attachment.obj>
On 27.11.2014 22:56, Daniel Dunbar wrote:> I realized that we never finalized the switchover to Postgres, instead the > default database was still SQLite (which has grown huge) and it was shadow > importing into the PostgreSQL database. > > I have now switch it over to only run against Postgres, which I suspect > will eliminate the failures we were seeing. Please let me know if you > notice any problems. It seems like the switch already gives a big > improvement to the response time of the perf homepage.Great. Thanks for looking into this.> I'm not sure what to do w.r.t. access to the machine, I think the best > solution is to try and move LNT off of llvm.org to a machine we don't need > to be as careful with.The only issue that regularly came up with LNT was the stability. Maybe this is now solved and LNT hopefully does not need so much maintenance.> FYI, I attached the LNT log in case anyone wants to look at the errors and > see if we can fix the SQLite implementation to not fail on them.The sqlite db seems to be locked (possibly due to a larger import). By moving to postgres, we may have "fixed" this issue. Cheers, Tobias
On 27.11.2014 22:56, Daniel Dunbar wrote:> I realized that we never finalized the switchover to Postgres, instead the > default database was still SQLite (which has grown huge) and it was shadow > importing into the PostgreSQL database. > > I have now switch it over to only run against Postgres, which I suspect > will eliminate the failures we were seeing. Please let me know if you > notice any problems. It seems like the switch already gives a big > improvement to the response time of the perf homepage. > > I'm not sure what to do w.r.t. access to the machine, I think the best > solution is to try and move LNT off of llvm.org to a machine we don't need > to be as careful with. > > FYI, I attached the LNT log in case anyone wants to look at the errors and > see if we can fix the SQLite implementation to not fail on them.Bad news, I just got an error 500 again. Could you possibly hand out a version of the logs, Daniel? Tobias
> I'm not sure what to do w.r.t. access to the machine, I think the best > solution is to try and move LNT off of llvm.org to a machine we don't need > to be as careful with.Just a thought. Would it make sense to put LNT server into a Docker [1] container so it's portable and then we can move it over to any (Linux based) host we like easily and reliably? I've been playing around with Docker lately (I really like it) so I'd be happy to hack something together for you to try out. I don't have much experience with LNT though and I don't know how to implement database fault tolerance with. I presume we would just have a separate container for the database but I'm not sure how the replication would be done. A possible home for these docker containers could be Google compute engine [2]. Google do make use of LLVM so I wonder if they would be willing to provide free cloud hosting services for the LLVM project. There are of course many other cloud platforms providers (e.g. Amazon EC2, Digital Ocean, Tutum...) but I'm not sure if they would be willing to provide free compute resources (and support) for us. I would hope (I don't know for sure) that these services would allow multiple users to manage the containers so that we could have multiple people able to manage them rather than having the single point of failure like we do now. But maybe this is too ambitious... Thanks, Dan. [1] https://www.docker.com/ [2] https://cloud.google.com/compute/docs/containers