Derek Atkins
2011-Feb-24 19:44 UTC
ANNOUNCE: Unexpected GnuCash Server Downtime: All Services Restored
Hi All, You may (or may not) have noticed that the GnuCash Email, SVN, etc. server was down since Tuesday. The server should be back up now (as of 13h14 US/EST) and all services should be running normally again. Please let me know if you see any issues. Failure breakdown: In short, the system lost a CPU fan (again!) and it shut down due to overheating. I replaced the fan and the system came up again. Long version: (Note that all times are in US EST (GMT-5)). I know the server was still alive as of 11am on Tuesday when I was in the Johannesburg airport and checking my email waiting for my flight. I know that it had shutdown sometime before 5pm EST on Wednesday when I tried to access it from Newark airport, the next time I had Network access. (The timing of this failure was spectacularly convenient considering I've been halfway around the world for the past three weeks!) When I finally got home at 11:30pm on Wednesday I noticed the system completely turned off; I powered it on and it started to come up, but 10 minutes later the overheating alarm came on. Considering my jetlagged, exhausted self (and the annoying noise it was making) I decided to shut the system down again overnight and deal with it this morning. This morning I opened up the box and discovered that one of the CPU cooling fans failed. Strangely this was the same CPU that had a failed fan in September. This time, however, instead of burning out it had thrown a baring. Since this was the same model as the original I decided I needed something better for this CPU so I started researching better cooling fans and found one available from Fry's Electronics. I went out this morning, purchased the cooler, and then installed it into the system. The main system was powered up at 12:24 today and then all the VMs slowly came back up. Many of the VMs decided to FSCK due to the unclean shutdowns and finally the last VM returned to service at 13:23. I'm sorry if the downtime caused any inconveniences. As I said, the timing was as perfect as it could have been considering my travel schedule this month! -derek -- Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory Member, MIT Student Information Processing Board (SIPB) URL: http://web.mit.edu/warlord/ PP-ASEL-IA N1NWH warlord at MIT.EDU PGP key available