SILVA outtages

coping with data-center meltdown

Yes, SILVA has been offline. In a week we would have reached two years of continuous uptime, but on Monday at 10 AM GMT our whole data center went down.

We had thought to be well prepared for the expected power outages during the yearly maintenance of the electrical systems at the Max Planck Institute in Bremen. By the time we noticed something was going wrong, our racks were already cooking at >95 °C. Somewhere in the catacombs beneath us, the pumps circulating the coolant had lost power. It took less than five minutes for the first servers to crash.

Todays outage was an indirect consequence. We're still repairing things and hardware that has been brought to the edge by the heat is continuing to fail.

We apologize for any inconveniences our down time may have caused you and hope that you will bear with us as we continue to try keeping the web site and its services running.

---