01.08.2016

First of all, please accept our apologies for Saturday's power outage in MasterDC Prague data center.

During Saturday afternoon we have been dealing with a power problem in the tower server room. When switching the technology for backup power line, there was a human error which caused the outage of primary power branch. Customers with dual power service as well as the cloud hosting customers did not experience any outage. The entire power backup system including two separate power branches, UPS units and diesel generators are designed to standard power outage. On Saturday, unfortunately due to human intervention an abnormal outage occured and that is why even diesel generators could not be started. As we informed our customers, power has been successfully restored in about 30 minutes. Promptly after that, our technicians started to help customer with restoring their servers. Monitoring system evaluated an abnormal behaviour of our network elements, namely switches, with few malfunctioning ports. Technicians in Prague and Brno administrators took care of the transfer to the working ports and the whole night and during Sunday actively dealt with customer requirements and assisted in restarting their technology.

We have information about the units where even customers with the Dual Power from two branches experienced power outages. As one of the possible causes, we evaluated a wrong wiring of customer technologies in rack. If you are interested in help with this matter, please contact us at support@master.cz.

Based on the outage, we have decided to improve the process of crisis intervention through the following measures:

  • 1) accelerating of deployment the information messages into the IVR system - this functionality was already in operation but all competent people had to concentrate on their part of the job associated with technical intervention.
  • 2) communication on social networks - we communicated the outage via www.twitter.com/MasterDC but our social networks are not managed by employees with 24-hour emergency service.
  • 3) we are also considering further expansion of our team "on-call" in order to be able to quickly deal with unusual situations in the minimum possible time.

We apologize again for any inconvenience this outage have caused. If you have any further questions, please contact the Prague Sales Director Petr Štěpánek on stepanek@master.cz. If you want to say something directly to me, I am - even on vacation - available on spacek@master.cz.

Thank you for the support that we have from many of you received, and thank you for staying with us.

On behalf of Master Internet

Filip Špaček, Chief Operating Officer

The right place for your data

OUR DATA CENTERS ARE LOCATED IN PRAGUE AND BRNO