The power failures caused by Hurricane Sandy in the USA made me think about all the things that can go wrong for a business, including the less obvious problems, when there is a power failure; and how important it is to be able to think and act quickly. In fact it reminded me specifically of a recent incident in SYSPRO and how some quick thinking solved what may have been a much bigger problem.
Late one evening in October, SYSPRO Johannesburg experienced a general power failure in the office. We were made aware of this fact via a text message that was sent to us from our generator. One minute later we received another text message to inform us that the generator had successfully deployed. Three minutes later we received yet another text message to say that power had been restored, and yet another three minutes later we received a final text message from our generator to say that it had successfully run down and stopped as a result of the power returning.
At this point we assumed that all was well, however, this time (unbeknown to us) this was not the case.
Two hours later Dell Systems in Ireland (the suppliers of our Servers) received a message sent by the Storage Area Network (SAN), informing them that the temperature in our server room was reaching a critical state. The core temperature of the SAN had reached 76˚C (168˚F), and the ambient temperature inside the server cabinets was at 45˚C (113˚F). Dell Systems in turn sent an e-mail to our Network administrators. At this point one of our Network Administrators raced back to the office to establish what was going on.
Upon entering the server room, he was hit by a heat wave. The ambient temperature at the door was 36˚C (96˚F) due to the fact that two of the three air conditioning units had ceased to work when the power had resumed. He quickly realized that he needed to act fast to cool the room down – he located some temporary replacement equipment (standalone fans!) and called our suppliers to arrange for the immediate installation of new air conditioning equipment. The order was confirmed at mid-night and installed the next day.
This incident reminded me of how we need to act as an ERP vendor responsible for the continual functioning of our customers businesses, no matter what type of environment they play in. If something goes wrong we need to be on hand, to be able to quickly gain control of the situation and to be able to implement a solution to resolve unforeseen crises. A lot of risk is limited by thorough planning, but a large percentage of quick thinking under pressure goes back to a knowledgeable, problem-solving mind set – something we try to instill in all of our employees from day one.
We pride ourselves on being a vendor that has customizable solutions for any business problem and that is able to adapt and solve problems as soon as they arise. I would like to commend our network administrator for his pre-planning and quick and creative solution-focused thinking in solving a problem which could have escalated and resulted in a catastrophic outcome for our business.
Has your business experienced a scenario where you needed to find a solution to a problem rapidly? We would love to hear more about how you have averted a potential crisis in your business environment!
