How to Keep IT Up and Running During a Disaster
Natural disasters can have devastating effects on IT infrastructure. How can you plan for these events and ensure that your IT remains operational when they occur?
The United States experienced 28 disasters, including storms, flooding, tornadoes and a wildfire, that cost more than a billion dollars each in 2023, according to the National Oceanic and Atmospheric Administration (NOAA). And those were only the most expensive, weather-related events in one country. Around the world, natural disasters, including non-weather-related phenomena such as earthquakes and tsunamis, wreak havoc on human life and on infrastructure -- including the IT that keeps life in the digital age running smoothly.
While the devastation caused by massive events understandably captures headlines, even relatively minor natural disasters such as large storms can affect IT operations. A 2024 report found that 52% of data center outages were the result of power failures. In the last decade, 83% of major power outages were weather-related. Even relatively minor storms can take out power lines.
Fourteen percent of respondents surveyed for InformationWeek’s 2024 Cyber Resilience Strategy Report said that their network accessibility had been disrupted by severe weather or a natural disaster. Sixteen percent ranked natural disasters as the single most significant event they had experienced.
Some businesses affected by natural disasters don’t survive in the first place: according to the Federal Emergency Management Agency, 43% of businesses never reopen and almost a third go out of business within two years. Loss of IT accessibility for nine days or more typically results in bankruptcy within one year.
Only 23% of respondents to a survey on the effects of Hurricane Sandy in 2012 were prepared for the storm. Despite the increasing prevalence of weather-related events because of climate change, the US Chamber of Commerce Foundation found that only 26% of small businesses have a disaster plan in place as of this year, suggesting that few have planned for how their IT will be impacted.
Here, InformationWeek investigates strategies for keeping IT operational when disaster inevitably strikes, with insights from data center operator DataBank’s senior director of sustainability, Jenny Gerson, and industrial software company IFS’s chief technology officer for North America, Kevin Miller.
Preventing Damage to Infrastructure
Depending on the location of an IT facility and the natural disasters common to the region, any number of steps may need to be taken to prevent damage to essential physical IT components.
“We take into account all kinds of natural disasters when we’re looking at where to site a data center -- we try to site it in the safest place we can,” Gerson says.
Jenny Gerson, DataBank
In earthquake-prone regions, buildings need to be able to withstand temblors -- additional reinforcements may be needed to prevent servers and wiring from being disrupted. Operators in areas prone to severe storms and hurricanes may need to both stormproof their buildings and ensure that essential equipment is located above ground level or in waterproof enclosures to avoid potential flood damage. Flood barriers may be advisable in some areas. Attention to potential mold damage after flooding may be necessary, as mold may create dangerous conditions for employees. And fire suppression systems may be able to mitigate damage before equipment is completely destroyed.
Using IoT sensing technology can provide early warning of disaster events and keep an eye on equipment if human access to facilities is cut off. Sensors and cameras can be helpful in determining when it may be appropriate to switch operations to other facilities or back up servers. Moisture sensors, for example, can detect whether floods may be on the verge of impacting device performance.
But, Miller notes, IoT devices can sometimes fail. “We’re seeing customers who are starting to rely more on options like Starlink,” he says. “There’s no physical infrastructure other than a mini satellite dish that’s providing that connectivity -- but [it offers the] ability for them to get data, feed it back, analyze it, and then make predictive assessments on what they should be doing.”
Onsite generators, including sustainable onsite power plants using solar or wind, and microgrids can keep operations running even if access to the main grid is cut off. And redundancy in cooling is crucial for data centers as well.
“Should the utility go down, we have a seamless way to get to our generator backup so there are no blips in power,” Gerson claims. “We always have backup cooling systems.”
Creating Backups
Geodiversity can make or break IT operations during a natural disaster. While steps can be taken to protect operations, they may not always be sufficient to prevent interruption. If a data center or other IT operation is taken offline, the ability to switch over to a location in an unaffected area or to more dispersed, cloud-based operations, can be relatively seamless if proper planning is in place.
This type of redundancy requires careful implementation of regular backups -- cloud technology makes this relatively efficient but hard backups may be useful as well. Setting shorter recovery point objectives, while potentially more expensive in the short-term, will likely make it easier to get things back up and running if an operation is taken offline by a disaster.
IoT devices may be helpful in recovering data that is not fully backed up. Many of these devices store data on their own before transmitting portions of it to the servers to which they are connected. In the case of a disaster, that stored information may be helpful in data restoration processes.
Regulatory Compliance
In disaster-prone regions, it is advisable to proactively facilitate relationships with government authorities and emergency response agencies. This can be helpful both in ensuring continued compliance and assistance in the event of a natural disaster.
“There are certain aspects of [disaster response] that need to be captured,” Miller says. “A lot of times in crisis mode, that becomes a secondary focus. But [disaster management] systems allow the tracking and the recording of that information.”
Being aware of deadlines for compliance reporting and being in contact with regulators if they might be missed can save money on potential fines and penalties. And notifying emergency response agencies may result in prioritization of assistance given the economic imperatives of IT continuity.
Disaster Plans
Having a disaster plan in place will make for smoother operations when a disaster inevitably occurs. Yet, according to InformationWeek's report, only 24% of organizations accounted for natural disasters in their response plans. Given the physical and operational risk posed by these events, it is clear that organizations need to take a harder look at how they may be affected.
While of course some of these disasters are regional -- not everyone will be affected by hurricanes or earthquakes -- others are nearly universal. Severe storms may strike anywhere. Keeping on top of potential risks before they occur, using AI and publicly available information, can make for smoother responses when a storm rolls in or a fire breaks out.
Kevin Miller, IFS
Creating clear channels of communication between leadership is among the most important planning steps. Establishing who is in charge and directing employees on how to modify their workflows ensures that operations will remain efficient. How exactly those workflows will proceed needs to be mapped out as well. Remote work infrastructure, with attendant security protocols, must be established well in advance.
Miller notes that planning can also facilitate mutual aid between utilities and effectively utilizing data from sensing networks to coordinate efficient deployment of collaborative crews who can correct on-the-ground problems more quickly than organizations might be able to on their own.
Organizing a full inventory of physical devices and their functions, as well as other potential vulnerabilities that may be affected, allows for an accurate assessment of how their loss might be compensated for.
“We have twice-annual preventative maintenance on all of our critical systems,” Gerson says. “But they get checked again should there be a natural disaster heading towards a facility: making sure that automated systems are ready to go, making sure our cooling systems are good to go.”
And if supply chains are disrupted, preventing access to necessary materials or technology, backups can be located for them as well.
Protecting data from potential exposure during times of crisis is paramount. Remote work may create certain vulnerabilities and transfer of backups from the cloud and from various locations may create additional exposure.
Rehearsing how backups can be located and implemented with minimal disruption allows for a calm, measured response and minimizes panic and potential hurdles. If a transition does not go well during a rehearsal, the problem can be addressed ahead of time.
Read more about:
Business Continuity/Disaster RecoveryAbout the Author
You May Also Like