By Tommy Jacobson
MCNC Vice President and Chief Operating Officer
The 2018 hurricane season was the most active season ever recorded, according to the National Hurricane Center. It was particularly unforgiving for many coastal communities in North Carolina. As hurricane season officially concludes today (Friday, Nov. 30), I was thinking back to September when Hurricane Florence impacted much of North Carolina, slamming the coast with high winds and record amounts of rain.
MCNC operational teams and staff worked well together leading up to the storm to communicate with constituents on MCNC’s readiness and to gain an understanding of what locations planned to turn off equipment. Additionally, the fiber and facilities staff did a really good job getting out to all sites in the path of the storm for site inspections to make sure all facilities were prepped as best they could be.
During the peak of Florence, there were 148 outages for endpoints served by NCREN that were largely the result of localized power events, telco-related outages, or in some cases flooded facilities. As with Hurricane Matthew two years prior, all MCNC hut locations east of I-95 at one point in time were operating via on-site emergency power generators due to facility power being unavailable. Generators ran for extended periods of time at our hut in Wilmington, as well as in Fayetteville, New Bern, Morehead City, Pembroke, Whiteville, and Bolivia. NCREN core optical and IP networks did not experience an outage.
Once again, another major hurricane impacting the state within the last 25 months and the core infrastructure worked just as designed. All sites went to remote power as required and all core operations related to the core facilities, optical, and IP networks remained online and fully operational.
MCNC staff focused efforts on reaching out to those impacted to determine the status of power and then escalating any issues dealing with a combination of failures of the electrical grid and last-mile service delivery from telco providers. In addition, the MCNC operational team communicated frequently and effectively in a remote setting resolving any other added issues as they became known. As the storm moved on, proper escalations with providers and MCNC management around critical sites that were down commenced.
Communication with MCNC senior management occurred on a daily basis, making sure everyone was aware of current status and requirements for assistance as warranted. The MCNC Business Continuity and Disaster Recovery (BCDR) Team also was activated and provided real-time communications to alert constituents of the current status of NCREN as the storm worked its way out of the state and operations returned to normal. Read the full recap all NCREN advisories posted on our website and shared on social media throughout the storm.
Hurricane Florence remains a weather event that many in North Carolina will not forget or recover from for years to come. Moving forward, MCNC will continue to fine-tune our process and procedures during emergencies and weather-related events. For example, we are looking at upgrading external video monitoring capabilities at our huts to see potential flooding conditions and wind conditions.
If you have some additional ideas, please reach out to us.
Blogroll & Previous Posts by Tommy Jacobson
(Satellite Image of Hurricane Florence courtesy of the National Oceanic and Atmospheric Administration)