An established disaster recovery plan helped the credit union serve members after a fire destroyed its data center.
By Dahna Chandler
The ordinary day changed, however, when the heat of a blaze replaced the heat of the summer.
“A fire began on a third floor balcony; the fire alarm sounded and we evacuated,” says Nicole Bowen, vice president of compliance, information technology, and facilities at the five-branch credit union. “I was not expecting an actual fire, although the fire alarm was unexpected and I knew it was not a planned drill.”
CU QUICK FACTS
Data as of 09.30.15
- HQ: Fairfax, VA
- Assets: $321.8M
- Members: 14,903
- Branches: 6
- 12-MO Share Growth: 14.98%
- 12-MO Loan Growth: 45.14%
- ROA: 0.98%
From her position in the back parking lot, she could see flames in the upper right corner of the building, above where the facility’s production data center is located. In fact, Fairfax FCU’s headquarters location houses all of the facilities back office, IT, lending, communications, and executive staff as well as a full-service branch.
“At that point, I knew it was show time,” Bowen remembers. It was time to put the credit union’s well-planned disaster recovery and business continuity plan — and the credit union team — to the test.
Putting Mission-Critical Systems Back Online
|Nicole Bowen, Vice President of Compliance, Information Technology, and Facilities, Fairfax County FCU|
Unlike natural or large-scale disasters, which affect multiple businesses at once, this fire involved only a single credit union. Having critical systems offline could be detrimental, and the potential loss of data put Fairfax County FCU’s ability to serve members in jeopardy.
“We have an important role to play in the financial lives of our members,” Bowen says. “We are the best financial partner for our members, and we always want to be available to serve them.”
The credit union’s reputation for playing that role well was at stake. Knowing this, Bowen set about keeping those critical systems online.
“While the fire department worked to extinguish the fire, we monitored our network from a remote location,” she says. “As long as we were able to connect to our production systems, we knew the data center was not catastrophically affected.”
But within a short while, Fairfax FCU’s system went offline and was down for several hours while the fire department got the blaze under control. During that time, Bowen and her team had no access to the facility and no way to determine the extent of the damage. Once they gained access to the building, they found the credit union’s production data center had suffered significant water damage and its servers and other equipment had been destroyed.
“The immediate impact was loss of service across several delivery channels for a period of time,” Bowen says. “Our branches were not able to provide service for the last two hours of the day. Online, mobile, shared branch, and telephone services were interrupted for the remainder of that first day as well.”
Although the fire occurred on a late Thursday afternoon, the credit union was able to open all facilities the following morning thanks in part to its disaster recovery and business continuity (DR/BC) plan. According to Bowen, the credit union opened for business on time with limited services and restored full service later that day.
“Following the immediate restoration of member services, we operated for many months out of our recovery data center,” she says. “It took a long time before operations were back to ‘normal,’ but we suffered no loss of data, which was our goal.”
Frequent and good communication sets a tone of calm and control so your staff can focus on the task of recovery.
Crisis Communications Mode
Bowen quickly learned the period immediately following an event like the fire is tremendously active, and measured and planned communications with staff and members is key. Once the credit union had restored its data production systems, Bowen facilitated the restoration of its communications channels and launched a multichannel communications strategy.
“Communication is critical, and the crisis communications plan is an integral part of any disaster recovery scenario,” Bowen says.
Fairfax County FCU’s DR/BC crisis communications plan focused on communicating an approved message about the fire and the credit union’s response. The plan kept information flowing among staff and executives as well as between the credit union and its members.
Yet, according to Bowen, flexibility in crisis communications is as crucial as clear structure. So is leveraging the multiple communications channels available today. And, because it is not possible to predict which communications channels will be available in a crisis, an adaptable strategy allows organizations to choose those best suited to the situation.
“A great plan includes a mix of high-tech, low-tech, and no-tech communications channels,” Bowen says. “High-tech channels include mobile/SMS, email, social media, and Internet. Low-tech communications include telephone blasts, hotlines, radio, and television. No-tech communications could be as simple as branch signage.”
Fairfax County FCU already had messaging templates in place as part of its crisis communications plan, so it was able to communicate on brand and stay on message with the public. It used modern smartphone technology to send mission critical group texts to staff and reassuring messages to members.
“With a few clicks of a smartphone, information about your event can reach thousands,” Bowen says.
The credit union also relied on internal messaging platforms online to communicate with members and staff as well as social media to communicate with external stakeholders.
“Keeping everyone well informed with timely, appropriate communications minimizes inconvenience to your members and staff,” Bowen explains. “Frequent and good communication sets a tone of calm and control so your staff can focus on the task of recovery.”
Solid Planning Nets Positive Outcomes
By having a solid DR/BC plan in place, the credit union minimized damage to its daily operations and maximized opportunities to serve its internal and external stakeholders. It stayed on message and on brand and enhanced its corporate image by restoring services quickly. Being able to communicate effectively throughout the crisis was an important part Fairfax FCU’s successful recovery, Bowen says, especially with regard to staff communications.
“We did not suffer any damage to our reputation and brand, and member satisfaction remained high,” Bowen continues. “Frequent, timely, and appropriate communications with our members ensured they were able to receive the service they need to continue with their daily lives. It also helped eliminate any uncertainty that can lead to dissatisfaction or panic if not managed appropriately.”
8 Rules To Overcome A Disaster
- Have a complete DR/BC plan in place. Constantly review and revise it, particularly as technology changes and the organization employs newer or additional technology.
- Incorporate a strong crisis communications plan into the DR/BC plan. Include templates containing key messages to convey during a crisis or disaster.
- Include vendors in the DR/BC plan. This is particularly important for the communications crisis strategy because they can be of great support.
- Integrate staff into the DR/BC planning process. Then use them for the plan’s actual execution when disaster strikes.
- Distribute the credit union’s workforce and resources when implementing the DR/BC plan. This will speed recovery and help the credit union respond to situations as they arise.
- Show appreciation to staff and others that help during the disaster recovery process.
- Design the disaster recovery strategy to meet recovery point objectives that are acceptable to the credit union.
- Stay on message and on plan throughout the recovery process. This minimizes data loss and prevents reputation damage.