By Allan Graham (Part 4 in a 5-part series)
In part 3 of this series, we covered work area recovery issues related to creating a backup trading room facility. Now we look at issues related to protecting people and maintaining data and voice services.
No matter how sophisticated and fully equipped the backup production facility is, it’s of no value if the required people can’t get there in a crisis. And their ability to get to and effectively use the backup facility depends on a number of factors depending on the type of event:
- Commuting pattern – How far are employees currently traveling to work? How far would they have to travel to the backup facility? Is the facility close enough to be practical but far enough away to minimize the likelihood it would be caught up in the same event? Would the employees need to stay in hotels?
- Dependents – Do the employees have dependents that need to be accounted for during a crisis? What if the plan trigger is a snow storm or flood that closes local schools? How will the employees manage childcare and pet care? Are there childcare and kennel facilities located near the backup facility?
- Safety and comfort – Is the backup facility in a safe area? Does it provide creature comforts (access to food, clean restrooms, etc.) that will support sustained work effort?
- Crisis management – Will crisis management phone trees be operational? Will management be able to check in on employees to ensure they are safe and accounted for?
- Backup roles – “Eliminating a single point of failure” is an established practice for the IT infrastructure, but it is also important when it comes to people. No one person should be so vital to business operations that his or her lack of availability would prevent recovery. This requires that multiple people be trained for each critical operational role.
In bringing data services to the backup facility, it is vital that the backup network infrastructure be as robust and reliable as your primary network infrastructure. It should be fully redundant and continuously monitored, and it should have no single point of failure. It should rely on multiple paths and multiple providers, so that if one provider is affected by the crisis or overloaded during the recovery, you have access to a second or third provider. To do this, make sure your backup infrastructure is located in a network-dense facility that provides the maximum options for connectivity.
Numerous options exist for ensuring application and data availability. These options are driven by three factors:
- recovery targets (RTO/RPO)
- technology in use
Data replication technology and network bandwidth requirements vary greatly. For example, a company running a mission-critical trading application in production may choose to employ a geographically load-balanced active/active solution for its backup platform. This solution can instantly failover without a service interruption thanks to synchronous data replication over very high-speed data circuits. While a very low risk-option, it is quite expensive and technically complex.
By contrast, a company with a less critical email system to protect may choose periodic file “snapshot” replication running on a much slower network connection. Today many people are looking at the services offered by the burgeoning cloud market as a way to balance recovery targets and budgets. Ultimately, the difference in cost and technology used should be a cooperative effort between the business unit and the IT department under the guidance of the business continuity organization.
Financial services firms must maintain access to diverse market feeds. Just as you plan for a network-dense facility with the maximum options for network connectivity, make sure your backup facility can provide access to all the required market feeds.
Voice services are critical both to continuing business operations and to communicating during the immediate response to the crisis. Ensure your backup facility provides access to multiple telecom providers, and consider leveraging voice over IP (VoIP) services, which can provide voice communications over the more robust and flexible internet network in the event that the telephone system is not operating.
In the final post of this series, we’ll present 10 best practices that will help focus your thinking and guide you toward a more successful BCP effort.