June 2010 E-newsletter
Planning for Tomorrow
It's difficult to imagine a little bushy-tailed tree squirrel wreaking havoc on a university's operations. But seemingly innocuous, mundane events (such as an errant rodent frolicking on power lines and sparking a blackout) are more often than not the culprits behind major operations calamities.
While media headlines may focus on catastrophic events, it certainly doesn't take a Category 5 hurricane or an act of terrorism to threaten an organization's continuity of operations. In fact, in a global disaster recovery preparedness online survey commissioned by Forrester Research, 42 percent of executives blamed power failures for causing the most significant operations disruptions.
In contrast, hurricanes accounted for just 10 percent of disruptions, and terrorism only 4 percent. IT hardware and software failures, network failures, floods, fires, human error and winter storms rounded out the list of primary factors that have forced organizations to declare a disaster.
Be Prepared for Threats
Regardless of the threat, industry experts agree that the best defense is simply to be prepared. To that end, Denison University is staying ahead of the curve by making continual progress on a continuity of operations (COOP) and disaster recovery (DR) plan for its Granville, Ohio, campus, which supports nearly 2,100 undergraduates and more than 200 full-time faculty members.
“I don't think anyone – students, faculty or administration – can fully anticipate the problems they will encounter if something happens to our services,” emphasizes Lisa Bazley, director of information technology services at Denison, one of the nation's leading liberal arts colleges. “What we're really trying to do here is prevent chaos.”
Denison isn't alone in its efforts to thwart the effects of future disasters. In fact, a variety of factors are driving increased interest in COOP among higher education institutions, according to research from Datamonitor, which provides independent data, analysis and opinion across numerous industries.
In the face of an ever-expanding list of potential threats, colleges and universities are growing more and more dependent on their IT systems to facilitate day-to-day operations. Furthermore, they are often seen as providing a central service to the community as a whole, which can be especially critical during a disaster situation.
And because higher education institutions must not only safeguard data relating to students, but also to significant research projects, grants and donations, and other crucial financial information, there is little flexibility in the length of time an institution has to recover from a disaster. Slow recovery can have significant financial repercussions and seriously cripple a university's ability to offer services, as well.
“We hope that we never have to use it,” Bazley says of Denison's COOP strategy, “but I'll sleep better at night knowing there is a plan in place. It benefits everyone who uses our services.
“If we have a water main break, or something causes a disaster and affects our services here, our goal is just to get things back to normal as quickly as possible,” she adds. “And this [COOP plan] will help us with that.”
Starting with SANs
Though still in the implementation stage, Denison's plan is designed around a pair of storage area networks and server virtualization, coupled with collocation in a network operations center at Ohio State University (leased from the Ohio Academic Resources Network, or OARnet).
After purchasing the first of two EMC CLARiiON CX4 SANs last year, Denison began transferring data from a multitude of individual smaller SANs, each of which was attached to a specific service at the university, including the e-mail system, Novell directory, web services and the ERP system.
“We underwent a massive project of migrating data from a number of different stand-alone sources into the one CLARiiON, which organizes and handles the data very well,” Bazley reports. “It's very efficient.”
The CX4 model selected by Denison not only offers the best energy efficiency, system-level availability and site-level protection in its class, but it also has one of the lowest costs of ownership. Optimized for virtualized environments, the CLARiiON comes preconfigured with Fibre Channel and iSCSI connectivity to deliver up to double the performance and scalability of comparable models.
“EMC has a great reputation in storage area networks,” Bazley points out. “I sought out an EMC solution because of that. There are cheaper solutions out there, but we were very confident and excited about putting together this plan with that model.”
Last fall, Denison purchased a second CLARiiON SAN, which will ultimately be synced with the college's primary SAN and deployed 30 miles away at the collocation site, most likely this summer.
“And from then on, the data will trickle down over our Internet connection and stay in sync, ensuring all of our files are up to date, secure and available in the event of a disaster,” Bazley reveals.
COOP Benefits from Virtualization
Denison is also taking advantage of the benefits afforded by server virtualization by rolling out a VMware solution, which enables the university to host multiple operating systems and applications locally and in remote locations, eliminating physical and geographical barriers.
By consolidating servers, educational institutions can increase the utilization of existing hardware from 10 to 15 percent up to about 80 percent, while reducing hardware requirements by a 10-to-1 ratio or better, according to VMware. In addition to slashing capital expenses and promoting energy savings and high availability of resources, virtualization is also being lauded for its contributions to COOP plans.
Unlike conventional disaster recovery solutions that require the duplication of the entire production infrastructure, virtualization allows Denison to gain rapid and reliable recovery without requiring identical hardware. And because traditional solutions often entail numerous steps that can be difficult to automate, the recovery practice tends to be slow, complex and prone to human error. Virtualization, on the other hand, opens the door to automated disaster recovery, subsequently increasing reliability and availability.
“Budget constraints have left organizations with little appetite to invest in traditional business continuity/disaster recovery solutions that usually involve idle standby equipment,” points out Lynda Stadtmueller, senior research analyst with Stratecast, a division of Frost & Sullivan.
“[Organizations] can't afford to pay for things that might never be used. Instead, they're looking for solutions that do double duty, such as server virtualization, which increases capacity utilization of existing servers and also makes it easy to move applications around.”
Furthermore, the expenses associated with traditional DR efforts often force organizations to protect only their most critical applications, leaving a number of other areas vulnerable.
Yet with virtualization, Denison was able to lower the cost of server infrastructure needed both for production and disaster recovery efforts, while simplifying the process. The university is currently operating 12 production virtual instances, with many more being tested in a virtual system that will eventually move to production, according to Teresa Beamer, network and systems manager at Denison.
“It's nice not having to deal with the extra boxes of hardware,” she says, noting that the university has consolidated a dozen traditional servers into each virtual machine, with the expectation that many more will be added. “We can easily bring in more servers if we need to and move things from one to another for failover.”
“By putting things on the virtual machines, we are able to switch applications around and not require individual hardware,” adds Bazley. “We started with 80 physical servers in our network operations center, and we have since virtualized many of our services. We continue to lower the replacement costs of hardware by simply reducing the amount of hardware in our network operations. This has been possible through virtualization of services and the consolidation of storage.”
Turning to Service Providers
Another step in Denison's COOP and DR project will be connecting several virtual servers to the SAN at the collocation site, “because access to data alone is not enough,” Bazley explains. “Without the actual software services attached to the data, you can't get to it.”
Denison is among a growing number of organizations that are choosing to outsource some of their critical functions to third-party hosting providers. “That's because third-party centers are often built to higher levels of security and redundancy than the organization can afford on its own,” says Stratecast's Stadtmueller.
“Our main focus from the very beginning was to implement a valuable disaster recovery plan,” emphasizes Bazley. “Now we will have a secure offsite location that we will be able to get up and running in much less time than if we were starting from scratch and bringing over tape. We will have the data already current and available, and we'll be able to get into that environment remotely from anywhere. That's the whole point of this configuration.”
“Business continuity plans are giving way to high availability configurations,” notes Stadtmueller. “That means that organizations are investing in networks and equipment that are â€˜always on,' rather than investing in plans to be invoked â€˜just in case.'”
And though its COOP plan is not yet fully implemented, Denison is already reaping the rewards of the overall solution's return on investment (ROI).
“We budgeted for annual replacement servers, but with this solution we didn't have to replace any of them,” Bazley points out. “We put the money into the storage arrays and VMware, which led to the elimination of older servers and other hardware.”
The solution also saves IT staff time that was previously spent maintaining all of the university's individual SANs. “Using the CLARiiONs with virtual services on our campus saves us money,” Bazley says, “as well as time on all the maintenance and upgrades.”
Mapping Out the Migration
When it comes to implementing a COOP plan, Denison has compiled an effective cheat sheet for other institutions. To start with, Bazley suggests that institutions be prepared for a learning curve when organizing and migrating data to a new SAN.
“Don't take for granted that all of the data will come over smoothly,” she warns, noting that Denison endured some migration issues, mainly around its Novell data. “We have had that system for 15 years now, and we had so many old files that had become corrupt. It created a problem for us.”
Additionally, Denison's IT experts recommend conducting some initial trials and analysis prior to launching a full-scale data migration. “Get familiar with the process and do some testing so you know what you're working with,” Beamer suggests.
Bazley concurs. “Do research and conduct tests on certain accounts to see what you're getting involved with,” she says. “We relied heavily on what we were told and would have benefited from a hands-on practical test prior to the planned migration event.”
Beamer also emphasizes the importance of upfront technical training for all appropriate IT staff members, which Denison completed with supplemental resources through EMC and CDW•G consultants.
The opportunity to partner with a company such as CDW•G was also beneficial to the process. “They have just been fantastic with us,” Bazley reveals. “We purchase a lot of our hardware from them. They are always easy to work with and very accommodating. I really value that relationship.”
In addition to providing exceptional service, CDW•G also receives good reviews when it comes to Denison's bottom line.
“We continue to check other sources, but they continue to come in with the best prices,” Bazley reports. “CDW•G is just rock solid for us.”
COOP Best Practices
How to keep your organization operating at its best while preparing for the worst
Numerous events in recent years have heightened awareness of the need to be prepared for the worst. At the same time, IT departments are faced with increasing pressure to deliver 24x7 access to information, as well as continuous operations not just after a disaster, but during one. That's why a plan that helps keep operations running with minimal loss of data, information and productivity is now a necessity. Here are some basic practices to get started:
- Project foundation: Be sure to align management and set expectations, which will help eliminate resistance later in the project.
- Threat assessment: Identify both external and internal threats. Information discovered in this phase becomes the basis for the recovery stages and the COOP plan.
- Strategy selection: Employ recovery procedures to eliminate conditions that would impact operations during a disaster.
- Plan development: Compile a choreographed sequence of actions to mitigate the identified threats and risks and ensure recoverability of key services.
- Implement, test, assess, maintain: Roll out the COOP, ensure that it works, assess it against published standards and be sure to keep it current.