University of New Hampshire serves up enterprise data management.
University of New Hampshire CIO Tom Franke hadn't been on the job for a single semester when the back-end storage for Microsoft Exchange maxed out. That made the message “waiting for response from server” as familiar on campus as the university's Wildcat logo. It also essentially set the agenda for Franke's initial few years at UNH.
Few years? That's right. Because ultimately, Franke says, the mail snafu in late 2005 led him to take a look at the data storage and systems that support the Durham, N.H., campus, as well as those shared with three other University System of New Hampshire campuses supported by his information technology team.
“In addition to the Exchange issue, which was the immediate driver, with the advent of more applications coming on board, we also were anticipating a data storage need,” says Joe Doucet, director of enterprise computing at UNH. Specifically, UNH's Computing and Information Services (CIS) planned to roll out two major apps: a library digital content management and a content management system to support administrative systems. Plus, it anticipated “explosive growth” in the use of its course management system.
Like most applications on campus, Exchange had its own silo of servers and relied on direct attached storage. The e-mail support staff had been planning upgrades to overhaul the mail service, as they saw it becoming increasingly unstable. A quick fix would have been a simple server upgrade to keep e-mail flowing. But spending money on something devoted only to e-mail storage didn't make sense, and Franke asked the team to hold off even if it meant more mail problems in the short term. To just revamp e-mail would be a Band-Aid fix to a broader problem: the lack of an enterprise approach to the university's storage and data management needs.
At the time, Doucet's group was looking to purchase a separate storage area network for enterprise applications. Franke and his four IT directors used that as the impetus to move the university toward a campuswide data storage infrastructure. The CIS organization has established an infrastructure that will support storage as a service for the Durham campus as well as provide increased support for Granite State College, Keene State College and Plymouth State University. This approach is the stepping-off point for Franke's efforts to make IT more cohesive and enterprise-driven in the years ahead.
Storage as a service focuses on four specific measures that serve as performance baselines for crafting metrics-driven service-level agreements with university customers. The four measures are:
- written utilization to total raw storage on the floor;
- allocated storage to what's deployed on the floor;
- overall frame utilization or what could be made available on the floor;
- and cost per configured gigabyte per year or month.
A common scenario for the university's IT unit is for a researcher to show up with a near-immediate need for storage for a project. “We typically had to scramble to provide this,” Franke says.
By consolidating its storage and backup needs and migrating apps to an enterprise environment, UNH has bumped up its available storage capacity without increasing its staff. Now, it can make 1 terabyte available in a day. “It's important because we need to be perceived as responsive,” Franke says.
On the back end of the SAN, a pair of Cisco MDS 9506 director-class Fibre Channel switches with a total of 192 ports connects to a pair of file servers that provide fail-over to one another. The SAN array has 30TB of storage capacity, and can expand to 168TB by adding high-performance Fibre Channel drives and high-capacity Serial ATA drives. The storage array also supports the Internet Small Computer System Interface (iSCSI) protocol for Ethernet connectivity.
The combination of FC and SATA drives will let the university offer tiered storage, with the SATA drives offering data availability equal to that of FC storage at a lower cost – $36 per gigabyte for SATA and $39 per GB for FC annually. The SAN hardware provides a foundation architecture that “meets our current and foreseeable storage requirements and will scale to support storage grids as our enterprise requirements evolve,” Doucet says.
nother benefit of the SAN is to increase storage utilization, which hovered between 40 percent and 50 percent before the university switched to network storage. “We had a lot of silo systems with DAS [direct attached storage], and there was no way to access that storage to meet demands” when new projects cropped up or if existing apps spiked demand, Franke says.
Over the next two years, as the IT department migrates apps to the SAN environment, it expects utilization will reach 70 percent systemwide. In the next 12 months, CIS will move all its central applications – finance, human resource, student, course management, data warehouse and other administrative apps – to the new storage platform as part of a larger infrastructure uplift that includes migrating to Hewlett-Packard blade servers running Oracle 10g Real Application Cluster on Oracle Enterprise Linux.
To help it set its baseline metrics and define a road map, UNH worked with a storage consultant. During a two-week assessment, UNH and the contractor scoured the campus to determine the current as-is storage environment and then to plot where it should go.
Securing and managing data also play into the new UNH enterprise data model.
“Various servers located around the university have various backup strategies,” Doucet says. “With storage as a service, people can get the whole package for a reasonable price.”
CIO Tom Franke says the CIS storage service will be so competitively priced that users won't want to go elsewhere.
Photo Credit: SHAWN HENRY
The IT unit's long-term plans involve consolidating all backup services to Veritas NetBackup. Currently, the team uses NetBackup and EMC Legato NetWorker to back up to tape. The next phase will be to establish a private backup network and a virtual tape library for the network operations center, which is about a mile from the UNH enterprise systems data operations center. The IT team has only just begun looking into VTL, Doucet says, but the change fits with the new service model and the goal of making data access quick and transparent to end users. By backing up data to disk, VTL would reduce the cost for incremental backups, ease maintenance and allow for faster restores because it would eliminate the need to retrieve data from tape.
Continuity of operations planning for disaster recovery also figures into the storage program. There had been a separate COOP effort under way, but now the technology component will flow out of the SAN service, say Franke and Doucet. “We don't have to buy a third-party product to do that,” Doucet adds.
Using its SAN, UNH plans to mirror its 500GB Oracle database to hardware at the Keene State data center and provide quick recovery of payroll, finance and HR systems data used to support the University System of New Hampshire campuses. By maintaining a mirror image of the database, UNH can quickly recover data if a crash occurs.
Recovery time previously required manual intervention and restoration from tape. Typically, that would mean an estimated delay of anywhere from 48 to 72 hours from the time a storage system collapsed until data was again available, Doucet says. But should a major disaster befall the data center, the estimate to completely restart operations from tape backups was eight to 12 weeks. That recovery point now hovers at less than an hour for most restores, although a major recovery could take as long as 24 hours.
What Lies Ahead
The challenge now is twofold: convince customers on campus and within the university system to cut over apps to the new storage environment, and then move those apps. The strategy entails winning over customers with cold, hard figures: setting out specific performance targets in service-level agreements and detailing performance in regular reports.
UNH estimates that it can run with its current system for another two years before it would need to make changes. But there will be “no need for forklift upgrades,” Franke notes. And because operations are centralized, shifts to newer technologies such as iSCSI will be feasible without straining the budget, he adds.
So far, the university has spent about $600,000 on the project. “It's difficult to say when we will recoup that cost because the investment is positioning us to be able to manage our storage and provide a service level that was not possible in our old environment,” Doucet says.
Looking back over his first two years at the school and the decision to move to the SAN storage service model, Franke says there's nothing he would have done differently. But he and Doucet agree they want to avoid a situation going forward where a problem such as the Exchange hiccup forces action – to be more proactive than reactive. “The ideal would be that we would be looking further into the future than that,” Franke says.
Because the university is not a top-down management model and most dollars for IT bubble up through projects in the individual schools, it's the only approach that will let the IT unit show true cost-to-benefit ratios, Franke says. “We will try to sell this by making it so attractive that there's no reason they would go anywhere else” for storage.
53 minutes Downtime UNH considers acceptable from its SAN in a one-year period. “We strive for four nines of uptime,” enterprise computing director Joe Doucet says.
55% Percentage of overall IT spending that ultimately flows through the central IT unit.
UNH IT at a Glance
Customers: More than 60,000 users (including some outside the university system at public and nonprofit organizations)
Desktop systems: 10,000-plus (about 25 percent Macintosh systems)
OSes: Microsoft Windows 2003 Server, HP Tru64, Red Hat Enterprise Linux, Mac OS X
Applications: Oracle databases (9i, moving to 10g RAC); Oracle Application Server supporting most front-end systems; Microsoft Exchange and file sharing
Backbone: 10/100-Mbps Ethernet
Wireless: 200 access points in academic areas, libraries, dining areas, administrative buildings and student unions
Internet pipe: 385Mbps
Internet2: Serving all four campuses
Want to Make SAN a Service? Here's How
Before you decide whether a service approach to storage will work, you should assess your current data management. For the University of New Hampshire, that assessment took two weeks and was followed by a series of workshops over a two-month period during which the IT team, working with a consultant, crafted its data management strategy and action plan.
Some questions to answer include:
- How will we better align our IT operations with the needs of the organization?
- What targets do we want to achieve to improve the quality of our IT services?
- How will we become more efficient or achieve cost savings?
- What are the business-critical applications we can't afford to lose?
- How fast are our storage requirements growing
- What are our peak performance needs?
- How much do we need to reduce our backup window, and how fast do we need to recover data?
Answering the above questions will help decide which way to go when designing a SAN. Options include:
- Fibre Channel vs. iSCSI. Fibre Channel SANs move data up to four times as fast as iSCSI SANs, but Fibre Channel is more complex and harder to manage. Based on Ethernet, iSCSI lets you tap into your basic networking knowledge and switches.
- Fibre Channel vs. Serial ATA (SATA) disk drives. Fibre Channel drives are faster, more reliable and more expensive. SATA drives have a higher capacity, but don't spin as fast and usually require more rebuilds.
- Disk backup vs. tape backup. Tape has historically been the most popular medium for backups, but a drop in price in recent years makes disk more attractive. Disk supports faster backups, and is easier to restore from than tape.