Districts Improve Network Uptime With SANs
Jamie Barr doesn't like to think about the consequences of a system crash. As network administrator for Gibson County School District, Barr oversees the technology infrastructure for nine schools in northwestern Tennessee that are highly dependent on their network.
“If something goes down, the whole district is down, and we can't afford that,” he says. “So much of teaching today is technology-driven, and teachers would have to redo lesson plans if we were down for more than a day.”
To prevent such unthinkable scenarios, Barr's team last year consolidated nine servers – one at each school – into a server farm outfitted with an IBM BladeCenter housing three blades; two clustered HP LeftHand NSM 2060 storage area networks with an HP ProCurve 3500 10/100G switch; and an Overland Storage ARCvault 12-tape Linear Tape-Open autoloader for server backup.
Chief among the reasons for the upgrade, Barr says, was a desire to provide more consistent uptime for the district's servers and applications.
Grand Central Station
The implementation of modern, feature-rich SANs is increasingly common in school districts overloaded with data. For IT departments, providing high availability and managing data in a way that keeps it constantly accessible are top priorities.
“Instead of having individual servers, you centralize with SANs,” says Andrew Reichman, a senior analyst at Forrester Research. “By pooling resources, you get a better economy of scale – one software tool used to ship data from one site to another and common equipment at both sites. All of that creates high availability.”
Clustered SANs have made all the difference for Gibson County. Barr's team uses one of the district's three blades as the main controller and has virtualized the others, creating five virtual servers.
“With SAN technology, I can create a virtual server with the host machine and store it on one partition on the SAN. If the host dies, I just get another machine, point it to the partition and pick up where we left off,” he says. “It works the same way if a virtual machine should become corrupted. The data can still be accessed by rebuilding the virtual machine and pointing it to the partition.”
Ponca City Public Schools followed a similar path. When Jason Ridenour took over as technology director for the Oklahoma school district three years ago, he, too, inherited an aging network infrastructure lacking failover capabilities for the district's disparate servers – one or two at each of its 11 schools. “If something went down, we had to scramble to get it back up,” he says. “And it wasn't just once in a while; we were rebooting the main server two or three times a day just to keep it going.”
Ridenour's first move was to transition to a network running Microsoft Exchange Server 2003. He then centralized everything in one location and installed an HP BladeSystem c7000 running VMware, which reduced the number of physical servers from 75 to eight. Each physical server hosts three or four virtual servers.
To ensure high availability, Ridenour deployed an HP StorageWorks Enterprise Virtual Array 4100 SAN with Brocade 4/24 switches. He started with 10 terabytes of storage, thinking it would last the school district for years, but he plans to add another 8TB this summer.
Today, component failures aren't as catastrophic as they once were. “We recently had a problem with a backplane in the SAN, which caused the network to go down,” Ridenour says. “With the old system, it would have caused everything to go down. But because of the redundancy and the new setup, we [kept] running while the backplane was fixed. Nobody knew there was a problem.”
Business Imperative
Alvarado (Texas) Independent School District has gone even further with incorporating SANs into the fabric of its infrastructure. The ISD moved to a Compellent Technologies Storage Center 4 SAN about four years ago. The SAN, which houses e-mail, finance and human resource systems, educational tools and other critical applications, is key to Alvarado's technology mission.
“We issue notebooks for fourth-grade students and up, and we're committed to creating student portfolios that start in kindergarten and go through high school,” says Kyle Berger, the ISD's executive director of technology services. “On the instructor side, we have curriculum-based software and other systems. Classrooms would come to a standstill without them.”
Storage budgets accounted for 17% of the average IT hardware budget in 2009, up from 10% in 2007.
Source: Forrester Research
Berger says high availability is crucial to everything his department oversees, and the SAN, which currently offers roughly 14TB of storage, is the hub. “With all of our SAN's capabilities, we can tier our storage and make sure everything is running all the time.”
The Compellent SAN has worked so well, in fact, that about a year ago Berger decided to see if he could capitalize on the technology. “I approached a school district 45 miles away that also had a Compellent SAN. Since we both had room on our SANs, we decided to serve as disaster recovery sites for each other,” he explains. “We have an Internet pipe that's empty at night and on weekends. So during that time, we push our data to them and they push [theirs] to us. It takes three clicks to set it up.”
Disaster recovery might have been an unexpected outcome of Alvarado ISD's SAN upgrade, but for Lake Elsinore Unified School District, it was a primary driver. The Southern California district serves 25 sites across a network that spans 140 square miles, accounting for 22,000 user accounts, 5,000 computers and 124 servers.
To keep things humming, Systems AdministratorJeff McCullough and his team upgraded network connections and reworked the infrastructure. They retained two legacy servers but added three HP ProLiant DL380 servers and two HP ProLiant DL320 servers running VMware ESX, which together host 42 virtual servers. HP ProCurve 3400cl switches connect the servers to two HP LeftHand NSM 2120-G2 12TB iSCSI IP SAN storage nodes. The SAN, which holds the virtualized servers, has a raw storage capacity of 22TB.
Today, all of Lake Elsinore USD's sites have access to the SAN, “so if we have a server out at one site, we can connect to the SAN and expand storage on the fly,” McCullough says. “If one of the servers is running out of room, I can increase storage capacity in about a minute.”
Having the ability to replicate drives to another site is a bonus, he adds. With virtualization, it's easy to create a virtual machine with the necessary applications and data at another location.
SANs have become so integral to the high-availability equation in these school districts that they all plan to increase capacity over time. “We really are a business, and we think like one,” Berger notes. “That means doing whatever we can to maintain high availability over time.”
Achieving High Availablity
Ensuring that critical applications and data are available on demand is a matter of necessity for all organizations, but there's more to it than simply implementing new systems.
The first step to creating high availability is identifying possible points of failure – critical servers that don't have cluster failover, for example, or single access to network and storage devices. The best way to do this, says industry analyst Greg Schulz, founder of the Server and StorageIO Group, is to shed the perception that high availability and business continuity are simply cost overhead expenses that can be accomplished using standby equipment.
This is where creative thinking by a committed, innovative IT staff comes into play. “If you have redundant network or storage access paths or redundant storage controllers, why not use them in an active mode with workload load balancers spread across those resources, instead of having them in an idle or standby mode?” Schulz asks.
Once potential single points of failure are identified, the next step is specifying the right technology. Using RAID-protected data storage is critical, and the storage area network is a great place to start. SANs centralize capacity, connect to servers in a way that doesn't interfere with local network traffic, and offer important features (snapshots, thin provisioning and deduplication among them) that are extremely helpful in ensuring high availability.
