SAN Sense
By moving to a SAN, this University of Florida IT department gained flexibility and uptime.
For more than 25 years, the IT staff at the University of Florida’s Department of Housing and Residence Education knew its mission. It managed the e-mail accounts, database access, and file- and print-serving for its 400 staff members. With Florida facing major budget shortfalls, however, the university decided to almost double the department’s load. Taking on four more departments with 300 additional users threatened the department with a tidal wave of data that might overload IT’s infrastructure, especially the university’s data-storage systems.
The existing direct-attached storage system couldn’t support the new additions, and IT managers knew storage space would soon run out under the onslaught. Compounding the storage problem was the overall need to accommodate a larger number of users, which meant drastically reducing downtime and creating more redundancy in the system.
Downtime was a particular nuisance with the existing system. The department had as much as five to 10 hours of unexpected downtime per year because of insufficient storage space and equipment failures. Some outages lasted several days.
“We knew it didn’t make sense to buy more direct-attached storage,” says Charles Benjamin, the department’s senior network administrator. “We wanted to go with a technology that wouldn’t be outdated soon, but not bleeding-edge technology. We saw this as an opportunity for change.”
To levy that change, Benjamin and Network Administrator Matt Pendleton decided to move to a storage area network (SAN), which afforded greater flexibility, reliability, redundancy and scalability. It was a big step, but ultimately it made all the difference.
“It was really important for us to achieve less than one hour of downtime per year, and we knew we couldn’t do it with direct-attached storage in a Windows environment because that’s not the preferred method for clustering for Microsoft,” Pendleton says.
Because of the additional load, it was even more important to create a system with more redundancy — something the current setup didn’t support. The more urgent problem, however, was that the department was simply running out of storage. With its current storage already at about 80 percent of capacity, Benjamin and Pendleton’s problem was quickly turning into a crisis.
What to Do?
They knew enough about the technology to know that only a SAN would solve their problem.
“It made sense in so many ways,” Benjamin says. “Nothing else would give us the efficiency a SAN model would provide, the redundancy we needed or the ability to grow our capacity.”
Benjamin and Pendleton initially evaluated several SAN vendors, looking for the most cost-effective, flexible solution. In the end, it was clear that only LeftHand Network’s SAN/iQ fit the bill. Considering the budget crisis, cost was by far the most important consideration.
“Price was an important factor, and prices between vendors varied considerably,” Benjamin says. “One vendor actually came in at double the price of the LeftHand solution. In looking at a number of vendors, we found that LeftHand was the most cost effective.”
Cost was only part of the equation, however. What really sold the IT team was SAN/iQ’s unique ability to grow volumes on the fly without disrupting Windows. Pendleton saw it firsthand by visiting existing installations with LeftHand regional sales manager Scott Sager.
“When I saw it in action, I couldn’t believe how fast it could configure the equipment — just 10 minutes,” Pendleton says. “If I have a 100-gigabyte volume and I’m reaching my max, I can tell the system to increase it to 200GB. Normally, with Exchange, if you change the size of a volume while Exchange is running, it will go down. This won’t.”
The system’s flexibility also became apparent when Pendleton saw the system up close.
“We needed storage flexibility for both our department and the other department we support,” Benjamin says. “And technology is not static; we don’t know how many files we will need to store in the future, so we need the capability to be flexible with our storage.”
The ability to create a redundant system that would see the department through any type of disaster also was critical. That ability set SAN/iQ apart.
“They needed a solution where they could create a synchronous replication across the two data centers, and SAN/iQ offers the ability for real-time replication across multiple devices and sites,” Sager explains. “Essentially, they were looking for a situation where they could stretch a single SAN across two sites in active-active mode, which is pretty unique.”
After evaluating a number of vendors and technologies, the team decided to go with four Hewlett-Packard DL320 servers loaded with LeftHand Networks SAN/iQ v.7.0 software.
Settling In
In preparing for the installation, Benjamin and Pendleton found there wasn’t much to do. They just had to check that the fiber between the main server room in the IT department and the planned secondary site in another building across campus was functioning properly. The department didn’t need to purchase additional switches or fiber optics to get the system in place.
When the servers arrived in July 2007, Benjamin and Pendleton decided on a two-step implementation. The first step was powering up all models and letting them “burn in” by running in test mode before being put into production. To satisfy Microsoft’s best practices, which specifies that databases and logs have separate volumes, the IT staff used LeftHand Networks’ Centralized Management Console program to set up volumes for all applications and servers that would use the SAN.
Full installation was the next step, and it went very smoothly, according to Pendleton.
“It was as easy as installing a server into a rack,” he says. “Once you have your network in place and the fiber between locations in place, it’s not difficult. All we had to do was plug in two modules in one location and two in our other locations and we were up and running in 20 minutes.”
And because the SAN is based on the Internet Small Computer System Interface (iSCSI) protocol, the team was able to change the location of the data easily by moving it from existing HP StorageWorks direct-attached storage to the LeftHand Networks SAN. “It was as simple as copying data from one to another,” Pendleton says.
The support the IT staff received from LeftHand Networks and CDW•G also made the switch easier. Not only did LeftHand send an engineer during installation to make sure everything was set up and running the first time, but it also included training for three IT staff members with the purchase, Benjamin says. LeftHand even sent a new module overnight when another went bad soon after installation, he added.
LeftHand also provided certification training. By fully certifying the IT staff, there would be no problems down the road with performing installations and upgrades or training additional staff, Sager says.
Now that it’s installed, two of the HP DL320 servers with Serial Attached SCSI (SAS) drives running SAN/iQ are part of a server farm located in the IT department. The servers, running Windows and Unix, are connected to two Gigabit Ethernet ports running through a Cisco Catalyst 6500 switch to a second location in a residence hall, which houses an additional two HP DL320 servers running SAN/iQ. A third location connected by fiber runs LeftHand’s failover storage module that serves as a quorum device for the cluster.
The Payoff
Today the SAN runs in the background. The 700 users in five departments don’t even know it’s there — only that things are running smoothly.
“We don’t have to think about it unless we need to grow a volume, and most of the time it auto-grows,” Pendleton says. “When it reaches a particular threshold, I just have to increase it, but it’s very quick. I kept testing the LeftHand support people by growing volumes during the heaviest part of our day, with no problems,” he says.
Pendleton says he will soon begin formally evaluating the SAN’s return on investment by comparing the department’s historical downtime with current downtime. With the addition of the LeftHand SAN, the department now experiences one hour or less of unexpected downtime per year, he says.
“I already know what we’ll find,” he says. “It’s definitely saving us money in terms of not having to purchase more storage since it can grow volumes. And it’s saving us work-hours, because we don’t have to dedicate overtime when something goes down at night because we are confident that this stuff will still be up. And we can forget about it and focus on other tasks in the IT department.”
Now that the SAN is running so well, Pendleton and Benjamin are looking toward the future, but Pendleton says adding more storage won’t involve further infrastructure changes.
“If we decide to add more space, it’s as simple as buying more storage modules from LeftHand, and once we receive them, it will only take 15 minutes — most of that boot time — to see the storage show up in the Centralized Management Console, so it’s available for allocation,” he says.
University of Florida at a Glance:
- Largest and oldest university in Florida
- Annual operating budget of more than $2 billion
- More than $500 million in research awards during 2005–2006
- Enrolls approximately 50,000 students annually
- 140 academic departments
- 500 IT professionals across the campus and the state
- 2,000-acre campus
- More than 900 buildings
- More than 4,000 full-time faculty and 8,000 staff members
Steps to a Successful SAN Implementation
Implementing a SAN can be a simple process — or a giant headache. Achieving the former and avoiding the latter is a combination of planning and support. Here are some tips on how to ease into the world of SANs:
- Find out in advance if your current network infrastructure supports the requirements for the SAN you choose.
- Decide before the installation where you will place the SAN storage modules, how you will configure it, your availability and performance needs, and your overall objectives and goals. If you do this in advance, you can solve some implementation problems in the acquisition phase.
- Communication with staff is key. If IT managers don’t tell end users in advance to expect downtime, for example, there might be some negative feedback. IT managers at the University of Florida’s Department of Housing and Residence Education publish maintenance windows to avoid this problem.
- Operate IT with customer service in mind. “If you use customer-centered principles rather than self-serving policies as your backbone of operation, then your tenure as an IT manager will be much less stressful and more rewarding,” Benjamin says.
Department of Housing & Residence Education IT at a Glance
Responsibilities: Manage the e-mail accounts, database access, and file and print serving for 700 users in five departments; maintain the data intranet infrastructure for more than 8,500 University of Florida students living on campus.
Storage: Four HP DL320 servers with SAS drives, with a maximum capacity of 300 gigabytes (a total of 12 terabytes of storage) running LeftHand’s SAN/iQ, v.7.0, on a Windows Server and Unix.
Network setup: Two HP DL320 servers running SAN/iQ connected to two Gigabit Ethernet ports and running through a Cisco Catalyst 6500 switch to a second location in a residence hall, which houses an additional two HP DL320 servers running SAN/iQ. The switches are connected on a fiber-optic backbone across campus, with dedicated fiber optics between the main server room and the secondary backup location.
Additional servers: About 40 HP ProLiant servers, either model DL360 or DL380, running additional network services and critical user services such as e-mail, file sharing, print server and database. All servers are connected through the Gig-E network using an Internet Small Computer System Interface connector, which allows the department to securely access the information stored on the SAN.
Applications managed: E-mail through Microsoft Exchange 2003 Enterprise; Microsoft SQL Server 2000 and 2005 databases; open-source MySQL database for network technologies and an in-house peer-to-peer detection system; and Microsoft Windows Server 2003 R2 Enterprise for file sharing and print server services.
Backbone: Gigabit Ethernet on Cisco Catalyst switches connected throughout the campus by fiber optics. Each point-of-presence (POP) is connected through fiber optics. There is a POP in each residence hall and residence village, as well as in each area the department supports.
iSCSI or Fibre Channel?
The type and amount of data an organization must store, how mission-critical that data is, how secure it must be, how distributed the environment is, how many regulatory requirements must be complied with, how quickly storage needs are expected to grow and the storage infrastructure currently in place are all important factors in determining the best type of storage area network for a particular environment.
Depending on the answers to those questions, IT managers then can determine whether to base the new SAN on the Internet Small Computer System Interface (iSCSI) protocol or Fibre Channel, a gigabit-speed network technology that’s also used with enterprise-level storage networks.
Many universities and colleges have chosen to standardize on iSCSI, a protocol that links data-storage devices over an Internet Protocol network, facilitating data transfers over intranets and managing storage over long distances. The protocol tends to be much easier to implement and manage than fibre channel–based storage area networks and can be substantially less expensive because little additional hardware — such as host bus adapters, network interface cards or a separate IP network for storage — is required. In university environments, iSCSI maybe a better fit, because it’s similar in many ways to existing network infrastructures.
The technology you choose depends on the amount of data you are storing and how much performance you need, says Greg Schulz, founder of the StorageIO Group, a Stillwater, Minn., consultancy.
“If you’re supporting multiple petabytes of data and need high-end storage arrays, the cost of fibre channel would be relatively cheap when you factor in resiliency and performance,” he says. “But if you’re in a smaller environment where the storage is measured in terabytes, fibre channel and iSCSI will be a better choice.”
Over the next few years as SANs mature, with greater speeds, lower costs and more data sharing capabilities, even more connection methods will become available. Most exciting, Schulz says, is block-based access, such as Data Center Ethernet and Fibre Channel over Ethernet. Data Center Ethernet is a high-bandwidth, low-latency network for carrying Ethernet and storage traffic, while Fibre Channel over Ethernet will allow fibre channel to be run over the same wires as existing data networks.
