May 19 2009

Beefing Up The Backbone with Faster Ethernet Connections

10 gigabit-per-second Ethernet is now out of the research center and revving up production networks.

$2,000: Average cost per port for 10 gigabit-per-second Ethernet in the fourth quarter of 2008.

Source: Dell’Oro Group

While they’ve been deployed in research labs for many years, 10 gigabit-per-second Ethernet switches are proliferating in university data centers as colleges require greater bandwidth to carry business applications.

At Indiana University, server virtualization fueled deployment of 10Gbps Ethernet.

“We have well over 600 virtualized servers today, so that aggregates your traffic,” says Matt Davy, chief network architect in Bloomington. “We’re seeing a lot of demand for 10Gbps connections to servers to support that infrastructure.”

Davy is in the midst of upgrading the network infrastructure with Hewlett-Packard ProCurve 5400-series 10Gbps switches and plans to upgrade most of Indiana University’s buildings with 10Gbps connections in the next few years. The Indianapolis campus that he also oversees is replacing analog cable TV with IPTV, which will drive bandwidth needs to the buildings.

Alan Weckel, director of Ethernet-switch market research for the Dell’Oro Group, says 10Gbps switches are becoming the standard for large campus backbones.

“If you look around at engineering, science and medical departments, there are a lot of high-capacity files being transmitted,” Weckel says. He forecasts overall deployment of 10Gbps Ethernet switches offered by manufacturers such as Brocade, Cisco Systems, Enterasys, Extreme Networks, Fujitsu, Hewlett-Packard and Juniper Networks to hit 1.9 million ports in 2009.

Extreme Makeover

The University of Nebraska’s Peter Kiewit Institute runs a high-profile IT and engineering program in Omaha that has a unique business model in which researchers partner with companies. So it’s fitting that the organization future-proofed its network three years ago by rolling out 10Gbps Ethernet switches from Foundry Networks (now part of Brocade).

“We were severely limited in what we could do in terms of bandwidth capabilities,” says John Callahan, former director of technological infrastructure for the institute in Omaha. “With our new network, we’re ready for almost any challenge.”

Callahan says deploying the BigIron RX and FastIron SuperX gear has opened up opportunities that the institution previously had to decline because the network was unable to accommodate some high-end requirements.

Category 6a twisted-pair wiring can support 10Gbps at distances of up to 100 meters.

Source: Telecommunications Industry Association

One benefit of the boxes is that they deliver built-in redundancy. Callahan adds that 10Gbps speeds also let him comply with security policies calling for traffic filtering without losing any packets or experiencing latency.

Gardner-Webb University is another school that’s jumped to bigger bandwidth. The Boiling Springs, N.C., school migrated from asynchronous transfer mode about four or five years ago, deploying a 10Gbps Ethernet backbone and 1Gbps to the desktop. “We wanted to be ahead of cutting edge,” says Eric Brewton, the school’s network services manager.

The university core comprises four Extreme Networks BlackDiamond 8800 switches, with a fifth for Internet access. The BlackDiamond switches deliver ample bandwidth and are highly reliable. “I haven’t had any network outages since installing the switches,” he says. “When I had one backplane go out, another took over and I was still up and running.”

Brewton touts the ease of imple-menting the network, which he oversees with the help of one network engineer. “This was probably the easiest network I had to install, and the easiest to manage,” he says.

The Peter Kiewit Institute spent about three days replacing every piece of network hardware in its building. Thanks to a rolling blackout, the upgrade required almost no downtime. “The longest someone was down was about 30 minutes,” Callahan says. He attributes the rollout’s success in large part to prior planning with engineers. The Foundry Networks switch interface is easy for the institute’s IT group to work with, which is important because the institute taps students to work alongside its IT pros. “We think it’s a great learning experience,” Callahan says.

Look Ahead

At Ferrum College, last summer’s upgrade to a segmented network was a necessity for performance reasons. If one portion went down, the entire network followed.

When CIO Christine Stinson evaluated equipment, she chose Juniper Networks’ EX 4200 and EX 3200 switches. “The Juniper switches gave us the ability to expand into 10Gbps very, very inexpensively,” she says.

Though the Ferrum, Va., college isn’t using 10Gbps yet, the boxes can be upgraded with new cards to fully realize the college’s investment.

The Juniper switching products also have green appeal. The devices use less energy and support 15.4 watts of Power over Ethernet on all 48 ports, versus only 24 ports in many competing products.

Stinson expects 10Gbps Ethernet to be important to the college. She anticipates a surge in the use of video conferencing for administrative functions and student recruitment. The school is seeing increased demand for streaming video in the classroom. And finally, she hopes to complete a Voice over Internet Protocol rollout in 2010.

“Now that we’re segmented, everyone’s access speed is much faster than it used to be,” she says. “We had so much downtime on our network that this just had to be done.”

Fast and Faster

Standards bodies and equipment manufacturers are hammering out 40-gigabit-per-second and 100Gbps Ethernet for server interconnects and backbones.

Alan Weckel, director of Ethernet-switch market research for the Dell’Oro Group says component manufacturers are hard at work developing 40Gbps. He predicts the technology will begin ramping up in 2011. As for 100Gbps, don’t count on seeing it until 2012 or 2013.

“Those will be really popular in research environments where there’s a lot of link aggregation and core oversubscription,” Weckel says. “When these uplinks begin shipping, it will help the performance of the network and help them collapse the network.”