Get Smart: Network Refresh 101 from Cleveland's Tri-C

With more than 10 sites, ­serving tens of ­thousands of students and nearly 6,000 employees, Cleveland's Cuyahoga Community College (Tri-C) is evolving to host a large and ­diverse ­communications network.

Because Tri-C's network dates back more than 20 years, our Network Services team is at a crossroads, maintaining a massive, stable ­infrastructure, while upgrading and quickly ­replacing pieces to meet an incredible pace of technology adoption.

In the past, most network upgrades were done by forklift: Over the course of only a few months, the entire network would be replaced with new equipment and configured to handle growth in all areas. But as the network grew faster (nearly doubling in size over the past 12 years), we reached a tipping point with the end of support for the Cisco Catalyst 6509 switch, which constituted our entire core and some access layer functions. The question became, how could we upgrade the core and access layers, with less time per person, less money, less downtime, while delivering the most significant upgrade ever?

Discovery

Visualization is very important when working with large networks. We used a Visio diagram of our entire network as our starting point. From there, we drilled down to the finer details and reviewed information across many monitoring systems.

To build a bill of materials, check existing inventory for ­accuracy — not only devices and their modules, but also port usage counts. If you can track growth back multiple years, keep all records in one place. Look at link utilizations. IT always wants to fully utilize what we buy, but sometimes we buy for the spikes.

We decided to prioritize greater uplink speed as opposed to redirecting time and money toward network traffic classification and prioritization. If you can aggregate some links instead of managing quality of ­service in detail, it also might make more sense for your environment.

Who on campus requested more ports, faster wireless or coverage in the back of a conference room? You kept all those requests, right? At the very least, see who remembers specific areas for growth. Not every request can be addressed. Assign weighted values to categories in addition to the basic usage numbers to determine the best investments. Perhaps an area with frequent and regular use should have its own wireless access point, as ­opposed to an event that uses the corner of a conference room twice a year.

It is easy to specify a warranty or support contract for everything. Review any changes in the support options, and do some basic calculations on the simplest devices for spare parts. Could an older model serve in a pinch?

Human safety also should be a major factor in any upgrade. The longer you keep equipment, the more unstable it becomes. Most large networks carry data for fire, radio, surveillance or other critical services. When drafting the final plan, highlight this dependency.

It is probable that an upgrade will involve too much guesswork and, likely, a misallocation of funds without the critical pieces of information spelled out here.

The importance of measurement in documentation also cannot be overstated. Solid numbers and powerful visualizations are important to both decision-making and solution proposals.

Segment the Upgrade

When it came down to which parts of the network would be upgraded first, decisions were made after long deliberation. We calculated some bad-but-not-worst-case scenarios for hardware and software issues, and reviewed with upper management. Replacing out-of-pocket a pair of Cisco 6509 switches at a campus core during a flood or trusting a new support provider to our entire core network simply involved too much risk for us.

Major points used to segment upgrade steps included identifying the greatest need for students and ­employees based on request; measured usage and predicted growth; identifying the greatest risks, based on vendor information and financial ­calculations; and researching our purchasing and leasing options in the upcoming budget year. We considered not only product warranty and bundle changes, but also new manufacturers and their interoperability between existing and other upgrades. Of course, small form-factor pluggables and ports, cables, code versions, routing protocols and power all must be reviewed as well.

Making It Work

Our final plan was segmented over four main upgrades:

1. Computer labs (Fall 2012)
   Replaced Cisco 3550 with 3560-X (upgrade to 1 gigabit per second and Power over Ethernet).
2. Core and wireless (Fall 2013)
   Replaced Cisco 6509; ­upgraded Cisco Wireless Services Module/Wireless LAN Control to Cisco Flex; added or replaced older Cisco access points with 3500 and 3600 models (and increased by more than 100 in strategic areas).
3. Data center switching and campus link replacements (Spring 2014)
   Replace Cisco 6513 with 6807 and 3850, and replace Cisco MDS 9134 Fibre Channel switches with 9148. This spring, we also plan to move all wiring above rack, replace all ­intercampus links with upgradeable links and dedicate a 10Gbps link for data-center-to-data-center traffic.
4. Cisco primary access layer (proposed Fall 2014)
   Replace Cisco 4507 switches with 4507R+E and 3850.

This phased approach hit our most noticeable areas first, ­balanced with risk mitigation and user feedback, before spreading out ­remaining upgrade needs in conjunction with our financing options.

The number of services dependent on networks has grown so fast, and in most organizations are real-time, human-safety critical. Minimizing downtime is a must; however, not everything has to be done at 2 a.m.

Looking Back

If you just finished an upgrade, pat yourself on the back, then turn right around and keep talking about it (and get started on the next). The same systems that gave you the data to perform the upgrade can help you sing its praises and bridge the increasingly important communication gap between IT and network users.