Trading Up

With a creaky network routing and switching infrastructure that couldn’t keep up with advancing applications, Lafayette College decided it needed a change: a cost-effective new network that would secure a variety of devices and people together on the same, centrally controlled system. Network virtualization was the key to a sleeker, more efficient infrastructure for the job.

The new network infrastructure allows Lafayette, in Easton, Pa., to set up individual virtual networks when they’re needed, assign the appropriate amount of security according to each user and reap the benefits of using a common network.

The new network started with a radical, built-from-scratch redesign of the legacy network infrastructure, using multiprotocol label switching (MPLS) virtual private networks (VPNs) that make it more flexible, scalable, secure and efficient than more traditional Layer 2 or Layer 3 solutions.

By virtualizing the network, MPLS VPNs allow sensitive information, such as credit card data and security camera transmissions, to share the physical network with regular traffic, without fear of compromise or damage. The technology also allows centrally controlled policy assignment, which strengthens security.

With so many options available in the networking marketplace, from admission control and monitoring to routing and switching solutions, we were able to consider many design scenarios. Given our requirements and our interest in centralized management of network policies, MPLS VPNs proved to be the best solution.

Out With the Old

Lafayette’s old network infrastructure, installed in 2000, was at the end of its five- to seven-year lifecycle. Aging hardware and an inability to implement many important emerging network standards over the past six years compromised the network’s utility. Advancing network speeds, coupled with advanced monitoring, security, admission control, wireless, cluster computing and videoconferencing technologies, further strained our largely stagnant infrastructure.

Fortunately, the college gave us the opportunity to design a new network from the ground up. Our conversations began with blank whiteboards and a long list of questions about network design:

• How can we design a system with centralized policy management?
• How can we keep our Layer 2 broadcast domains pushed as far as possible to the edge of the network?
• What would we do about admission control?
• How would we reconcile the need for mobility with the need for security?
• How would we address the increasing demand for bandwidth?
• How would we manage the growing number of life-safety and other nonuser devices attached to our network?
• How would we make the network highly redundant and resilient?

To answer these questions, we reviewed traditional design methodologies and matched each to our criteria to see which approach seemed to be the best fit.

Traditional Layer Approach Falls Short

Before reviewing our MPLS-based design, it’s important to understand the strengths and weaknesses of more traditional Layer 2 and Layer 3 design.

Traditional Layer 2 networks can be easily segmented based on user or device role by placing each host into an appropriate VLAN. Usually, these VLANs then span the campus, creating massive broadcast domains. However, large broadcast domains cover multiple switches in multiple locations and make problems difficult to troubleshoot. The increased number of hosts, the increased amount of bandwidth consumed by broadcasts and spanning tree issues lead to network instability. With a small number of VLANs, though, access control lists (ACLs) can be applied to the routers in the network, limiting access among the VLANs.

Traditional Layer 3 networks fall short when it comes to segmenting user types but can scale to large networks. Typically, one VLAN is allocated per location and contains a variety of hosts. Although broadcast domains are kept small, it’s very difficult to manage ACLs on routers, which complicates segmentation based on user role or device. Lafayette’s pre-existing network was designed this way.

Scalability Meets Segmentation

We wanted to combine the scalability of Layer 3 with the segmentation inherent in Layer 2. Layer 3 virtualization met the challenge. Layer 3 virtualization segments host at Layer 2, then build a separate routing table for each role-based virtual network. Policy can then be applied centrally at the points where all the virtual networks meet.

There are several ways to virtualize at Layer 3, including generic routing encapsulation (GRE) tunneling, virtual routing and forwarding (VRF) -lite, and MPLS. Each method distinguishes itself by its scalability, but all are based upon a VRF instance, which is a set of virtual routing tables and protocols that appear on a router.

Lafayette deployed MPLS, a technology many network carriers use to switch packets across their networks. Using MPLS, label-switched paths are built dynamically between routers. By leveraging these paths, many VRFs can be provisioned across many routers. Once properly configured, adding VRFs becomes relatively simple.

Our design begins with multiple role-based VLANs on all edge switches in the network. Using Cisco’s standard out-of-band network access control (NAC) solution, users and devices are placed into these VLANs. Each VLAN then terminates on a switch virtual interface (SVI) on the distribution routers and each SVI is assigned to the appropriate VRF. Using MPLS, the VRFs are then connected, making several virtual networks. The virtual networks each have a default route to a firewall and tie the firewalls together at a meeting point. Policy is centrally applied and managed at each firewall so communication between networks is either allowed or restricted.

For our implementation, we designed several device and user-based VRFs (see “Lafayette College Data Network by the Numbers,” at left). Our device VRFs typically didn’t need to communicate with hosts outside of the Lafayette network. Therefore, our device VRFs use private Internet Protocol numbering, freeing up registry-assigned space for users. With the large amount of private space available, we carved it up in a self- documenting way (see “Virtual Network Numbering 101,” Page 40).

After completing the design, we developed three phases for implementation. The first phase, completed in July 2007, entailed configuring and installing the new core and distribution routers as well as a redesign of our Internet edge topologies. At the end of this phase, we interconnected our existing infrastructure to the new distribution routers.

The second phase, completed in November 2007, involved converting all copper patches to new switches connected directly to the new distribution infrastructure. We retired all of our old routing and switching gear at the conclusion of phase two.

The final phase involves renumbering the IP network and implementing network admission control to put users and devices into their corresponding VRFs. We are set to complete this phase this spring.

Gained Efficiences

This virtualized network architecture allows us to provision secure, mobile, robust network access for both user roles and a variety of network-based devices. For example, we can now implement an IP security camera that’s completely isolated from user traffic, yet rides on the same physical infrastructure as our users. Benefits also extend to the realm of credit card transactions: We’ve built a VRF for our point-of-sale systems on campus, securing those transactions from users. In addition, we have other VRFs for devices such as classroom audiovisual controllers, heating and air conditioning systems and door-access controllers, among others.

We’ve also constructed a VRF for our electrical and computer engineering department, because it wanted to connect oscilloscopes and other data-acquisition devices to the network, but also wanted complete isolation for these devices. Now the lab manager can administer the devices over the network knowing they’re secure from the other network traffic in their own VRF.

Going forward, we will continue to build VRFs for other network-based technologies that would benefit from being connected to a network but need to be isolated from other traffic. Virtualization allows us to have any number of logically separate networks riding on the same physical infrastructure.

Provisioning quality of service (QoS) on our virtualized infrastructure is much simpler and more scalable, as all policies are centralized at the core. We can now tune network performance based on VRF, whether implementing a Voice over IP VRF, videoconferencing or telepresence VRF, or any other application that might benefit from its own virtualized network.

Network management is simpler with intelligence centralization and policy enforcement. Combined with a central policy-enforcement mechanism (the firewall-services module), network-monitoring systems enable staff to gather intelligence about trends in network use, bottlenecks and growth. This information is invaluable in sustaining the investments made in the new network by helping to guide future modifications to policies and infrastructure.

The research, teaching and business objectives of the college community depend more and more on technology, whether for conducting business, data-acquisition experiments, artistic expression, film production, statistical analysis or the study of a foreign language. The virtualized MPLS-based network infrastructure has become a necessary condition for, and enabler of, student learning, faculty scholarship and administrative effectiveness.