The best way to do that is remove switches — a whole tier of them – in conjunction with increased port speed.
Simply jacking up server-to-switch and switch-to-switch speeds to 10 Gigabits per second isn’t enough to optimize the performance of data center networking for mobile VMs. Reducing latency by transitioning a three-tier network architecture to two, and then to one, and removing Spanning Tree as an impediment are key ingredients, too.
Higher, non-blocking throughput from 10G Ethernet switches allows users to connect server racks and top-of-rack switches directly to the core network, obviating the need for an aggregation layer. Also, server virtualization is putting more application load on fewer servers due to the ability to decouple applications and operating systems from physical hardware.
More application load on less server hardware requires a higher-performance network. And a higher performance network includes replacing the Spanning Tree protocol in Ethernet with a new technique that enables use of multiple active links between switches. Such techniques, like the TRILL work in the IETF, are intended to overcome limitations of the Spanning Tree protocol in scale and topology re-convergence.
TRILL is intended to be a Layer 2 protocol with link state routing enhancements to enable shortest path multihop routing so users can build large-scale Ethernet and Fibre Channel-over-Ethernet data center networks. TRILL is designed to overcome the slow topology re-convergence times associated with Spanning Tree, which limits scale and is more susceptible to link failures.
Vendors are also proposing their own TRILL-like methods for flattening data center networks. Cisco System Inc.’s FabricPath is a “pre-standard superset” of TRILL that includes TRILL but extends its capabilities. Juniper claims its Virtual Chassis technology, which interconnects multiple Juniper switches into a single “switch” supporting hundreds of Gigabit Ethernet ports, accomplishes the same goals as TRILL — chiefly, to collapse layers of switching in a data center network and facilitate more “east-west” communication between servers than “north-south.”
Brocade Communications System Inc.’s One architecture implements TRILL, while Avaya Inc.’s VENA implements the IEEE’s answer to TRILL, 802.1AQ Shortest Path Bridging. Alcatel-Lucent plans to add TRILL-like multichassis/virtual chassis linkage this year to its new OmniSwitch 10000.
The New York Stock Exchange is flattening its Juniper Networks Inc.-based 10G network to reduce latency and take out a layer of market feed traffic processing.
“It basically takes out one hop so … anywhere from a 10 to 20 microsecond improvement” in latency, says Andy Bach, senior vice president and global head of communications for NYSE Euronext, the owner of the New York Stock Exchange.
Multiple 10G links straight from the server rack switches to the core eliminate a tier of aggregation, Bach says. NYSE is also starting to put a couple releases of Juniper’s data center fabric (DCF) through its paces in its lab to further flatten the network and activate multiple links.
Bach would not comment when asked if what he referred to as DCF 1.6 and 2.0 were early releases of the Juniper Stratus architecture.
“It’s Virtual Chassis and how they interlink it,” Bach says of the multiple active link implementation. “That’s where we’re headed. As we get into 2012, we’ll be there. Spanning Tree is long gone.”
Another driver towards flat data center network is the movement towards a unified switching fabric that converges storage protocols onto Ethernet. This also requires a very low latency, lossless architecture that lends itself to a two-tier approach.
Storage traffic cannot tolerate the buffering and latency of extra switch hops through a three-tier architecture that includes a layer of aggregation switching, industry experts say. In addition to 10G, TRILL and Shortest Path Bridging, standards to facilitate this are the IEEE’s Data Centre Bridging/Converged Enhanced Ethernet work, and ANSI T11’s Fibre Channel-over-Ethernet. DCB defines an Ethernet fabric tuned for lossless transmission, and FCoE specifies LAN/SAN convergence.
Another important standard in network flattening is the IEEE’s work on “VEPA” and its associated techniques. VEPA – for Virtual Ethernet Port Aggregation – is designed to eliminate the large number of switching elements that need to be managed in a data center. VEPA offloads heavy network processing from hypervisors and virtual switches in servers and relies on the physical switches within the network to compute access control lists, virtual LANs, NAC, managing media access control address tables, aligning policies and filters to ports and/or VMs.
Extreme Networks is a proponent of VEPA in its data center switching architecture.
