Network vendors recently introduced a variety of 10G Ethernet wares aimed at customers seeking to bump up bandwidth in large data centres and campus backbones.
Products range from 10G Ethernet core switches to smaller “pizza box” 10G switches and network interface cards (NIC) that support copper.
Among the products expected to launch are:
* Enterasys Networks Inc.’s X-Series core 10G Ethernet switch, based on carrier routing technology acquired from now-defunct Tenor Networks.
* New modules from Foundry Networks Inc., adding eight-port 10G blades and CX4-based 10G copper links to the vendor’s core switch.
* An all-copper 10GBase-CX4 switch from Fujitsu Ltd., for linking large servers in a data centre.
* The industry’s first copper 10G Ethernet server NIC from Chelsio Communications Inc., based on the 10GBase-CX-4 standard.
One organization that has had 10G Ethernet since 2002 is CERN, the European Organization for Nuclear Research, where Enterasys’ latest X-Series box will be installed. The 16-slot switch, with 2.56Tbps of total switch capacity, can hold up to 64 10G or 768 Gigabit Ethernet ports in a chassis.
CERN now uses 10G switches from Foundry, Cisco Systems Inc. and Enterasys. It plans to put in an Enterasys X-Series box outfitted entirely with 10G Ethernet ports as an aggregation switch. The box will move data collected from supercomputing clusters to storage systems, according to Wolfgang von Ruden, head of IT at CERN.
“Our bandwidth needs right now are enormous and always growing,” von Ruden says. Many of the hundreds of experiments at CERN are capable of producing consistent datastreams at 6Gbps to 7Gbps over days or weeks, he says.
Another organization getting its feet wet with 10G is Florida A&M University in Tallahassee, which plans to install Foundry MG8 switches with 10G Ethernet uplinks. These boxes will connect to a regional fibre loop around the city. The effort will upgrade the school’s campus ring bandwidth from 155Mbps OC-3 SONET.
“We’re going to put these switches in and see what all the hype is about” with 10G Ethernet, says Wayne Dunwoody, director of network technology at Florida A&M. He says he hopes the 10G Ethernet loop will provide a backbone that is faster and as reliable as the current OC-3 ring. If it works out, Dunwoody says, the school plans to boost its multimedia applications, such as IP video broadcasts, while increasing its distance-learning offerings, which integrate video, document sharing and other collaboration applications for students taking courses from satellite schools in Tallahassee.
With its X-Series, Enterasys is bringing carrier technology into its enterprise product line. The box was built primarily by engineers formerly with Tenor, a multi-service carrier switch start-up that went bankrupt last year. Enterasys bought Tenor’s technology and hired its core engineers shortly after the firm’s closing. The X-Series is the first product to result from the Tenor purchase.
Enterasys CTO John Roese says the new box is more than a Tenor switch wearing an Enterasys sticker. The switch was also fitted with technology – ASICs and software – that lets it work with Enterasys’ LAN switch security strategy, Secure Networks. This strategy lets switches identify network attacks and shut down connections, or isolate traffic onto secure virtual LANs. The X-Series surpasses Enterasys’ previous high-capacity backbone N series switches introduced last year.
Roese says carrier-class attributes of the X-Series include separation of areas called the control plane and the data plane. The control plane handles switch management and configurations, while the data plane concerns a switch’s packet-moving and routing processes.
Some competitive products, such as those from Cisco and Foundry, operate control plane and data plane functions in the same software and hardware. In the X-Series, they are kept separate in the circuitry to allow the switch to continue operating if certain hardware within the device fails. This separation also makes it harder for a network attacker to bring down an X-Series switch by accessing the device’s configuration console through insecure telnet links or glitches in other management interfaces. Force10 Networks’ E series and Juniper’s routers also use this approach.
Increased density
Foundry also is pushing higher 10G port density with its eight-port 10G module for the MG8 backbone switch. The new blades allow the switch chassis to hold up to 64 10G ports. However, the blades are limited to 50Gbps backplane connections, which prevents all eight ports from running at 100 per cent capacity all at once.
Another Foundry 10G Ethernet offering also might help continue the recent price push-down by switch vendors, says Max Flisi, an analyst with IDC. Foundry is introducing a fixed-configuration switch with eight 10G ports priced at about US$4,500 per port with optics.
Costs for 10G gear have come down over the past year, with the average per-port price dropping from US$26,000 to about US$9,000 per port between June 2003 and 2004, IDC says. More recently, 10G products from Foundry, Extreme and HP cost about US$4,000 to US$7,000 per port.
10G “is a small market, but it’s going somewhere,” Flisi says. “With the history of Ethernet, it’s always been a matter of when you reach that price point that’s compelling enough to get everyone to widely embrace the technology. We’re getting there with (Gigabit Ethernet), but we’re not quite there yet with” 10G.
Research by Dell’Oro Group Inc. estimates that the average Gigabit Ethernet switch costs about US$280 per port, factoring in modular and fixed-configuration prices. With 10G still costing about three times as much as Gigabit (on a per-Gigabit basis), observers say we’re still a way off from hitting the pricing sweet spot.
“There hasn’t been much widespread interest in migrating to (10G) in the LAN core, other than in some very large (college) campuses and research,” says Lawrence Orans, principal analyst with Gartner. Besides pricing, the need for 10Gbps of bandwidth simply isn’t there in many enterprise backbones and data centres, he says. “A gigabit is still a lot of bandwidth for many companies, and it is still easy to trunk together two or four for larger connections.”
The development of 10G Ethernet over copper cabling could drive further adoption of 10G, analysts say. The IEEE 803.3ak standard for running 10G over InfiniBand-style CX4 cabling was ratified earlier this year, and so far Cisco, Hewlett-Packard Co., Foundry, Chelsio and Fujitsu are in the market.
10G Ethernet over copper is good “since you don’t need the expensive optics that are required with other (10G) technologies,” says Zeus Kerravala, an analyst with The Yankee Group. “Copper (10G) Ethernet is a good option to have for short data centre links” such as switch-to-switch, or for connecting servers to switches, he adds.
This is the aim of Chelsio and Fujitsu as they announce a co-developed 10G switch/server NIC product this week at the Supercomputing 2004 show in Pittsburgh. Chelsio, which introduced optical 10G Ethernet NICs earlier this year, is the first to offer a server NIC based on the CX4 standard. Fujitsu, which mainly manufactures 10G components used by switch vendors, is also the first to offer a fixed-configuration all-copper 10G box, with six 10GBase-CX4 ports. These NIC and switch products are targeted at large data centre deployments, or server-clustering configurations in research laboratories.
10GBase-CX4 uses a cumbersome InfiniBand cable, with a distance limit of about 50 feet, so its use is restricted to computer room links, where racks of CPUs or switches are close together.
“We’re definitely going to look at” 10GBase-CX4 switch ports and server NICs, says Richard Nelson, director of information processing at the University of Southern California’s Information Sciences Institute (USC-ISI), a computer research centre.
USC-ISI runs a 64-node cluster of Intel Corp. machines as part of a U.S. Department of Homeland Security-sponsored research project called Deter, which simulates how viruses attack network systems. Nelson says that application will need more bandwidth very soon, and copper 10G Ethernet seems an inexpensive way to connect machines, rather than expensive, proprietary server interconnect technologies such as Myrinet or InfiniBand connectors.
“Ethernet is more adaptable than those things,” Nelson says. “If I have to take apart that cluster some day, I can run my LAN on the (10GBase-CX4) equipment. You couldn’t do that with InfiniBand or Myrinet.”
