So you think you’ve got a tough network build? Jeff Seifert recently oversaw the installation of 16,000 wired ports over two weeks.
“In the last week there’s been over 4,000 changes to the network in terms of port VLANs changing and adding printers,” he adds.
And his team had to hit an inflexible deadline.
That’s because Seifert is the technology lead for Cisco Systems at the Toronto Pan Am/Parapan Am Games, which officially opens tonight although preliminary events have been going on since Tuesday.
Cisco is the Games’ technology and communications supplier, and along with its partners — Toronto integrator Scalar Decisions for IT security, Lanscope for network visibility, Allstream for managed IP communications, AmpThink for stadium Wi-Fi — is responsible for the IT infrastructure that supports operations across 57 southern Ontario sites for 250,000 media and spectators, and 10,000 athletes and officials from 41 countries.
The partners built two data centres (one for backup) which have been up for a year, a technology operations centre; installed over 650 LAN switches and routers, 2,000 Wi-Fi access points, 1,571 VoIP phones and 760 printers; supplied 2,500 laptops for administrators; and laid 100,00 meters of cable and 10,000 meters of new dark fiber for a 10 Gb network.
“Flawless execution is what it’s about,” Seifert, who is also Cisco Canada’s CTO, said in an interview Thursday. It helped that Cisco’s staff of 100 had the assistance of 350 students in its networking academy program.
Unlike the Olympics, where the network has to be in place months before the event starts, Pan Am technicians didn’t have access to some venues until recently because some facilities are rented.
For example, Seifert’s team couldn’t get into the Rogers Centre, where the opening ceremonies will be held, until July 2 — and only had six free days before rehearsals started for the opening ceremonies.
And in the middle of the games some of the cabling there will have to be unhooked because the stadium’s main tenant, baseball’s Blue Jays, will be back in town for a three-day stint, and then reconnected.
Pan Am Games aren’t the size of the Summer Olympics (next year’s Olympics in Rio de Janeiro will need 70,000 wired switch ports, plus 10,000 APs), which meant Cisco [Nasdaq: CSCO] didn’t have the budget it had when it was the communications supplier for the London 2012 Summer Olympics.
So necessity became the mother of innovation.
“In the Olympics you build two separate networks: there’s the Games (timing and management) network and the administration network, and you keep them isolated,” Seifert says. “We took the approach for the first time to put them together to put both together for one network, and use security and MPLS (multiprotocol label switching) and other technologies to keep it secure.”
(The third network is the Wi-Fi network used by the media and the public).
Building a single network will likely be followed in the future at the Olympics, he said. Unified communications is the trend, he pointed out, and “the Games are getting so expensive it makes a lot of sense.”
Network security is, understandably, a sensitive topic, with Games partners reluctant to give too much away. But Seifert did say that for Toronto the cyber threat team is for the first time analyzing Netflow traffic. The tool is Lancope Inc.’s StealthWatch network anomaly behavior detection platform, used for looking at both possible inside and outside threats.
Cisco chose Lancope because it has offered a joint solution running on Cisco UCS servers for customers for some time, Seifert said, in part because the software can process massive amounts of data. In an interview Gavin Reid, Lancope’s vice-president of threat intelligence. A console lets administrators set up real-time alerts for certain behaviour as well as regularly-timed reports.
As for the data centre, “from a technology perspective it is all known pieces we have worked with before,” said Roger Singh, CTO and co-founder of Scalar Decisions, which won the right to supply the IT security and data centre integration in a competitive bid.
There are several layers of firewalls, plus endpoint and network protection aimed at preventing those with Games provided devices from clicking on malicious links or downloading unsafe attachments. There is also protection against infiltration through partners attached to the network.
While the media bring their own devices, there is no attempt to force them to harden their equipment. Strict access control is one level of protection, and the wireless network the press uses has no access to the Games network.
The biggest challenges for Scalar, Singh added, were the fixed deadlines and integration.
