There were 88 major IT security incidents in the lead up to and during last summer’s Toronto Pan Am / Para Pan Am Games, according to a leader in the event’s security operations centre.
They included finding three instances of Zeus trojan activity, the discovery of two pieces of ransomware — which rendered one PC useless — and three stolen laptops, says cyber-security manager Enzo Sacco of Toronto’s Scalar Decisions, which provided managed security services for the Games told the SecTor 2015 conference on Tuesday.
However, there was no stolen data from event’s systems or interruption to the games. Arguably, that’s a medal-winning performance, considering there were about 750 alerts a day at the event’s peak.
“I wouldn’t say we got lucky,” Sacco said in an interview. “Some of it is a lot of potential security incidents got mitigated because of our multi-layers of security. But unlike other organizations, they have legacy software, legacy systems. We started fresh” — so no old Java or Adobe Reader.
In fact, he said, as pieces of technology were implemented they were immediately made part of the vulnerability management system.
“We already had a good level of security baked in from the beginning,” he says.
Chosen by official technology supplier Cisco Systems to be the security provider, Scalar designed and managed IT security for the 33 venues (including event data), Internet, two data centes, business applications and three WiFi networks (for the organizing committee, guests and the public).
Data traffic could range up to 1,200 GB a day.
“A lot of the (event) investigations were related to malicious code — people accessing websites they weren’t supposed to, clicking on things they’re not supposed to,” Sacco said.
“Yes, there were times when machines got compromised, but there were also added layers of security — be it even on the end point when access to certain directories were disabled. So malware could be observed coming in and being downloaded and being disallowed by the base operating system, or the anti-virus picked it up.”
Except once. Despite employee training one Games employee clicked on a link in an email and discovered a PC locked by ransomware. It couldn’t be disinfected.
Quang Tu, one of Scalar’s analysts assigned to the Games who looked into the incident after it happened, told the conference that for some reason neither the intrusion prevention system nor the anti-malware caught the ransomware.
“We weren’t sure why,” he said. “We had to go back and re-assess our security technology and add extra layers of email technology.”
(One lesson: People are still the most vulnerable part of an organization).
Preparing secure systems for an event like the Pan Am Games comes with its own challenges, Sacco said. These include delivering systems that are ready for a fixed debut (the opening ceremonies) as well as worrying about the potential for attack because it’s a high-profile international event.
It didn’t ease worries that around the time of the Games there was international economic forum held in the city, which in 2010 saw police cars burned during the G20 Summit.
To keep on top of all outside threats there was a threat intelligence committee that included representatives from the federal computer incident response team, Cisco, province of Ontario, and CIBC (a Games sponsor) to understand the threat landscape.
“One of the things we strived for (in the security architecture) was keep it simple,” Sacco told the conference, focussing on the perimerter and end points. “We didn’t want to over-complex the situation given it was a short period of time.”
In fact, he said, 90 per cent of the security team’s work was the planning: the rest was the execution.
Among the lessons he passed on for those who might have to oversee similar temporary events is to make sure design requirements are baked in from the start, “not bolted on after the fact.”
Testing, table top games, more testing, and trying to anticipate every eventuality were also big parts of the planning.
