When Martin Bally became CISO of TRW Automotive in 2013, the billion-dollar international manufacturing giant had narrowly escaped being the victim of malware that could have collected and sent data out of the corporation.
That led him to overhaul the incident response plan of the company (now part of the German-based ZF Group), which includes four Canadian plants. Peers checked the plan and it looked good.
Then a wide group of staff sat down at a table and ran through an attack exercise to test the plan — a game really, where the incident was outlined by pulling facts from decks of cards. And like a game of poker, this one had a few unexpected draws — including the sudden discovery of a second breach.
“It showed us there’s always a level of re-checking containment,” Bally recalled. “We felt we had our incident contained in the exercise, but the randomness (of the cards) showed there was another leakage.”
Among the lessons learned was the plan had to have a process to verify containment until the incident was closed.
Which was better than the unnamed organization that learned to its dismay during an exercise that staff couldn’t find the incident response plan.
So-called tabletop exercises — more formally known as incident response plan tests — are mocked by some. But Kevvie Fowler, partner in KPMG Canada’s risk consulting practice and national response leader, said they are absolutely worthwhile.
“More often than not organizations will have an incident response process they test regularly for events such as virus infections,” Fowler said. “Because they have (only) a few virus infections every month they assume they’re okay, their plan is tested. The issue comes in when an incident that doesn’t happen very often actually occurs and they realize their plan is incapable of managing, such as a data breach. So it’s critical that organizations test their plans with incidents they don’t see on a day to day basis, but are still relevant to the risk they face.”
In fact in some regulated industries IR plan tests are mandated.
Broadly speaking there are three levels of tests, Fowler said:
–A strictly paper-based test so the IR team can look for process changes since the last test, update contact information etc.;
–Tabletop. Stakeholders sit around a table and run through a scenario. Good for when a plan has been updated to see if it works);
–Full simulated test. Also done at a table, or in the war/disaster recovery room. The most resource-intensive, everyone touched is involved — including public relations, outside counsel and third party suppliers. Not only does it test what is done but how it’s done — how would the system be patched? Do it on this laptop. Who is to be called? Do you have their phone number? In effect it measures real-time response.
These exercises need a facilitator to run the test, filing in gaps in scenarios if needed and making the team move on if a problem can’t be solved in a reasonable amount of time (chalk it up to one lesson learned) rather than hold up the test. Depending on the test the facilitator also makes sure steps are carried out (if at a point the company lawyer would be called, someone should pick up the phone; if the lawyer is in the room the “call” is simulated, but not the conversation). There also should be someone dedicated to record what is being done so lessons can be learned. Fowler recommends doing a de-brief right after the test, when minds are still fresh and the facilitator can add his/her observations.
But a good test is more than “OK, pretend all of the laptops with Flash need to be updated, what would you do?” There needs to be some unpredictability or the staff won’t learn, although the scenario should always be relevant to the organization. Fowler points out that tests should
There are a number of ways to run a tabletop exercise, some of which are provided by regulatory and other agencies (see Resources below). Some consulting companies create randomness through board games or decks of cards, like the one Bally and TRW used from GreyCastle Security of Troy, N.Y.
As GreyCastle CEO Reg Harnish described it, participants chose one card from an Event deck (for example, the comptroller’s PC has been compromised and hard drive encrypted), and one from the Severity deck, which has instructions on how many cards to take from the Impact deck. Then cards are taken from the Controls deck (controls can help mitigate the Event, like a firewall), a Challenge deck (with unpredictability factors like the CISO is on vacation, or the Control you drew is temporarily disabled), and a Response deck.
The idea of all this is to build an unexpected event. Once the scenario is painted it’s time to pull out the IR plan and see if it works (and the team works together).
It’s also time to discover if the essential tools are within reach need for a real incident — at least one phone unconnected to the usual office lines, an independent Internet connection and a printer with paper and toner.
Harnish says tests shouldn’t go longer than four hours. He also suggests CISOs don’t dither over implementing the results: Two weeks after the test the plan should be updated to cover the gaps discovered. It’s also the time to introduce training to staff.
These tests will expose holes in the plan: People meeting each other for the first time, not knowing their roles, war rooms not available, conference bridges that don’t have enough lines, not knowing how to contact assigned people or their backups, even not knowing where data is held (in the cloud, on a branch server?).
Fowler emphasises the incident response plan has to deal with specific breach or incidence scenarios the organization is likely to see — a plan can’t be general to handle all types of events. Test each scenario at least once a year. (He also suggests
Finally, in a paper on incident response testing Mitre Corp, noted that without clear objectives, planners cannot design a meaningful exercise. “The objectives enable planners to clearly structure the scenarios within the exercise to determine if their organization possesses the capabilities necessary to operate successfully within a hostile cyber environment and defend against cyber threats.”
RESOURCES
Incident testing, SANS Institute
Cyber exercise playbook: Mitre Corp.
Final tip from Harnish: Run your incident response test at the same time as a penetration test.
