Computerworld (U.S.)
Technology managers trying to justify and prioritize IT security spending are searching for some way to quantify the risk management benefits.
But a lack of standard processes and the wide variability of factors that affect risk are making it hard for companies to collect such metrics, users said earlier this month at a conference in Washington, D.C., organized by Gartner Inc.
“There is an increasing focus on measuring security effectiveness,” said Carl Cammarata, chief information security officer at automobile association AAA Michigan in Dearborn. Companies are realizing that “you can’t manage what you can’t measure.”
Driving the trend is the fact that security budgets have been rising by 20 per cent annually over the past couple of years, said Richard Hunter, an analyst at Stamford, Conn.-based Gartner.
“These have been pure costs, and CIOs and CEOs are asking what they are getting from all that (spending),” Hunter said. “If the response is, ‘You are getting better security,’ the next question is, ‘How do you know?’ “
As a result, security administrators are under growing pressure to find quantitative measures to demonstrate the efficacy of their security strategies.
“You need to have a baseline to measure against. If you don’t have any measurements, you don’t know where you are,” said Gregory Waters, a senior information assurance engineer at TWM Associates Inc., an IT auditing firm in Fairfax, Va.
The numbers can come from a variety of sources. For example, said Gartner, a company could collect metrics on the number of attacks it faced during a specific period, the type of attacks, the percentage of attacks that were successful, the time that elapsed between the onset of an attack and when it was first detected, and the time it took to launch countermeasures.
The metrics could also relate to a company’s overall risk profile based on an assessment of the vulnerabilities and threats faced by an organization and the countermeasures in place to deal with them.
Meaningful Metrics
Some vendors, such as Foundstone Inc. in Mission Viejo, Calif., and TruSecure Corp. in Herndon, Va., offer tools they say will help companies numerically score their risk on a sliding scale based on such assessments.
Used properly, such metrics can help security administrators give business managers a better snapshot of a company’s risk profile, Cammarata said. At AAA, merely using statistics and benchmarks from organizations such as the SANS Institute in Bethesda, Md., and the Computer Security Institute in San Francisco no longer cut it, Cammarata said. “My managers want to know what these statistics mean to my organization specifically,” he said.
Consequently, AAA is planning to gather internal metrics to build a one-page “dashboard” that will give managers a better, more relevant picture, he said.
Northrop Grumman Mission Systems in Reston, Va., is pursuing a similar dashboard approach, said CIO Diane Murray. “It will give us a high-level management view of how well we are doing” on the security front, she said.
Such information can also be useful to auditors for evaluating a company’s compliance with regulatory requirements. But gathering such metrics and using them in a meaningful way can be hard, especially when dealing with an issue such as risk, said Bill Spernow, chief information security officer at the Georgia Student Finance Commission in Tucker.
“The raw statistics that we need to create a measurable foundation do not exist,” he said. Moreover, numbers may not always tell the full story, because there are too many variables and dependencies involved in measuring risk, Spernow said. At best, they are “trend indicators” that could create a “false sense of security” if relied upon solely, he added.
Standards such as ISO 17779, which covers IT governance and data security, can provide a good basis for understanding what’s needed to build effective IT security, he said.
SIDEBAR:
Where the risks are
Earlier this year, Gartner Inc. ranked the top security issues facing companies in 2003. They are:
– Efforts to secure Web services and WLAN implementations
– Identity management and provisioning
– Intrusion prevention and event correlation – Efforts to secure “holes” instant messaging can open in networks.
– Preparations to prevent or secure networks against viruses.
