A security association soon to launch metrics for measuring infrastructure security said today’s IT managers are applying inconsistent approaches to measuring security improvements across their organization’s IT infrastructure.
Bert Miuccio, CEO with the Center for Internet Security (CIS), said the inconsistency lies not only within, but across enterprises as well. “Government and industry spend lots of time and money to improve cyber security, but often the focus is more on compliance with best practices rather than outcomes,” he said.
Hershey, Pennsylvania-based CIS, a non-profit organization promoting IT security among businesses, plans to release before year’s end, IT security metrics defined through collaboration among a group of security professionals from corporate, government and academic organizations.
The metrics include two outcome metrics: mean time between security incidents, and mean time to recover from security incidents. The remaining are process metrics: percentage of systems configured to approved standards, patched to policy, with anti-virus; the percentage of business applications that have undergone a risk assessment, and a penetration or vulnerability assessment; and, the percentage of application code that has undergone a security assessment, threat model analysis, or code review prior to production deployment.
The metric definitions will be available to the public as a community resource. But a value-add for CIS member organizations is the availability of a hosted software designed to help IT managers track and evaluate security performance over time by recording metric data and generating reports. The idea is that these reports will reveal a correlation between measured outcomes and the implementation of specific security practices.
For instance, if the mean time between security incidents is on the rise, an IT manager “can look at process outcome indicators and see what factors might be leading to that decreased performance so it’s a way of drilling down into the data,” said Miuccio.
And with that knowledge, he continued, “they can start to make adjustments to the processes, they can shift resources, they can emphasize one process over another, redesign a process, and can implement a best practice.”
The software also allows IT managers to compare their organization’s security infrastructure with those of anonymous businesses in a similar vertical.
Miuccio said although CIS is targeting companies of all sizes with these metrics, they will likely be of greater use to larger enterprises, which typically have more sophisticated security programs and larger security investments.
According to Brian O’Higgins, chief technology officer with Ottawa-based intrusion prevention technology vendor Third Brigade Inc. , CIS’ metrics appear to have the right focus that ultimately impacts the business because often “security tools don’t always measure things that are the most important to the business units.”
To that point, O’Higgins cites a recent study by the Rotman School of Management and Telus that found IT professionals were the least satisfied with system log management tools among others. While the log management reports provide a plethora of data like system logs and alarms, he said the tools necessary to measure these recorded outcomes are still in their infant stages and don’t particularly demonstrate relevancy to the business.
“So, going all out,” said O’Higgins, “and measuring everything doesn’t mean you’re necessarily going to improve.”
While measuring outcomes is a necessary next step after amassing process data, O’Higgins acknowledged that, as with any tool on the market, there are opportunities for misdiagnosis that could lead IT managers to believe components of their infrastructure are more secure than they actually are.
But as relevant as metrics are at any given moment, they can’t remain static either, said O’Higgins, and must change with the morphing security threat landscape.
