When Karl Sigler looked at Trustwave’s recent research, released today, into how bad IT teams are at patching software, he wasn’t surprised.
As many have reported, there are lots of internet-facing systems with applications that are weeks and sometimes months behind in installing critical security updates issued by vendors, leaving their organizations open to compromise.
What angered him, though, was the number of systems that the research found with end-of-life applications still online when Trustwave SpiderLab scanned the internet with the Shodan search engine for looking for exposed services.
”That’s kind of irresponsible,” Sigler, Trustwave’s senior security research manager, said in an interview. “When you have a system that is publicly exposed that is years out of date, that’s almost beyond irresponsible.”
For example, just under 40 per cent of hosts running the Apache Tomcat web server were running versions 8.0x or 7.0.x. Adobe now supports only versions 8.5 and higher.
Why some IT departments are running unsupported applications isn’t clear. They might have been forgotten but left online, Sigler said, or IT thought they were decommissioned but for some reason they came back online when power was restored – all of which are related to poor asset management.
Whatever the reason, Sigler said, IT managers have to take the issue of unsupported products more seriously.
He was commenting on the release this morning of Trustwave SpiderLab’s Telemetry Report on patching high-profile vulnerabilities discovered and fixed this year alone. (Registration for the report is required)
Over half of servers scanned had a weak security posture even weeks and months after a security update was released, the report says.
For example:
–six months after Microsoft’s advisory in March that the threat group it calls Halfnium was targeting Exchange servers, 13,000 publicly accessible servers were still vulnerable to the ProxyLogon attack;
–three months after VMware reported multiple critical vulnerabilities in its vCenter management console almost 49 per cent of hosts hadn’t been patched;
–three months after Pulse Connect released patches to close highly critical vulnerabilities in its Pulse Secure Gateway there were still over 6,300 vulnerable devices detected by Shodan;
–four months after F5 Networks issued a security advisory about a vulnerability to its BIG-IP products, 233 hosts (just under seven per cent of devices discovered) were still vulnerable to attack.
Patch management critical
There is no accepted standard for how fast patches or mitigations should be applied. Ideally, Sigler said, a patch should be installed within 12 hours of release. A critical patch that isn’t applied within a week will be a problem, he added.
Patches can break rather than fix things, he acknowledged, so they need to be tested. And there are other complications: In a distributed organization the IT team may have to work with teams in different geographies, time zones or languages.
Which is why, Sigler said CISOs/CIOs have to ensure there is a formal process for installing patches. Otherwise “the time to apply a patch will be increased by months.”
However, he said that “it’s almost a rarity these days to see organizations that have a formal process for updates.”
If your organization doesn’t have such a patch management policy – which covers testing, deploying, and documenting security patches – start with reading a paper called “Critical Cybersecurity Hygiene” by the U.S. National Cybersecurity Center of Excellence. A patching policy starts with asset management – IT can’t patch what it doesn’t know it has – followed by assigning risk levels to applications. The most critical patches should be installed on the most important assets. Many IT vendors have free white papers outlining patching best practices.
What’s vital, Sigler said, is that patch management is a documented process. That protects IT if, and when, knowledgeable staff leave.
Failure to be vigorous can be catastrophic. In 2017 the failure of Equifax to patch a vulnerability in a server using the Apache Struts web framework led to the theft of personal and financial data on approximately 148 million individuals — including 19,000 Canadians. Equifax had been alerted there was a patch available. However, the vulnerability management software that was supposed to identify any server needing a patch missed the lone one running Struts. In addition, managers who knew of the vulnerability failed to tell lower-level staff. Each assumed the other had told the patching team.
