By Richard Bray
While Ontario was struggling to get back to normal after August’s massive power blackout, the Blaster virus was making things worse, worming its way into critical areas of emergency measures, hydro administration, environment and health care computer networks. As Dr. James Young, the province’s public safety commissioner, said, “It’s making our job more difficult.”
But it could have been prevented. A patch to prevent Blaster had been available for weeks.
Ontario’s problems in this area are directly related to two flourishing and related businesses in the IT security field. One is composed of groups and individuals who release malicious code based on published vulnerabilities. These people succeed if their worms, viruses and other wildlife can cripple networks before the necessary patches are developed and applied.
Complementing these enterprises are vendors offering “patch management” solutions. They succeed in this multi-billion dollar-market when their clients’ systems resist attack.
In the middle stand IT administrators – very much including many of the readers of CIO Governments’ Review – attempting to support internal and external users with reliable, uninterrupted and secure service. For these people, life is a balancing act among budgets, personnel and resources. They can purchase patch management solutions, but their users must then do without something else. As well, because the people who seek to destroy systems are usually ahead of the people protecting them, patch management is only as good as the timely availability of patches.
A good deal of time can pass between the first detection of a new vulnerability and the publication of a patch. The hardware or software vendor must reproduce the problem: to make sure it is not a hoax; to determine the cause of the problem; to find out which products and versions it affects; and, because most vendors are working on more than one major vulnerability at any given time, to assign it a priority. Only then does the vendor begin working on a solution.
One major network vendor reports that it can take from 10 hours to three weeks just to reproduce a problem. Given the complexity of today’s networks and applications, and their degree of interconnectivity, it can take much longer than that to write new code or reconfigure hardware that performs properly in most – let alone all – situations.
Meanwhile, if malicious code writers around the world have heard about the vulnerability, they can begin working overtime on a way to exploit it. Until quite recently, most people would never dream of publicizing any new vulnerabilities they discovered. Instead, they would keep them secret, dutifully report them to the relevant vendors, then wait for a response. Sometimes the wait would stretch into months and sometimes the response would never come. In the absence of vendor action, indignant companies and individuals began to “go public” with vulnerabilities. One result has been more highly destructive virus outbreaks.
Even when vulnerabilities are reported, and vendors do issue fixes, users often remain ignorant of their availability or are reluctant to install them. In some cases, because previous patches like January’s notorious Windows NT “fix” crashed their systems, administrators believe the cure is worse than the disease. In other cases, “patch fatigue” has set in, as IT departments incessantly pull down their networks for yet another service pack. System administrators can spend up to a quarter of each working day installing patches. Carnegie Mellon University reports that, in the vast majority of security incidents, IT administrators knew their software was at risk because an available patch had not been applied.
Microsoft, for its part, is taking a hard look at automating updates. New software would ship with the ability to download patches over the Internet and install them. The XP operating system already has this feature, which users are free to switch off.
Automated updates make sense for individual users or small networks, but in more complex systems, they have the potential to cause even more problems. System administrators have learned to test patches as thoroughly as possible before rolling them out, because most networks are unique, using hardware and software that has been installed, modified and configured to perform in a certain way. Unless the updates were highly customized, they would probably provide partial coverage, at best.
If there is a definitive answer to the patching paradox, it has so far been elusive. Faced with an unending series of updates, every public sector IT manager must either pick and choose among them or else devote the resources to installing them all, with the knowledge that even a comprehensive and costly program may not be enough.
Richard Bray ( writer@canada.com) is an Ottawa-based writere specializing in high technology issues.
