Ten seconds to midnight, December 31, 1999. Megacorp’s brightly lit computer centre hums with the sound of servers. The sprawling room is festooned with a few cheesy garlands left over from the company Christmas party and a couple of dozen limply floating balloons. Gathered round a desk in the middle of the room are a few unfortunate employees who have pulled skeleton crew duty. Henderson has smuggled in a bottle of bubbly, the contents of which have been more or less evenly distributed in eight raised glasses. Everyone’s eyes are glued to a computer on the desk, logged onto a millennium Web site showing a clock ticking out the last few strokes to the dreaded year 2000. Everyone prepares to celebrate – three . . . two . . . one. The entire North American power grid fails. The monitor dies. The room plunges into darkness. Silence . . .
. . . then a distant pounding – growing louder and louder until the sound is deafening. Godzilla stomps through the city, smashing buildings and crushing terrified citizens under his massive feet, while invading Martians pick off the survivors with lepton-powered melting rays, reducing them to steaming puddles of human sludge.
The first scenario is probably only slightly less likely than the second. The chances that massive power failures will cripple North America as the new century begins appear to be virtually nil. But that does not necessarily mean there will not be problems – problems on a more manageable scale, comparable to the effect of bad weather or a malfunction in a generating station, but enough to cause headaches for those who depend on electricity. Such problems are not certain – not even probable if you believe the official pronouncements – but they have not been ruled out.
Canada’s power utilities all say roughly the same thing about the Year 2000 problem: They cannot guarantee there will be no trouble, but they have checked and tested their systems carefully, they expect no failures, and if there are any problems they will almost certainly be minor.
Bill Imms, director of the Year 2000 project at Ontario Hydro, says the utility is essentially on target with its Year 2000 program. With almost 1,900 digital systems to test – and in some cases repair – Ontario Hydro had hoped to finish that phase of the project by the beginning of this year. In fact, the last 23 systems took until late February. All units have completed detailed business contingency plans, and these are now being implemented. The priority in 1999 is large-scale integration testing, Imms says, and that includes taking entire generating stations forward in time – figuratively of course – to 2001 by resetting clocks. Several stations have already gone through that process without incident. A well-publicized test involving the high-voltage transmission system that serves Toronto, conducted on a weekend in early March, was a success.
Nova Scotia Power has gone a step farther. A number of plants that have been set forward into 2000 have been left there, running as if the new millennium has already begun.
Reports from the front lines suggest electrical utilities aren’t heavily affected by the Y2K problem. “What we’ve found is that not much is either not compliant, or if it’s not compliant, it doesn’t affect the functioning of the device,” says Glenn Schneider, manager of public affairs at Manitoba Hydro in Winnipeg.
Imms says the nuts and bolts of the electrical system are largely analogue and mechanical, with relatively few digital components. The largest concentration of digital technology is in nuclear generating stations, he notes, and these are also the only facilities that probably could not be switched to manual operation if computerized control systems failed (which could in itself be the root of a problem).
The Canadian Electricity Association released a report in January saying the Year 2000 rollover will cause “only a minimal impact on electric system operations in North America” and “computer errors found thus far do not appear likely to threaten electricity supplies to customers.”
HOW SURE ARE THEY?
Peter de Jager, the man credited with bringing the Year 2000 problem to widespread attention with his 1993 article in Computerworld entitled “Doomsday 2000”, is frustrated by the hedging in the electrical industry’s public statements. Utilities and the bodies that represent them are carefully avoiding saying nothing will go wrong. When an industry association reports that “most entities do not report anything that would have created an open circuit,” de Jager observes, “as a student of English that means to me that someone did.”
However, de Jager says, while there may be problems there is unlikely to be widespread trouble. In fact, he says officials from some utility companies tell him privately they are more confident than they are letting on, but are playing it safe because of legal advice.
And Francis Bradley, a spokesman for the Canadian Electricity Association, says the utilities’ refusal to issue any guarantees reflects the fact that many things other than the Year 2000 problem can cause power outages. Nobody can guarantee there will not be power interruptions on January 1, 2000, he says, because nobody can guarantee there will not be power interruptions on any day.
Even if utilities have identified and corrected all their internal problems, though, concern about trouble could be a self-fulfilling prophecy. Greg Walker, manager of the Year 2000 project at Alberta Power, which serves much of rural Alberta, says his company is concerned about the effect on the power grid if internal Y2K problems – or a desire to be cautious – cause a number of major industrial customers to shut down factories on New Year’s Eve, dramatically reducing the demand on the grid.
“The thing about the electrical system is it doesn’t react well to surprises,” Walker says. Few people realize that if the supply of electricity significantly exceeds demand, the grid can become unstable. The obvious solution is to reduce supply by cutting generating capacity. However, Walker says, Alberta Power’s generating capacity comes in 400-megawatt blocks. Shutting down capacity may mean the system goes from oversupply to undersupply, causing brownouts. In hopes of avoiding a seesaw effect on the night of December 31, Alberta Power is working with large industrial customers to determine what their plans are and try to anticipate how the load on its generators may fluctuate.
Ontario Hydro has voiced a similar concern, and in a recent speech Imms urged large industrial customers to stick to business as usual next New Year’s Eve.
WHAT DOES IT MEAN FOR I.T.?
Many IS departments are showing little concern about the problem. “I can’t see how distributing power would be a major problem,” says Robert Allard, vice-president and chief information officer at Sun Life Assurance of Canada in Toronto. Allard adds that Sun Life, like many large IS shops, has not only uninterruptible power supplies (UPSs) but diesel generators to supply electricity to its data centre in case of a longer blackout. “If suppliers cannot supply us with diesel (fuel),” he adds, “that might be a tougher issue.”
Joe Boivin, executive director of the Global Millennium Foundation in Ottawa, says most large data centres in organizations that rely heavily on IS are similarly equipped and so should be able to weather any power problems the date changeover brings. Boivin does not dismiss the danger of electrical problems – “there will be failures,” he says, “hopefully they will be localized failures.” But he thinks well-run data centres should be equipped to handle the worst that seems likely to happen.
Not everyone is so sure there is nothing to worry about, though. “We are proceeding on the assumption that there will be failures regardless of the quality of the Year 2000 remediation program,” says Mike Smith, a principal at consulting firm Ernst & Young in Toronto whose specialty is business contingency planning. “Logically that has to apply, given how good we’ve been in the IT industry at developing and testing applications.” Smith points out that the best IT development organizations typically catch 98 per cent to 99 per cent of the bugs in new applications before they go into service. That implies one or two per cent of the problems are not caught, and that could be enough to cause headaches.
Kelly Cote, manager of the Year 2000 program at Noranda Inc. in Toronto, takes the utilities’ reassurances with a grain of salt. “The statements that the power companies have made certainly reassure us,” she says. “The more information the power companies release, the more comfortable we are.” However, Cote is planning for possible power interruptions on January 1, 2000, and in the following days. “Any responsible company I think would take caution when it comes to mitigating the risk for their critical systems.”
Cote says the effect on the company’s manufacturing operations is the greatest worry, but the impact on IS is also a concern. She will not discuss in detail what Noranda is doing to insulate itself against Y2K power glitches, except to say contingency plans are being put in place.
A REAL-LIFE DISASTER
To see an example of what might happen to a company confronted with a serious Y2K-induced power loss, look at QL Systems Inc., a Kingston, Ont.-based online database operator that was left without power for eight days in January 1998 because of a natural rather than a technological phenomenon – the ice storm that paralysed much of eastern Canada that month.
On the evening of Wednesday, January 7, shorting power lines and transformers lit up the sky around Kingston as lines gave way either under the weight of their own ice coatings or were taken down by falling branches. QL Systems lost power at about 10 p.m. when overhead wires leading to the building fell.
The company had enough UPS capacity to keep its computer systems running for about three hours, says Donna Ashton, director of QL’s computing centre. It did not have generators. Even if it had had them, Ashton points out, it is doubtful whether fuel trucks could have negotiated the streets that weekend to refuel them – downed branches and power lines made access to the downtown area where QL is located very difficult for several days after the storm hit.
“We’ve all gone through a lot of power failures in our lives,” Ashton says, “and to think that the power would be out more than a day or two at the most was something that we had never addressed before.” As it turned out, QL’s offices were without power for nearly six days.
By Friday morning, Ashton says, “it finally got to the point where we had to make a decision.” QL implemented its disaster recovery plan. Computer operations were moved to a hot site in the suburbs of Toronto, a process that took about 24 hours. The company used the hot site for about a week – although power was restored to its Kingston facilities on Tuesday, water lines had been drained to stop them freezing and “we had a lot of physical plant things to do” before moving operations back home, Ashton recalls.
That raises an important point about backup power for computer centres – keeping the computers running is not enough. As Smith points out, if there is no heat, or the fire alarms or bathrooms don’t work, employees can’t work in the building for any length of time.
NEED FOR A DISASTER RECOVERY PLAN
QL Systems’ disaster recovery plan endured a trial by ice and came through fairly well. Ashton says the company has made minor changes to the plan, and taken a few extra precautions such as replacing the overhead power lines into its building with underground conduits less likely to feel the brunt of bad weather. That means QL may be better prepared than some companies for a Year 2000 scenario in which weather plays a part.
In fact, the Year 2000 bug seems unlikely to lead to power interruptions as long as those that occurred during last year’s ice storm. Brief outages or brownouts in the early hours of the new year, caused by scattered small glitches, seem the most plausible worry. So if you have reasonable power protection – UPS backup at a minimum, generators if you really can’t afford for your computers to go down – and an up-to-date, tested disaster recovery plan, chances are you will be all right. The only problem, Smith points out, is that too many organizations lack proper disaster recovery plans or have not tested them properly.
He advises every organization to develop a plan if it does not already have one, and test it thoroughly. He also recommends that even if a short shutdown would not seriously affect your organization, you consider proper power backup. “You don’t want these devices going up and down like a yo-yo,” he warns, “because that starts to stress delicate circuitry, which has a lasting impact.”
So, the possibility of power problems around the turn of the millennium, however low the risk, is a kind of wake-up call to all IS departments to take a good hard look at their emergency preparedness. “It’s an added incentive to do what we should do anyway,” says Peter de Jager, adding that if the whole Year 2000 problem has any beneficial effect, that might be it.
SHOULD WE BE CONCERNED ABOUT NUCLEAR?
If Y2K preparations in the electrical industry are as far along as we are told, we probably have nothing to fear except, as the saying goes, fear itself. The problem is that fear itself could be a serious problem. Widespread nervousness about the effect of Year 2000 bugs on nuclear power plants could reach the point where plants might be shut down as a precaution, largely for political reasons.
On February 8, the Peach Bottom Nuclear Plant in Pennsylvania had problems with monitoring computers during Year 2000 testing. Nothing exploded, no radiation leaked from the plant, and officials maintained the system that malfunctioned had no bearing on public safety. Further, Francis Bradley, spokesman for the Canadian Electricity Association, says the only connection between the problem at Peach Bottom and the Year 2000 was that the failure happened during Year-2000 testing – the actual cause, he maintains, was human error.
But can we believe everything we are told about problems at nuclear plants? Certainly many people don’t. Peter de Jager thinks that may be the greatest cause for concern about the power supply. “My biggest fear,” he says, “is that we have to shut down power plants because we’re not 1,000 per cent sure….Do I think that’s a reasonable action? No, I don’t – but I think that they may be forced to do that by the hysteria.”
What impact would that have? In most of Canada, none at all, since the only Canadian nuclear power plants outside Ontario are one in Quebec and one in New Brunswick. Southern Ontario, which depends heavily on nuclear, could be a different story – and that area, of course, is home to many of the country’s largest data centres.
Bradley says the scenario is improbable. “I haven’t heard anybody in any position where they’d be able to make that kind of decision even suggest shutting down nuclear power,” he says. Bradley adds that Canada leads the United States in the Y2K readiness of its nuclear plants, largely because all the Canadian plants use very similar Candu reactors, so the utilities can share information very effectively.
So the story for now at least is that as far as nuclear is concerned, we have nothing to worry about. But who among us, come December 31, will not be nagged by that damned book/movie Fail Safe?
Grant Buckler is a freelance writer specializing in information technology. He is based in Kingston, Ont.