On Tuesday, the emergency 911 service was down for two and a half hours in the early morning across the province of Nova Scotia and in some areas of New Brunswick and Prince Edward Island. The disruption affected landlines and some cellphones.
Bell, which operates the affected 911 network, restored the service at 9.30 am AT, saying in a statement to CBC News that the cause of the outage was a 6 am software update to prepare for 10-digit dialing for New Brunswick, “which triggered an unexpected call processing failure to 911.”
Bell stressed that 911 outages are rare, and receive the full attention of their engineering teams, adding that the issue was an “isolated incident.”
But it is important to remember that the problem is not with the 911 systems, which are “essentially bullet-proof,” said Mark Tauschek, vice president, distinguished analyst and research fellow at Info-Tech Research Group, but rather with Bell’s public switched telephone network (PSTN). More importantly, Tauschek added, it was not likely a software issue, rather a configuration issue.
“We’ve started conflating software updates and configuration updates, which are not the same thing,” Tauschek explained. “Recall the recent 16-hours plus Rogers internet outage in July of 2022 – that was not a “software bug”; it was a configuration change to BGP (Border Gateway Protocol) routing that took the whole network down until they could rectify the issue and route internet traffic back through Rogers’ network.”
The same thing potentially happened with the configuration update to allow for 10-digit calling, whereby the call routing tables probably did not know how to handle a 3-digit call because it had not been routed to a new 10-digit local number, suggested Tauschek.
Bell updating call routing to 10-digit dialing in Eastern Canada is a once in a lifetime configuration update, hence, not something the company will find fruitful to plan for again, said Tauschek. But a post mortem is necessary, and the lessons should be applied to change management processes going forward. He added, “Bell will likely have to do that either way, because the government will be on them like they were on Rogers.”
Bell is seemingly of the same mind, stating that it “adjusted its processes and safeguards to ensure this type of issue will not happen again.”
That statement, however, did not stop industry minister François-Philippe Champagne from expressing his discontent with the matter in a tweet: “The telecoms outage that affected 911 services in Nova Scotia early this morning is unacceptable. As I have said before, Canadians expect and deserve reliable telecoms connection to access emergency services at all times.”
The minister responsible for the Nova Scotia Emergency Management Office, John Lohr, also called the outage ‘extremely concerning’, adding, “I think it’s clear that could cost somebody their life if they didn’t receive the services they need.”
Public trust in Canada’s national telcos is also seemingly at a low, as many users took to Twitter to convey their indignation. Some expressed their dissatisfaction with their service and the exorbitant rates, while others pointed out the fact that a similar outage happened for six hours, 15 years ago in the 613 area code when Ontario shifted to 10-digit dialing, yet Bell managed to repeat the same errors.
