With files from Howard Solomon.
Rogers Communications experienced a wide-scale internet and wireless service outage across Canada on July 8.
Update July 9 at 6:30 a.m. EST
Looks like services are finally returning to normal. Rogers wireless and internet services began recovering at around 9:30 p.m. EST on July 8. The telecommunication giant issued the following update on the morning of July 9:
“Following our previous updates, we have now restored services for the vast majority of our customers and our technical teams are working hard to ensure that the remaining customers are back online as quickly as possible. As our services come back online and traffic volumes return to normal, some customers may experience a delay in regaining full service. Once again, we sincerely apologize for the disruption this has caused our customers and we will be proactively crediting all customers.”
Global News reported that Rogers has confirmed that it did not suffer a cyber attack.
Update July 8 at 5:34 p.m. EST
Rogers says it’s still determining an ETA to restore services. It also plans on “proactively crediting all customers.” More information to come.
Update July 8 at 4:07 p.m. EST
The outage remains active as of 4:07 p.m. EST, almost 12 hours since it began. In a new notice, Rogers says it’s working with its “global partners” to address the issue and that it’s making progress.
Rogers sources its telecommunication equipment from Ericsson, a Swedish telecommunications and network hardware supplier. In April 2021, an Ericsson software update knocked Rogers wireless services offline for nearly 24 hours.
Rogers still has yet to release the official reason for the blackout.
Francois-Phillipe Champagne, Minister of Innovation, Science and Industry said in his public statement that his office has contacted Rogers to ask the company to resolve the issue quickly and provide clear communications to those impacted.
The original story continues below
“An outage is impacting wireless and home services. Phone and chat support are also affected. We are working to restore as quickly as possible,” reads a notice on Rogers’ support page.
According to Downdetector, more than 20,000 reports of issues were generated starting at around 5 a.m. EST.
Heavily affected locations include Toronto, Ottawa, Montreal and Winnipeg.
While the company still hasn’t provided a reason for the outage, it has acknowledged it on its customer service Twitter account.
Fido, Rogers’ flanker mobile service brand, is also reporting more than 10,000 outages on Downdetector.
Outage affecting services across Canada, including Interac and 911 services
In a reply to a customer inquiry on Twitter, TD Canada Trust said Interac e-Transfer is down due to the outage and all banks are affected.
Several major businesses, including Starbucks, Rexall and Beck Taxi, are not currently accepting Interac payments. Credit payment methods remain operational.
Without reliable internet in their homes, remote workers are flocking to coffee shops to connect with their workplaces.
“We had a meeting at 9 a.m. but I couldn’t access Wi-Fi either at home or at the office. I tried two Tim Hortons locations then connected to weak Wi-Fi,” said Nicole Zhang, account manager of North American Ecosystem Institute.
One Starbucks retail location in Toronto told the publication that its Wi-Fi network has been slammed due to the influx of patrons.
Staff at Enterprise car rentals in Toronto has been forced to process car rentals on paper because their wireless tablets rely on the Rogers network and aren’t working.
The Toronto police service is seeing a slowdown in operations due to the outage.
ADVISORY:
City of Toronto
– The Rogers Network is experiencing some technical difficulties
– Some people will have trouble connecting
– There are some connection problems calling 9-1-1
– We are working to resolve these issues
^dh— Toronto Police Operations (@TPSOperations) July 8, 2022
The Canadian Radio-television and Telecommunications Commission (CRTC) announced on its website that it’s experiencing phone interruptions caused by the outage.
Global News reported that the Canadian Centre for Cyber Security has contacted Rogers and “offered assistance, in the event they should need it.”
Bell and Telus are doing fine, mostly, but regional providers are affected
A Bell spokesperson confirmed to IT World Canada that its services are functioning normally.
Telus said on Twitter that it’s “experiencing higher than expected wait times” due to a “competitor network issue.” In a statement to the publication, the company added that some Telus users in Ontario may experience slower than normal data speeds due to increased usage by those without access to home internet service.
But some are not as lucky. Ontario-based TekSavvy, a regional internet service provider that delivers a portion of its services using Rogers’ network, reported that its networks are down across Ontario and Quebec.
Organizations need to keep resiliency in mind
Internet interruption can be caused by a wide range of things: Hacking, misconfigurations, network congestion and even the weather.
With organizations needing not only internet connectivity for PCs and servers but also for VoIP phones, point of sales devices, industrial equipment and other IoT systems the incident shows the need for organizations to pay for a WAN internet failover/backup service. It’s offered by many major and small ISPs. A solution can involve the addition of a router to the IT network that switches to the alternate service when disruption is detected.
It can be provided in a number of ways. Assuming an organization’s provider is a wired service, the failover could be a cellular or cable internet service.
Canadian providers include Rogers, Bell, Ooma, TeraGo, ITel, Call-One, Skyway West and many more. Note this is not an inclusive list.
“This outage certainly demonstrates a number of challenges businesses need to cope with,” said Ed Dubrovsky, managing director of Cypfer, a Toronto-based incident recovery consulting firm.
Large communication service providers are not immune to cyber-attacks or critical technology infrastructure failures, he pointed out. Digital currency is exceptionally vulnerable to failures which may impact the ability to process digital payments.
An increasingly remote workforce is reliant on limited infrastructure typically from a single provider, he warned, so organizations may see an impact of entire segments of their employees disabled.
When a communication infrastructure is distrusted, the process of returning to operations may create vulnerabilities that can be exploited by in-waiting attackers, he added.
