Welcome to Cyber Security Today. From Toronto, this is the Week in Review for the week ending Friday, July 22nd, 2022. I’m Howard Solomon, contributing reporter on cybersecurity for ITWorldCanada.com.
 |
 |
 |
Terry Cutler of Montreal’s Cyology Labs will join me in a few minutes to discuss recent news in cybersecurity. But first a review of some of what happened in the last seven days:
The Canadian Anti-fraud Centre, which handles reports on all types fraud, has been victimized by a phishing scam. Crooks are sending out emails purporting to be from the centre warning victims their personal data is being misused. The goal is to trick people into clicking on a link to find out more. While the sender’s address spoofs the Anti-fraud Centre, the link in the email is a tip-off that it’s a scam because it goes to an address called “mountainbuffalo.” Terry and I will discuss how organizations can protect themselves from such scams.
We’ll also talk about the big Rogers internet outage earlier this month.
Developers of e-commerce platforms have again been warned to tighten security around their applications. This comes after researchers at Recorded Future discovered payment card skimming scripts have been inserted into three online ordering platforms used by over 300 restaurants in the U.S.
The U.S. has seized cryptocurrency accounts with about US$500,000 stemming from ransomware attacks. Some of the money included ransoms paid by healthcare providers in Kansas and Colorado. North Korean hackers are believed to have been behind the attacks. An FBI investigation into the incidents led to the discovery of the previously unseen Maui strain of ransomware.
Four new ransomware strains were described this week. One is called Luna. According to researchers at Kaspersky, it goes after Windows, Linux and VMware systems. The other three have been spotted by researchers at Cyble. They are called Omega, Lilith, and RedAlert, which, like Luna, can exploit Windows, Linux and VMware systems.
Linux vulnerabilities and poorly-secured cloud application configurations are being leveraged to expand a criminal botnet. Researchers at SentinelOne said the botnet run by the 8220 Gang has jumped to include 30,000 hosts. Those infected devices are chewing up power on corporate servers for running cryptomining applications.
Cisco Systems released security patches for a number of products, including its Nexus Dashboard for data centres and cloud network infrastructure, certain models of Cisco Small Business routers and the Cisco IoT Control Center.
Owners of Apple devices should make sure they have installed the latest security patches. This week the company issued updates for iOS, iPadOS, macOS, tvOS and watchOS. Usually they are installed automatically, but it doesn’t hurt to check that’s been done. It’s more important than ever to be running patched devices after researchers at ESET discovered spyware called CloudMensis has been targeting Macs.
(The following transcript has been edited for clarity. To hear the full discussion play the podcast)
Howard: Let’s start with something that I didn’t get a chance to discuss last week because I was off, and that’s the huge service outage suffered by Rogers subscribers two weeks ago today. Almost its entire communications network — internet and cellular — went down for hours. That hit businesses, government departments and hospitals. Some people were unable to make 911 calls. It was an unprecedented event, and one that leaves the company open to complaints that it doesn’t have resiliency. Rogers blamed the outage on a maintenance update by its programmers to its core network, which caused some of our routers to malfunction. Today Rogers will present a report to this country’s telecom regulator, the Canadian Radio-televison and Telecommunications Commission, on what happened and what it plans to do to prevent a recurrence. Hopefully a version that hasn’t been too sanitized will be made public. There will also be Parliamentary hearings. Terry, what did you think when this happened? Defensible or inexcusable?
Related content: What the CRTC wants Rogers to explain
Terry: We’ve seen these types of malfunctions happen in the past with other vendors and suppliers. So when I saw this happening I thought an update had gone wrong or there was something wrong in the core switch. In my mind it came down to three things: Human error, a disgruntled employee or, worse, a cyber attack from Russia because of the new sanctions imposed by the Canadian government.
Howard: But Rogers says, no, it was a maintenance update, it was completely our fault. It wasn’t an update from a third-party supplier. What struck me is network maintenance updates wouldn’t be uncommon at any telecom carrier. So why would this one have had such a huge impact?
Terry: In my experience even doing updates on a computer or a switch can be a problem. If a switch was set in limbo, for example, if it was malfunctioning but still showed signs that it was perfectly fine, the moment you do an update it could malfunction. That’s why IT guys are always sweating whenever they’re doing a firmware update or something, because if it goes wrong it’ll fry the device. We’ve seen cases where we were doing updates to a hard drive’s firmware and it wiped out the drives. In this case the core switch update went wrong and all the switches and all the [cell] towers lost communications — but they were still in an online state and still able to answer requests. But they weren’t able to fulfill the request because the routes and services were missing. What was interesting is why would anybody want to do a major update on a Friday morning? That was the only part that really bothered me. If you’re doing major plays like this at least do it on the weekend, or at one o’clock in the morning when maybe less load.
Howard: And of course one would expect that Rogers’ developers would have tested and retested again on a parallel network to make sure that there wouldn’t be a problem.
Terry: That is a great comment, I always hear, ‘We have a development network, a testing environment, and then we have the production network,’ which is supposed to be identical. In over 20 years of IT it’s extremely rare that I see a development environment be identical to production. There’s always some minor change, one minor difference and that minor difference could make all the difference in an outage.
Howard: What does this say about the ability of Rogers developers to write clean code?
Terry: I don’t know if I would necessarily blame them, because you know they’re all using industry-level technology like Cisco, Juniper, Ericsson and such. I think that maybe one of the routers was in a limbo state. Or one of the common problems that we see sometimes is what’s called a unicode problem. That’s where maybe a developer copied and pasted code from an HTML page into a text format and because it didn’t convert properly there was a character that wasn’t able to be interpreted properly which could cause the whole update to fail.
Howard: One of the things the crisis showed was how unprepared Canadian organizations are. They’re too trusting that one internet supplier will do for all their needs. They’re really betting that supplier is not going to have a major outage. But Interac, which runs the network that Canadian businesses rely on for credit and debit card payments didn’t have the ability to switch to another network if the Rogers network went down. I suppose Interact might argue that over the years Rogers’ network has been fine, but this incident also shows that Interact just wasn’t prepared.
Terry: At the same time the Rogers network was still somewhat online. Phones could still connect to it, it just wasn’t able to fulfill the request. But there was no failover.
Howard: As result of this incident Interact quickly said it will add another network for failover. But even before this incident businesses could have subscribed to an internet failover service to protect themselves if they wanted to shell out for that. Again, that’s a cost. But I think that many organizations just trust the major Canadian carrier that they’re with is not going to have problems.
Terry: One of the carrier network designs now is a technique called all-in-one IP, where all of the core services run in one location. The security policies are running in there, the routes, and then you have all the cell towers on it. The perimeter talks to the backend. [In an outage] those towers are technically still online but behind the towers. They can’t connect to the core service. That’s why sometimes you could be traveling and still connect to a Rogers tower. But then after that it just dies, so it stays connected. That’s why the failover didn’t kick in. We’ve seen that happen countless times in the server world where the server is online but the service is in limbo.
Howard: Here’s another thing: Telecom carriers have been meeting under Canadian federal guidance for years to share best practices to make their networks more secure. For example, the Canadian Security Telecommunications Advisory Committee was established in 2010. It includes a telecom resiliency working group. So resiliency is no secret in the telecom industry. It makes you wonder what are they doing when they’re having their discussions and the representatives from the various telcos go back to their offices. I’ve been asking Ottawa about the work of that group. And all I got back was a statement saying resiliency is a shared responsibility between the public and private sectors.
Terry: This sounds like more like a best practices advisory committee: Here’s a list of dos and don’ts. But they don’t have the specifics of Rogers’ internal network. They might say okay, you need to add another billion routers to this network. But if those routers receive a bad update it won’t make a difference. Some telecom sharing should be involved. For example, Rogers could work with Tellus and Bell, but then you’re opening up vulnerabilities. And then there’s competition. If you share how your network will be better — or worse — than the competition, they could start stealing your customers. It’s a very fine line.
Howard: Rogers has acknowledged that it’s now working on separating its wireless network from the internet network so that one problem can’t bring down both of them.
The other thing that bothers me about this incident is that some people in the media are discussing it with complaints about the lack of competition in telecommunications here. You have a major outage and somebody says that shows that there’s a lack of competition. I don’t see the connection between a lack of competition and what appears to be a major mistake made by someone or some group within a major carrier.
Terry: This problem can happen to any carrier, including Telus and Bell, because they’re using the all-in-one IP network design. I think what’s going to happen is you want to add more redundancy. More technology, more complexity and prices are going to go up, but people don’t want to pay more. I agree Rogers should be segmenting off its network. But if that’s going to add more complexity and customers have to log in to two separate systems … they might not like it.
Howard: We certainly will look forward to seeing the Rogers report that was sent to the CRTC. And I certainly expect sparks to fly at the Parliamentary hearings that will be coming up shortly.
Terry: I think human error is going to be blamed. But at the same time it’s extremely difficult to avoid these types of errors because you can go through several checks and testing, but you never know what’s going to happen when they hit the upload button.
Howard: Moving on, I thought we should look at the recent report of the new U.S. Cyber Safety Review Board into the Log4j2l crisis. You may recall that crisis began with a discovery late last year of a vulnerability in this open source library that allows logs to be kept in applications. Many developers add this tool to their applications rather than creating a logging library themselves. The problem is that Log4j is so widespread it’s hard to update. Not only that, some have been abandoned by their developers and so aren’t being updated.
Here’s one of the report’s key findings: “The Log4j event is not over. Log4j remains deeply embedded in systems. And even within the short period available for the review board’s review community stakeholders have identified new compromises, new threat actors and new learnings. As a result we in IT industry have to remain vigilant against the risks associated with this vulnerability and apply best practices.” Do you agree?
Terry: I agree. If we just back up a bit, the review board are housed under the U.S. Department of Homeland Security, and they’re loosely modeled on the National Transportation Safety Board. It investigates train derailments and plane crashes. But the Cyber Safety Board’s information is a bit limited on how much companies or governments are disclosing threats. So the report showed big warnings that companies need to address. The biggest is they’ve got to work closer together. But we’ve been talking about this for 15 years. The board calls the Log4j2 vulnerability an endemic. Which means that this is going to be lingering around for the next 10, 15 years because there are so many intricacies between the logging capabilities and not proper patching. Maybe some new flaws will be found … So even though the Apache Software Foundation issued patches the biggest problem is around patch management.
Howard: So first of all, you’re behind the eight ball as an IT department if you don’t have an inventory of all your corporate applications.
Terry: Absolutely. A lot of companies may say ‘We’re not vulnerable,’ but they may have wrong information. So inventory is going to be key here. A lot of companies don’t realize that they have Log4j running because they’ve never done an audit or an asset inventory.
Howard: And it’s not merely that you have to keep an inventory of your applications, you need to have an inventory of what’s in the applications. This brings up a concept called a software bill of goods. This has been a long proposed. It lists all the open source components that are inside an application. That will help IT departments know which applications have to be patched when vulnerabilities are found.
Terry: But some of the issues we’re going to find in the future — and we’ve always had this problem — is when you start mass deploying the updates. It might break some functionality somewhere and if you’re the IT guy you don’t know how these services are being used. Your job is to make sure the service is online but the moment you patch it, it could break a functionality. Next, your help desk is lighting up [with complaints] and you have no idea what just broke.
Howard: And I think it all points to how imperative it is for developers to have secure code writing practices.
Terry: But at the same time they’re under the eight ball, too, because upper management wants this release to come out ASAP even though it’s not finished yet. They might roll it out half-baked because of the pressure. So unfortunately, when things are rushed vulnerabilities could be created.
Howard: But rushing shouldn’t be an excuse.
Terry: I agree, but it’s a problem.
Howard: Another thing I want to look at today is a warning from the Canadian Anti-fraud Center that it’s the victim of a phishing scam. As I said at the top of of this podcast, someone has been sending out emails from a spoofed CAFC address to people claiming a fraud complaint in their personal information has been sent to the center. To see details the victim has to click on a link. That link is infected. The email looks convincing because it has a reference number and it’s written in good English and in particular the sender’s email looks legitimate but there are clues that this email is fraud. One of them is that email link. Look closely and it doesn’t go to the antifraud center. It goes to some internet address called “mountainbuffalo.” Sharp-eyed people shouldn’t be fooled, but it struck me that the spoofing of the email address could fool a lot of people.
Terry: This is classic phishing. We [penetration testers] use similar techniques when we phish employees at a company to make sure they’re doing their awareness training properly. We can change the display name of the sender and make it look like it came from anybody … Users need to be suspicious of a sense of urgency in these emails suggesting you need to click on something. Instead you should be trained to just hover over links before you click on them. And maybe the English in the message might be broken. Those are ways to know if it’s a phishing scam.
Howard: How can organizations protect their brand from being spoofed like this?
Terry: It’s extremely difficult. I get this question all the time. ‘How come spamming is still happening? Why don’t you just shut these systems down?’ It’s because scammers will set up a system on a legitimate IP and start sending out spam emails as if it’s like coming from the Antifraud Center. When there’s enough complaints against that domain the internet provider will block that IP from ever working again. But then the scammers will just set up a new system with a new IP address and carry on. It’s very very difficult to stop these scammers from spamming everybody.
