Welcome to Cyber Security Today. From Toronto this is the Week in Review edition for the week ending Friday, April 8th, 2022. I’m Howard Solomon, contributing reporter on cybersecurity for ITWorldCanada.com.
 |
 |
 |
In a few minutes I’ll be joined by David Shipley, CEO of New Brunswick’s Beauceron Security, to discuss some of the news from the past seven days. But first a brief roundup of the headlines:
Some countries recognize April to be National Supply Chain Integrity Month. David and I will look at what IT leaders should be doing to ensure their software, hardware and partners aren’t abused by threat actors through indirect attacks.
Part of the supply chain are software development tools, such as VMware’s Spring Java application development framework. We learned that a serious security vulnerability dubbed Spring4Shell has been found that needs patching. David and I will discuss how serious this is.
And we’ll also examine a report on a tabletop exercise of a simulated cyber and physical attack against the power grid in the Western U.S. and Canada.
Also in the past week the FBI said it helped stop the launch of a botnet of thousands of compromised WatchGuard Firebox security appliances. The scheme was created by group within Russia’s military intelligence service. Known to researchers as Sandbox, the threat group has been accused of being behind the 2015 cyberattack on Ukraine’s power grid and the NotPetya wiper malware. The FBI says this time Sandbox was creating a botnet ready to spread what’s called the Cyclops Blink malware. I reported last month that WatchGuard Firebox and Asus routers were being infected by this malware. But on Wednesday the FBI said it worked with Watchguard to detect and remove infections from many devices. Others will have to be mitigated or have security patches installed by IT departments.
Meanwhile German police said they seized the servers of Hydra Market, a large darknet platform for selling drugs, money laundering and selling stolen and forged documents. Police also seized bitcoin worth about $25 million.
IT administrators with Linux systems in their environment were urged to install the latest security patches from their distributions immediately. This is because a serious vulnerability has been found in the operating systems’ security module.
Most data thefts come from attackers outside an organization. But employees or former employees get sticky fingers, too. The latest example comes from Block, the company that owns the mobile money transfer application called CashApp. Block is notifying 8.2 million current and former CashApp users that a former employee was able to download reports that included names, brokerage account numbers and in some cases brokerage portfolio holdings. There was no explanation of why the former employee still had access to data.
And a British chain that sells books and arts and crafts called The Works had to temporarily shut some of its stores after hackers got into its IT system. The attack disrupted some store cash register systems. The company said customer payment data wasn’t compromised but so far it hasn’t nailed down exactly what data might have been affected.
(The following transcript has been edited for clarity)
Howard: Let’s start with the vulnerability, or vulnerabilities, in the Spring Java Application Development framework. It’s been dubbed SpringShell or Spring4Shell by some security researchers. What is it?
David Shipley: It’s one of those classic Java bugs with heaps of complexity around using objects in ways people didn’t intend for them to be used, basically allowing [an attacker] to get remote code execution to pop a shell either a reverse shell or a web shell in a very specific set of circumstances. These involve JDK 9 and above, as well as Apache Tomcat. Initially when this first came out there was a very sort of large hue and cry and alarm about it. Now the pendulum has kind of swung a little too far to the other side where it’s like, ‘No big deal. Patch it when you can.’ The reality is some experts have said there will be shells from this. We’ve already seen one major vendor, VMware having to issue patches for some of its products that use Java webadmins as part of their systems.
Howard: So how serious is this vulnerability?
David: It’s not log4j, it’s not as easy to exploit or as widespread as log4j is. But it is not trivial either, because anything they [attackers] can get you to pop shells on an Apache server — of which there are quite a few, and this is not an uncommon configuration — there will be good usage. So it’s not one of those patches you want to delay or ignore.
Howard: Is applying a patch enough?
David: In this circumstance that seems to be the consensus from security researchers. The trick will be applying the patch when you actually control the application and you know how and where it’s used. All of the things that you use that use Java inside them that are also subject to this vulnerability. Waiting for those vendors to patch their stuff, that’s going to be the complexity with this particular situation.
Howard: One analyst at the SANS Institute says that if lessons learned from the panic over log4j and its library vulnerability that came out a few ah months ago were implemented your organization should have a decent inventory of which applications leverage the impacted frameworks in the in this vulnerability. Is that a big ‘if?’
David: That’s a huge ‘if.’ That’s a four-lane highway ‘if.’ Most of the organizations that I know that were dealing with log4j over the holidays and into the New Year were dealing with this specific issue: ‘We’ve got to reach out to all our vendors. We’ve got to do our own application inventory.’ So maybe some of that work can be recycled, but it comes to applications that you own or build internally. I doubt that everyone was doing a full-stack review of every single vendor stack, every single vendor’s open-source dependency stack — and having that organized. It sounds like a compliance stream come true, but it also sounds like a pipe dream.
Howard: Also this week the volunteer group known as the Dutch Institute for Vulnerability Disclosure — also called DIVD — released the English translation of a recent Dutch book on hacking. It has a chapter on how DIVD helped discover the vulnerabilities in the Kaseya VSA IT management suite last year. Those vulnerabilities were used by the REvil group to distribute ransomware to companies through managed service providers that used VSA. What struck you in reading this chapter?
David: I really enjoyed first the almost the joyful sense of old school [White hat] hackers hacking things and learning. The conversations where they realize, ‘Oh my god, I’ve got an 0-day [vulnerability.]’ And then they realize that this could have massive implications around the world — that sort of pit-of-the stomach awful feeling. And you can really get a sense of the ethics and morals of this particular crew … It reminds us all of the positive use of the word ‘hackers’ — the discovery of things in new and novel ways.
What I also really liked from reading this chapter was Kaseya didn’t dismiss them, so they had their CTO on. They started digging into it, and they were working as best they could. The part of the chapter that kind of leaves me with more questions is Kaseya is working on the patches with the hacker group, but all of a sudden this ransomware incident happens. Did the vulnerability leak? There was no evidence of that. There were some suspicions that may be an intelligence agency or Russia was already in widely in Kaseya. Once they realized that the vulnerability been discovered, did they use ransonware to cover their tracks? We’ll never know. The final piece that I thought was really interesting from the chapter, particularly in the global context we’re in now, is that some employees are critical of Kaseya’s culture. Particularly to outsource a lot of coding to a group of developers in Minsk in Belarus. It left me thinking at the end of this chapter– particularly because of the [Ukrainian] conflict we’re now is Kaseya still using developers in Minsk?
…
Howard: These two incidents — the vulnerability in the Spring application development framework and Kaseya — could both be used for indirect attacks on organizations. These are called supply chain attacks and this is a good segue into the next segment that we’re going to discuss, because this is National Supply Chain Integrity month. It was created a few years ago by the U.S. cybersecurity authorities to remind IT companies and firms that indirect attacks through their supply chains are as worrisome as direct attacks by threat actors. First of all, when we talk about the IT supply chain, how big is it? What does that encompass?
David: Nowadays, particularly with the push to cloud computing, it’s almost everything unless you are building your own desktop OS, server OS and your own applications from complete scratch. Otherwise you have exposure to a complex global interdependent supply chain. If you’re an Office365 customer your supply chain is Microsoft, if you’re using Google Workplace it’s Google. All of the apps and things that we use have that relationship and you rely on all of those things. What’s interesting is it’s even more complex than just your relationship with a vendor. It can be the vendor’s relationship with people providing support. So remember that the Octa breach actually involved support supplier Sitel and a subsidiary called Sykes. Supply chain is an incredibly challenging strategy issue for people to address because most of the companies that we talk to don’t even have a complete software inventory of whose stuff they’re running to begin with.
Howard: There are many examples of supply chain attacks and perhaps the most notorious was the SolarWinds attack where the software update mechanism of the SolarWinds Orion IT management suite was compromised, and a number of organizations who downloaded that update left them vulnerable to being attacked.
David: And who do you trust? The example that always terrified me was the NotPetya attack in 2017. You had two small Ukrainian firms, one of which had software for doing taxes. They get popped and the software is used to push out a ransom worm that takes out hundreds of thousands of devices inside Ukraine — and accidentally jumps into Maersk, Merck and other companies because their networks were interconnected. Andy Greenberg does a great job covering this in his book Sandworm.
… And that’s why Microsoft president Brad Smith and others have been so big about going after governments for hiding vulnerabilities they know about, weaponizing vulnerabilities to use in espionage or potential sabotage attacks. [It’s been alleged that the U.S. designed Windows Eternal Blue exploit was stolen and ended up being behind NotPetya.] …
When you talk about Supply Chain Integrity Month, it’s the whole idea that it’s not just good enough to try and secure yourself. You have to think about the relationships you have with your vendors, nd how well they’re patching their stuff, and where they’re based.
Howard: Have IT leaders learned the lessons over the years for protecting their supply chains?
David: Not even close, not even close. When you look at the things like the NIST cybersecurity framework, one of the basics is knowing what you have, who’s using your systems etc. Most organizations don’t even get that done because we’re so eager to just keep the lights on.
Howard: Possible supply chain vulnerabilities include the code in your apps which leads to the idea of getting a software bill of goods for every application IT departments use. Where’s that idea?
David: It’s been mentioned a few times by the President of the United States in various executive orders. The challenge is going to be making it as simple and easy to use for technology as the nutrition guide on a box of cereal. And I think it has incredible utility because, back to the earlier point the SANS report made, if you had done your incident response correctly for log4j you created an updated inventory. Imagine that inventory could be built for you and maintained automatically. That’s the critical part. As an idea, I love the software bill of goods, particularly as the way of quickly being able to alert for things like Spring4Shell or log4j. But the standardization of that requirement, we’re not there yet. Hopefully, soon.
Howard: So what should IT leaders be doing to secure their supply chains?
David: Mapping the supply chain is probably a really good start. When you create those maps you have an understanding of who do we do business with, what software do we have? Then you can start asking questions about that software and the provider, how they provide it. The next most important thing is asking your vendors who they depend on to deliver your services. In the case of my company, we depend on Microsoft and Azure. So when customers are dealing with us we explain how we work with Microsoft … So if a headline pops up about an issue with Azure a customer can say, ‘This software that we use uses Azure, I can reach out to the vendor and say ‘Are you affected by this particular issue?'”
Howard: Finally I want to look at a report issued Thursday by the North American Electric Reliability Corporation otherwise known as NECR. It’s a body that co-ordinates efforts by Canadian and American power suppliers to ensure the North American power grid can’t easily be taken down by electric storms, physical or cyber attacks. This report was on lessons learned from a tabletop exercise it held last November simulating cyber and physical attacks by an unnamed nation-state. There were 3,000 participants playing virtually, and they included officials not only from power companies and suppliers but also from the governments of the United States and Canada. The two days of simulated attacks included explosions tripping generators offline, cyber attacks against industrial control systems and physical attacks on pipelines. In some cases participants only knew about incidents through simulated TV reports — just like. In real life. There were also simulated disinformation campaigns on social media. Some important employees received vague but credible threats against themselves and their families by robocalls. This was a really tough simulation. What did you get out of this report?
David: I was encouraged by the language, particularly that was highlighting how much Canadian participation had increased in this particular exercise. It makes me feel good that we’re starting to show up and take these issues much more seriously particularly, in light of. President Biden’s comments that Russia was exploring attacks against critical infrastructure as part of its potential retaliation for [Ukraine-related] sanctions. so I’m really glad that we’re coming to the table more … The fact that the exercise demonstrates the length of time that a major grid outage could have and how tenuous restoration could be within even a two-week time frame was an eye-opener. When we think about resiliency the advice that we still give to Canadians and organizations is [be ready] for about 72 hours. What we should start to think about is longer horizons of a week or two weeks. Finally, the breadth of the scenario they crafted used a variety of cyber, physical and social aspects. This is exactly what we would expect in the real world. For tabletop exercises to be useful they should be challenging and realistic, and I think they did a really good job with that.
Howard: One lesson that I saw in the report’s conclusion was that there has to be better communication between partners. And in the electric grid there are a heck of a lot of companies — natural gas suppliers, companies that actually do the power generation, in some cases there are separate distribution companies, there are regulators, there are governments. Of course that’s one of the whole points of a tabletop test — do you have an incident response plan, is there good communications not only between people within a company but also with the people outside the company who you deal with.
David: And the goal of these exercises isn’t to do high fives and congratulations we passed the exercise. It’s to work the scenario, to find those kinks and those problems — like where’s your incident response plan? It’s in our Sharepoint. Your Sharepoint’s not accessible [because the computers are down]. Great lesson learned: a physical copy of the incident response plan might be a good thing. One of the evolutions of the scenario that I think about a lot is participants were learning through little simulated newscasts. If I was going after the North American grid I’d go after the media companies. It’s entirely possible to bring down the media and then slow the incident response.
Howard: One of the things that the report said was in the simulation there was a plunge in power supply. After two days about 50 per cent of the [normal] capacity had been restored but the report also says going farther out after two weeks full power had still not been restored to pre-attack levels. Which sort of gives you an idea if there is a nation-state that launches an attack it can cause serious damage and organizations still aren’t ready for that. Although to be fair if somebody’s lobbying a hand grenade at an electrical power generator it won’t be easy to find generators to immediately plug in. Having said, that the whole point of having a resllient network across North America is if one part of the grid can be taken out power can be in some cases be routed around. But as I said, it was a hard test.
David: It was and I think it’s reassuring to know that they’re putting this calibre of a test out there. I often think about how much more complex managing the grid is going and so to see this level of exercise for the grid complexity we have now great. But in the foreseeable future it’s going to become even more complex if you’ve got people who can take power from the grid store, it in their [electric] Ford 150 and power their house, what if the truck gets malware?
Howard: I noted that the after-action report also said that the scenario included a widespread loss of landline and cellular communications. The report has this line: ‘Participants agreed that the loss of communications would essentially halt the electrical grid restoration process.’ So efforts to bring power back would be stopped just by making sure that the phone lines are out.
David: A couple of years ago there a massive outage of internet and cellular capability in Atlanta Canada when, as the story goes, two critical fiber linkages at the exact same time were cut. That was an eight-hour outage. One of the things from Ukraine that’s been really interesting is competitive telco companies there are now fully integrated together and helping each other stay online. Are we ready for that kind of carrier co-operation.
Howard: This was one heck of a big tabletop exercise. Let’s shrink it down to a single company doing one. In your experience are the simulated tests being done by companies tough enough?
David: The organizations I know who are doing tabletop exercises are mostly large complex enterprises and governments. Very few mid-sized organizations. And I’ve I have yet to come across a single small one doing a tabletop. Those who do it typically do it well. The formula I’ve been hearing is they bring in someone from the outside who interviews them, builds the scenario and then brings the players together for the day. And they really do work it hard and they make it realistic, because otherwise they’re wasting their money — these exercises can cost tens of thousands of dollars. But when done well they deliver value. I think there is a challenge of scaling this down for midsize and small businesses.
I think just having a good incident response plan and then just doing a reading of that plan [with employees] and thinking about it is the most likely scenario [for a small firm]. If you’re a small business listening to this podcast, ransomware should be your practice [scenario] right now. Your second one is a vulnerability in some critical piece of software you rely on. There’s a race between the patch and the attackers — how would you deal with that? Those are realistic scenarios that you could do. And practice your instant response plan. You’ll find flaws, and that’s the point. You’ll find flaws and you’ll fix them. Better to live it in an exercise than when it’s the real deal.