Ivanti CSA Investigation/Detection Details  

October 9, 2024

Authors: Rui Ataide, Andrew Nelson, and Hermes Bojaxhi

GuidePoint Security has recently been engaged on several incidents related to five Ivanti Cloud Service Appliance (CSA) vulnerabilities CVE-2024-8963, CVE-2024-8190, CVE-2024-9379, CVE-2024-9380, and CVE-2024-9381.  We wanted to share some tips based on our experience on how to investigate these appliances for signs of compromise. If you believe you are a victim of this exploit and would like further details on this vulnerability and the indicators of compromise, please reach out to GuidePoint via our Report an Incident page.

CVE-2024-8190 was publicly disclosed on September 10, 2024, with a severity score of 7.4 (High). An OS command injection vulnerability in Ivanti Cloud Services Appliance versions 4.6 Patch 518 and before allows a remote authenticated attacker to obtain remote code execution. The attacker must have admin-level privileges to exploit this vulnerability.  A public POC exploit has been published by Horizon3.ai.

CVE-2024-8963 was publicly disclosed on September 19, 2024, with a severity score of 9.4 (Critical). Path Traversal in the Ivanti CSA before 4.6 Patch 519 allows a remote, unauthenticated attacker to access restricted functionality. 

CVE-2024-9379 was publicly disclosed on October 8, 2024, with a severity score of 6.5 (Medium). SQL injection in the admin web console of Ivanti CSA before version 5.0.2 allows a remote authenticated attacker with admin privileges to run arbitrary SQL statements.

CVE-2024-9380 was publicly disclosed on October 8, 2024, with a severity score of 7.2 (High). An OS command injection vulnerability in the admin web console of Ivanti CSA before version 5.0.2 allows a remote authenticated attacker with admin privileges to obtain remote code execution.

CVE-2024-9381 was publicly disclosed on October 8, 2024, with a severity score of 7.2 (High). Path traversal in Ivanti CSA before version 5.0.2 allows a remote authenticated attacker with admin privileges to bypass restrictions.

In the Wild

During its investigations, GuidePoint has observed the unauthenticated access vulnerability exploited in conjunction with SQL injection and command execution since the beginning of September 2024. Since details on how CVE-2024-8963 (unauthenticated access) is achieved are not publicly available, GuidePoint will not reveal this technique here to avoid mass exploitation of vulnerable CSA appliances.

Based on GuidePoint’s experience, these vulnerabilities have been successfully exploited by what is likely a nation-state actor, whose goal is to remain stealthy and maintain persistence. However, since the disclosure of the vulnerabilities, the type of threat actors and their goals may change.

On the Ivanti CSA appliance itself, GuidePoint has observed the following indicators of compromise:

• Upload of webshells via the exploit
• Execution of native tools via the exploit (nc & curl)
• Tunneling and scanning tools

GuidePoint has also observed lateral movement into the Windows environment where techniques such as DLL side loading, creation of services, and scheduled tasks were used.

GuidePoint has observed at least one webshell was uploaded to multiple CSA appliances in early September, prior to the disclosure of the vulnerabilities. During September 11 – 13, shortly after the disclosure of the first vulnerability on September 10, there was another round of webshells uploaded to these CSA appliances. Furthermore, it was during this time that some of the other tools and backdoors started to appear on some of these appliances.

The Attack Surface

Ivanti CSA devices can be easily identified through sites like Censys with the search in Figure 1. At the time of this post, Censys showed 723 publicly exposed devices.

(services.software.cpe=`cpe:2.3:a:ivanti:cloud_services_appliance:*:*:*:*:*:*:*:*`)

Figure 1: Censys Query

Based on some preliminary analysis GuidePoint conducted on these Censys results, there are still possibly 379 vulnerable appliances as of the writing of this blog. Based on GeoIP information, they have the following distribution by country:

Figure 2: IP Address Geolocation of Vulnerable CSA Systems

GuidePoint has identified 88 appliances that contain at least one webshell, with 42 of those containing at least two webshells. Since webshell filenames can vary, these numbers only account for the webshell filenames that GuidePoint is aware of.

Triage Steps

GuidePoint recommends that organizations perform a triage of their CSA appliance, regardless of the patch status,  to ensure that they were not compromised prior to applying any patches. This is important because the patching process does not address any potential backdoors that may have already been created on the appliance.

The commands in this section can be executed directly on the CSA appliance itself to determine signs of exploit attempts and potential success.

Please note that the queries below are meant only as preliminary investigative steps that should be followed up with further analysis if any signs of successful compromise are observed. By default, appliances retain logs for around seven days, so GuidePoint recommends using available backups for analysis of the early to mid-September timeframe. In cases where backups are not available, then the triage process below will likely only find file system artifacts, such as webshells.

Identify files changed during September and October 2024 under application directory

find /opt/landesk/ -type f -name \*.php -exec stat {} \; | grep -B7 -E "Change: 2024-[01][90]-"

Here you are looking for any PHP files created since September 2024. In all instances, GuidePoint has identified webshells that are typically less than 100 bytes. In some instances, there were also additional larger webshells. A sample result may look like this:

Let’s assume you find files that meet this criterion: file1.php, file2.php, and file3.php. Use these filenames (without the extension) in the next query.

Detect usage of identified webshells from previous command

zgrep -iE "POST /.*?/(file1|file2|file3)\.php " /var/log/messages*

Identify Suspicious Web Activity

zgrep -iE "POST /[^g].*?/(datetime|broker|reports|users)\.php " /var/log/messages*

Identify Suspicious Command Execution and SQL Injection

zgrep -iE "(php-ini|php-cgi|echo|curl|bash|/dev/tcp/|\`)" /var/log/messages* | grep -v GET

zgrep -iE "(php-ini|php-cgi|echo|curl|bash|/dev/tcp/|/webroot/gsb|\`)" /var/log/sec* | grep -viE "/su?bin/(broker|backuptool|dbtool -[uc] -b -V255|iptables -L)"

Check the content of /etc/php.ini file for the following directives:

“allow_url_include = On”

“auto_prepend_file = "data:;base64,<BASE64 code>"

Convert command execution from audit log into human-readable format to identify suspicious activity

awk '/cmd=.*?success|failed.$/ {
  timestamp="";
  cmd="";
  res="";
  uid="";
  if (match($0, /msg=audit.([0-9]+)/, arr)) {
    timestamp = strftime("%Y-%m-%d %T %Z", arr[1]);
  }
  if (match($0, /uid=([0-9]+)/, arr)) {
    uid = arr[1];
  }
  if (match($0, /cmd=([0-9a-fA-F]+)/, arr)) {
    hex_cmd = arr[1];
    for (i=1; i<=length(hex_cmd); i+=2)
      cmd = cmd sprintf("%c", strtonum("0x" substr(hex_cmd, i, 2)));
  }
  if (match($0, /res=([a-zA-Z]+)/, arr)) {
    res = arr[1];
  }
  if (timestamp && uid && res && cmd) {
    print timestamp, "uid=" uid, res, cmd;
  }
}' /var/log/audit/*

Additional Investigative Steps

• Check for any outbound communication from the Ivanti appliance to the Internet. You are looking for any destination IP addresses that are atypical of the CSA appliance, especially with higher bytes in and out of the network. Ensure the query encompasses the September 1, 2024 – present timeframe.
• Check for any unusual traffic from the CSA appliance to your internal network. Here, you are looking for any port scanning or any connections from the CSA appliance to internal IP addresses on unexpected ports such as 22, 445, and 3389.
• If you find signs of compromise in your CSA appliance, pause the VM or take a snapshot of it prior to shutting it down or making any changes to it to preserve forensic artifacts. Some of malicious artifacts are located under the /tmp directory, which would be lost during a reboot.
• If there are signs of compromise, GuidePoint advises that you build a new appliance from scratch using the latest version available from Ivanti.

*** This is a Security Bloggers Network syndicated blog from The Guiding Point | GuidePoint Security authored by Hermes Bojaxhi. Read the original post at: https://www.guidepointsecurity.com/blog/update-from-the-trenches/