Updates
Oct. 20: Identified an additional vulnerability (CVE-2023-20273) that is exploited to deploy the implant. Fixes for both CVE-2023-20198 and CVE-2023-20273 are estimated to be available on October 22. The CVE-2021-1435 that had previously been mentioned is no longer assessed to be associated with this activity.
Oct. 19: Added additional attacker IP and username, defense evasion observations, and new Snort rules. Also added new information regarding our assessment that the activity is being carried out by the same actor.
Overview
- Cisco has identified active exploitation of a previously unknown vulnerability in the Web User Interface (Web UI) feature of Cisco IOS XE software (CVE-2023-20198) when exposed to the internet or untrusted networks. This affects both physical and virtual devices running Cisco IOS XE software that also have the HTTP or HTTPS Server feature enabled.
- Successful exploitation of CVE-2023-20198 allows an attacker to gain privilege level 15 access to the device, which the attacker can then use to create a local user and login with normal user access.
- Next, the attacker can use the new unauthorized local user account to exploit a second previously unknown vulnerability (CVE-2023-20273) in another component of the WebUI feature. This allows the adversary to inject commands with elevated (root) privileges, giving them the ability to run arbitrary commands on the device.
- The recommendation that Cisco has provided in its security advisory to disable the HTTP server feature on internet-facing systems is consistent with not only best practices but also guidance the U.S. government has provided in the past on mitigating risk from internet-exposed management interfaces.
- Cisco support centers collaborated with the security team after using methods and procedures to correlate similar indicators in a very small number of cases out of our normal substantial daily case volume.
- These are critical and high severity vulnerabilities, respectively, and we strongly recommend affected entities immediately implement the steps outlined in Cisco’s PSIRT advisory.
Cisco identifies suspicious activity
We discovered early evidence of potentially malicious activity on September 28, 2023, when a case was opened with Cisco's Technical Assistance Center (TAC) that identified unusual behavior on a customer device. Upon further investigation, we observed what we have determined to be related activity as early as September 18. The activity included an authorized user creating a local user account under the username “cisco_tac_admin” from a suspicious IP address (5.149.249[.]74). Instances of this activity ended on October 1, and we did not observe any other associated behavior at that time other than the suspicious account creation.
On October 12, Cisco Talos Incident Response (Talos IR) and TAC detected what we later determined to be an additional cluster of related activity that began on that same day. In this cluster, an unauthorized user was observed creating a local user account under the name “cisco_support” from a second suspicious IP address (154.53.56[.]231). Unlike the September case, this October activity included several subsequent actions, including the deployment of an implant consisting of a configuration file (“cisco_service.conf”). The configuration file defines the new web server endpoint (URI path) used to interact with the implant. That endpoint receives certain parameters, described in more detail below, that allows the actor to execute arbitrary commands at the system level or IOS level. For the implant to become active, the web server must be restarted; in at least one observed case the server was not restarted so the implant never became active despite being installed.
The implant is saved under the file path “/usr/binos/conf/nginx-conf/cisco_service.conf” that contains two variable strings made up of hexadecimal characters. The implant is not persistent—meaning a device reboot will remove it—but the newly created local user accounts remain active even after system reboots. The new user accounts have level 15 privileges, meaning they have full administrator access to the device. This privileged access to the devices and subsequent creation of new users is tracked as CVE-2023-20198.
Upon successful exploitation of CVE-2023-20198, the attackers can exploit another component of the WebUI feature to carry out command injection with elevated (i.e., root) privileges to write the implant to the file system. (This refers to CVE-2023-20273, described in more detail below.)
We observed the threat actor gathering information about the device and conducting preliminary reconnaissance. We also observed the attacker clearing logs and removing users, likely to hide evidence of their activity by using the following commands:
clear logging
no username cisco_support
no username cisco_tac_admin
no username cisco_sys_manager
We assess with a high degree of confidence that these clusters of activity were carried out by the same actor. In October, the actor removed evidence of the “cisco_tac_admin” username—which had been created in September—as part of their cleanup efforts, suggesting the September and October clusters of activity were part of the same operation. The first cluster was possibly the actor’s initial attempt and testing their code, while the October activity seems to show the actor expanding their operation to include establishing persistent access via deployment of the implant.
Initial access and implant delivery
The CVE-2023-20198 vulnerability received the highest Common Vulnerability Scoring System (CVSS) score (10/critical). Successful exploitation allows the attacker to gain access to the device with full administrator privileges. After compromising the device, we observed the adversary exploit a second vulnerability (CVE-2023-20273), which affects another component of the Web UI feature, to install the implant. This allows the attacker to run arbitrary commands with elevated (root) privileges, thereby effectively taking full control of the device. In this particular attack, the actor then used the ability to run arbitrary commands to write the implant to the file system. CVE-2023-20273 has a CVSS score of 7.2 (high). We identified the CVE-2023-20273 activity by leveraging existing Cisco protections.
Implant analysis
The implant is based on the Lua programming language and consists of 29 lines of code that facilitates the arbitrary command execution. The attacker must create an HTTP POST request to the device, which delivers the following three functions (Figure 1):
- The first function is dictated by the “menu” parameter, which must exist and must be non-empty. This returns a string of numbers surrounded by forward-slashes, which we suspect might represent the implant’s version or installation date.
- The second function is dictated by the “logon_hash” parameter, which must be set to “1”. This returns an 18-character hexadecimal string that is hardcoded into the implant.
- The third function is also dictated by the “logon_hash” parameter, which checks to see if the parameter matches a 40-character hexadecimal string that is hardcoded into the implant. A second parameter used here is “common_type”, which must be non-empty, and whose value determines whether the code is executed at the system level or IOS level. If the code is executed at the system level, this parameter must be set to “subsystem”, and if it is executed at the IOS level, the parameter must be “iox”. The IOX commands are executed at privilege level 15.
Figure 1: Implant code
In most instances we have observed of this implant being installed, both the 18-character hexadecimal string in the second function and the 40-character hexadecimal string in the third function are unique, although in some cases, these strings were the same across different devices. This suggests there is a way for the actor to compute the value used in the third function from the value returned by the second function, acting as a form of authentication required for the arbitrary command execution provided in the third function.
Guidance and mitigation
We strongly recommend organizations that may be affected by this activity immediately implement the guidance outlined in Cisco’s Product Security Incident Response Team (PSIRT) advisory.
Organizations should look for unexplained or newly created users on devices as evidence of potentially malicious activity relating to this threat. One method to identify if the implant is present is to run the following command against the device, where the "DEVICEIP” portion is a placeholder for the IP address of the device to check:
curl -k -X POST "https[:]//DEVICEIP/webui/logoutconfirm.html?logon_hash=1"
Note: The above check should use the HTTP scheme if the device is only configured for an insecure web interface.
This will execute a request to the device’s Web UI to see if the implant is present. If the request returns a hexadecimal string, similar to what was outlined in the third function above, the implant is present. We note this will only work as an indication of compromise if the web server was restarted by the actor after the implant was installed.
We also have the following Snort coverage to address this threat:
- 3:50118 - Covers the implant installation (CVE-2023-20273).
- 3:62527 - Covers interaction with the implant, but not code execution.
- 3:62528 - Covers interaction with the implant, but not code execution.
- 3:62529 - Covers the passing of code into the installed implant.
- 3:62541 - Covers exploit attempts for initial access (CVE-2023-20198).
- 3:62542 - Covers exploit attempts for initial access (CVE-2023-20198).
The recommendation that Cisco has provided in its security advisory to disable the HTTP/S server feature on internet-facing systems is consistent with best practices and also guidance the U.S. government has provided in the past on mitigating risk from internet-exposed management interfaces. This is also in line with Cisco’s ongoing work with industry partners as part of the Network Resilience Coalition.
Cisco support centers collaborated with the security team after using methods and procedures to correlate similar indicators in a very small number of cases out of our normal substantial daily case volume.
IOCs
5.149.249[.]74
154.53.56[.]231
154.53.63[.]93
Usernames:
cisco_tac_admin
cisco_support
cisco_sys_manager
In addition to the curl command referenced above, perform the following checks to determine whether a device may have been compromised:
- Check the system logs for the presence of any of the following log messages where “user” could be “cisco_tac_admin”, “cisco_support” or any configured, local user that is unknown to the network administrator:
%SYS-5-CONFIG_P: Configured programmatically by process SEP_webui_wsma_http from console as user on line
%SEC_LOGIN-5-WEBLOGIN_SUCCESS: Login Success [user: user] [Source: source_IP_address] at 03:42:13 UTC Wed Oct 11 2023
Note: The %SYS-5-CONFIG_P message will be present for each instance that a user has accessed the web UI. The indicator to look for is new or unknown usernames present in the message.
- Check the system logs for the following message where filename is an unknown filename that does not correlate with an expected file installation action:
%WEBUI-6-INSTALL_OPERATION_INFO: User: username, Install Operation: ADD filename
Source: Flashpoint
Source Link: https://blog.talosintelligence.com/active-exploitation-of-cisco-ios-xe-software/
