阅读： 16
告警疲劳现象严重影响了安全运营团队的运营成效：运营专家绝大部分的时间精力消耗在低质量低风险告警及事件的研判过程中，高隐匿性攻击行为的少量关键线索难以被快速召回。造成这一现象的一个关键影响因素在于，大规模收集的日志、告警、事件等数据的质量层次不齐，难以为运营过程提供足够且有效的信息量支撑。为此，观测、记录、分析、优化威胁检测数据的质量，是提升安全运营能效的关键环节。
本文将介绍MITRE公司组织的ATT&CK Evaluation项目的评估方法，分析MITRE在威胁检测能力归类方面的方法论。
一、MITRE ATT&CK Evaluation检测能力归类方法的演化
ATT&CK知识框架的影响力已在业界形成共识。MITRE从2018年以来持续运营基于ATT&CK的检测分析能力开放评估项目[1]，特别是针对企业威胁分析，组织并模拟了四轮攻击模拟评估，包括APT3（2018年）、APT29（2019年）、Carbanak+FIN7（2020年）、Wizard Spider + Sandworm（2021年）。
图1 ATT&CK Evaluation项目
MITRE在指定靶场上进行攻击模拟，参与厂商机基于部署的传感器与分析平台给出分析报告，以评估针对以上评估场景下技战术识别的覆盖情况。尽管参与厂商大概率提供了检测日志、事件、告警与ATT&CK矩阵的技战术映射关系，但是如何在各个厂商之间，对齐各家检测能力的粒度，是影响最终评估效果对比的一个关键因素。MITRE公司对此项工作的定位是：
“Vendors use their own terminology and approaches to detect potential adversary behavior. They provide this information to us in their unique way, and then it is our responsibility to abstract the data using detection categories to talk about the products in similar ways.”
因此，需要一种分类规范，以对厂商的检测能力进行公平的归类。MITRE在每一轮评估项目中提供了“Detection Categories”检测能力归类规范说明。随着评估项目的完善，该检测能力归类方法也在持续演进。以下对几轮的检测能力归类方法进行简介与分析。
1. APT3 Enterprise Evaluation 2018
APT3项目中，MITRE采用的基本检测能力归类策略描述如下：
“These categories are divided into two types: “Main” and “Modifier.” Each detection receives one main category designation, which relates to the amount of context provided to the user. A detection may optionally receive one or more modifier category designations that help describe the detection in more detail.”
即，采用了必选的主分类（Main）和可选的次分类（Modifier）。同时，主分类主要依据检测结果提供的上下文、信息量来划分的。检测能力归类主要包括以下类别，总结如下：
| 主类别 | 次类别 | 含义 | 举例 |
| Main Detection Types | None | The vendor is unable to detect red activity due to capability limitations or other reasons. If data is available that is not directly relevant to the procedure tested, this will be categorized as “None.” In these cases, a vendor may receive a categorization of “None” with additional notes and screenshots about that data.
可概括为：无检测能力，无对应实际攻击行为的可观测数据。 |
No data is collected related to the particular action being performed.
An alert fires at the same time that the red team performs the procedure, but it is not related to the technique under test. The tool allows a user to run forensic analysis of a file. This forensic analysis shows the file’s capabilities but does not directly show that a procedure was performed. |
| Telemetry | The capability produces some minimally processed data that is accessible to an end user and directly indicates that the red team activity occurred after the user performs human analysis. There is no evidence of complex logic or an advanced rule leading to the data output, and no labeling occurs other than simple field labeling. The detection needs to be demonstrably and logically related to the actual procedure performed. Proof of detection could include the view, query, or API search used to access the data and/or the detection output (e.g., table view or process tree).
可概括为：基本对应原始采集日志，且日志中明确包含攻击行为线索。 |
正面例子：
Command-line output is produced that shows a certain command was run on a workstation by a given username. 反面例子： The capability shows all the potential behaviors a malicious file could perform but does not indicate which behaviors actually occur. This is not a Telemetry detection because the detection is not demonstrably related to the procedure being performed. |
|
| Indicator of Compromise | The vendor identifies the red team activity based on known hashes, IP addresses, C2 domain, tool names, tool strings, or module names. Proof of detection could include the rule name, API/query used to access the data, and/or detection output.
可概括为：基于已知IOC匹配的检测结果。 |
The red team C2 IP address is identified as malicious. | |
| Enrichment | The capability captures data (usually data as described above in the “Telemetry Available” category) and then enriches it with additional information such as a rule name, labels, tags, or ATT&CK tactics or techniques that would assist in a user’s analysis of the data beyond what would have been originally presented. The enrichment must be demonstrably and logically related to the ATT&CK technique under test. There is no evidence of complex logic or an advanced rule leading to the data output beyond a simple “if X, tag with Y” condition. Proof of detection could include the view, query, or API search used to access the data and/or the detection output (e.g., table view or process tree).
可概括为：基于原始日志，实现对行为的ATT&CK技战术基本映射和标签富化。 |
正面例子：
A simple rule looking for any execution of foo.exe produces an alert called “Foo Discovery Occurred” with the supporting data “cmd.exe foo”. Data showing “cmd.exe foo” that is tagged with the information “Reconnaissance observed.” Data showing “cmd.exe foo” that is tagged with the information “ATT&CK Technique T9999 Foo Discovery.” 反面例子： Data showing “Process: cmd.exe foo” that is tagged with “Process created” counts as Telemetry rather than Enrichment because “Process created” would already be apparent to an analyst and is not related to the technique under test. |
|
| General Behavior | The capability produces an alert detection for suspicious or potentially malicious behavior based on some type of reported complex logic or rule (beyond a simple “if X, display Y Rule Name,” which would be categorized as Enrichment). The alert must be demonstrably and logically related to the technique under test, and the capability must have a visual designation indicating this is an alert (e.g. an icon indicating an alert, a notification, an impact score, or a similar visual representation). This detection may provide an ATT&CK “tactic”-level description (e.g. Discovery, Execution, etc.) and/or a general description indicating the behavior is anomalous but does not provide specific details on the procedure detected (i.e. a “black box”). (An ATT&CK reference is not necessary for detection to be categorized in this way, though would be noted if included.) Proof of detection could include the rule name or module that performs the detection as well as detection output.
可概括为：通过分析手段实现了对攻击行为的显示告警，该告警是ATT&CK Tactic战术层级的通用行为告警。 |
An alert called “Malicious Discovery” is triggered on a series of discovery techniques. The alert has a score indicating the alert is likely malicious. The alert shows does not identify the specific type of discovery performed.
A “Suspicious File” alert triggered upon initial execution of the executable file. |
|
| Specific Behavior | The capability detects suspicious behavior based on some complex rule or logic and provides an ATT&CK “technique”-level description of the activity (beyond a simple “if X, display Y Rule Name,” which would be categorized as Enrichment). (An ATT&CK reference is not necessary for detection to be categorized in this way, though would be noted if included.) The detection includes additional language and/or explanation to provide more detail beyond a general designation that the behavior is malicious. The alert must be demonstrably and logically related to the = technique under test, and the capability must have a visual designation indicating this is an alert (e.g. an icon indicating an alert, a notification, an impact score, or a similar visual representation). Proof of detection could include rule name or module that performs the detection as well as detection output.
可概括为：通过分析手段实现了对攻击行为的显示告警，该告警是ATT&CK Technique技术层级的特定行为告警。 |
The capability produces an alert named “UAC Bypass.” The alert contains details showing the capability detects a change in process integrity levels over a sequence of related events.
The capability produces an alert named “Credential Dumping” for Mimikatz logonpasswords credential dump. |
|
| Modifier Detection Types | Delayed | The capability does not detect the activity in real-time or near-real-time when the red team executes the action, but subsequent alerts, data, enrichment, or additional processing produce a detection for the activity. The Delayed category is not applied to data based solely on the time it takes for regular detections to appear in the capability, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. Proof of detection must include explanation for the delayed detection and proof that could include other alerts, narrative, queries used to obtain the detection, and/or the detection output.
可概括为：相关检测数据未能及时触发，例如云端检测和MSSP分析等，可见示例。 |
The capability does not detect the activity in real-time or near-real-time when the red team executes the action, but subsequent alerts, data, enrichment, or additional processing produce a detection for the activity. The Delayed category is not applied to data based solely on the time it takes for regular detections to appear in the capability, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. Proof of detection must include explanation for the delayed detection and proof that could include other alerts, narrative, queries used to obtain the detection, and/or the detection output. |
| Tainted | The capability detects the activity based on previously identified suspicious/malicious behavior that is related to or “tainted by” the detection. An example could include previously identifying a process as malicious and marking subsequent events related to that process as malicious, either by direct action/result or other relationship. Proof of detection must include clear visual evidence indicating that the detection uses tainted propagation, including what other activity has led to this detection.
可概括为：基于前序检测结果，通过某种机制传播（如关联）形成的新的检测结果，可见示例。 |
The capability produces a General Behavior alert for “Malicious Process Detected,” and then shows in a tree view that cmd.exe ran ipconfig. The ipconfig Telemetry detection is shown to be tainted due to a line going from the parent General Behavior detection to the child Telemetry detection, so the Telemetry detection would be categorized as both Telemetry and Tainted. | |
| Configuration Change | A detection is made possible by a special configuration change or additional API access that allows data not normally accessible to the end user to become available. This category is applied to detections that are produced because of changes made to the initial configuration (as described by the vendor at the start of the evaluation). Proof of detection should include the output of the detection that results from the change along with notes about the reason for the change, how it is changed, and how an end user could request it.
可概括为：厂商经过检测机制、配置文件中的配置参数修改后，才产生或展现出来的检测结果。 |
Data showing account creation is collected on the backend but not displayed to the end user by default. The vendor changes a backend setting to allow Telemetry on account creation to be displayed in the user interface, so a detection of Telemetry and Configuration Change would be given for the Create Account technique.
The vendor toggles a setting that would display an additional label of “Discovery” when the foo, foo1, and foo2 discovery commands are executed. A detection of Enrichment and Configuration Change would be given (as opposed to a detection of Telemetry that would have been given before the change). A rule or detection logic is created and applied retroactively or is later retested to show functionality that exists in the capability. |
表1 APT3 Enterprise Evaluation 2018检测能力分类
2. APT29 Enterprise Evaluation 2019
整体上，APT29攻击模拟检测评估中，采用的检测能力归类策略与APT3一致，即划分为主、次两种大的类别，其中主分类主要依据检测结果提供的上下文、信息量来划分的。
与APT3不同，相关主类别和次类别进行较大调整，主要是剔除了APT3中的Indicator of Compromise和Enrichment主类别，增加了MSSP主类别，对通用和技战术先关的主类别进行了重构。次类别融合了APT3的一些主类别，调整幅度更大。APT29评估项目的检测能力归类机制主要结构如下：
图2 APT29评估项目的检测能力归类机制
本轮次的划分机制总结如下：
| 主类别 | 次类别 | 含义 | 举例 |
| Main Detection Types | None | No data available that was automatically collected, processed, and was made available within the capability related to the behavior under test. If data is available that is not directly relevant to the procedure tested, this will be categorized as “None.” In these cases, a vendor may receive a categorization of “None” with additional notes and screenshots about that data.
可概括为：无检测能力，无对应实际攻击行为的可观测数据。与APT3基本一致。 |
No data is collected related to the particular action being performed.
An alert fires at the same time that the red team performs the procedure, but it is not related to the technique under test. |
| Telemetry | Minimally processed data collected by the capability showing that event(s) occurred specific to the behavior under test. (i.e. showing the procedure/command that was executed). Evidence must show definitively that behavior occurred and be related to the execution mechanism (did happen vs may have happened). Evidence must be related to what caused the behavior. There is no evidence of complex logic or an advanced rule leading to the data output, and no labeling occurred other than simple field labeling
可概括为：基本对应原始采集日志，且日志中明确包含攻击行为线索。与APT3基本一致。 |
Command-line output is produced that shows a certain command was run on a workstation by a given username. | |
| MSSP | Data is presented from a managed security service provider (MSSP) or monitoring service based on human analysis and indication of an incident occurring. MSSP are inherently delayed due to the manual analysis necessary and will be marked as delayed to remain consistent with other delayed detections.
可概括为：基于远程MSSP的检测结果。相对APT3为新增。 |
An email was received from an analyst describing the context of actions related to data exfiltration. | |
| General | Processed data specifying that malicious/abnormal event(s) occurred, with relation to the behavior under test. (i.e. cmd.exe /c copy cmd.exe sethc.exe, is abnormal/malicious activity or an identifier stating that “suspicious activity occurred”. No or limited details are provided as to why the action was performed (tactic), or details for how the action was performed (technique).
可概括为：一般类型检测数据，如异常行为等，不包含对技战术的详细映射和识别。无APT3的对应类别。 |
An alert describing “cmd.exe /c copy cmd.exe sethc.exe” as abnormal/malicious activity, but not stating it’s related to Accessibility Features or a more specific description of what occurred.
A “Suspicious File” alert triggered upon initial execution of the executable file. An alert stating that “suspicious activity occurred” related to an action but did not provide detail. |
|
| Tactic | Processed data specifying ATT&CK Tactic or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on the potential intent of the activity, or helps answer the question “why this would be done” (i.e. Persistence was set up or there was a sequence of Discovery commands).
可概括为：该数据映射到ATT&CK Tactic并包含了ATT&CK Tactic战术层级的信息，能够给出并解释告警对应行为的“Why”问题。基本对应APT3的General Behavior类别。 |
An alert called “Malicious Discovery” is triggered on a series of discovery techniques. The alert has a score indicating the alert is likely malicious. The alert does not identify the specific type of discovery performed.
An alert describing that persistence occurred but not specifying how persistence was achieved. |
|
| Technique | Processed data specifying ATT&CK Technique or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on how the action was performed, or helps answer the question “what was done” (i.e. Accessibility Features or Credential Dumping).
可概括为：该数据映射到ATT&CK Technique并包含了ATT&CK Technique技术层级的信息，能够给出并解释告警对应行为的“How”问题。基本对应APT3的Specific Behavior类别。 |
An alert called “Credential Dumping” is triggered with enough detail to show what process originated the behavior against lsass.exe and/or provides detail on what type of credential dumping occurred.
An alert for “Lateral Movement with Service Execution” is triggered describing what service launched and what system was targeted. |
|
| Modifier Detection Types | Alert | Data is presented as priority notification to the analyst as an indication of a suspicious or malicious event occurring for further investigation (e.g.: icon, queue, highlight, popup, etc.). Not a modifier of Telemetry.
可概括为：指示检测结果为告警类型。相对APT3把数据粒度更精细化了，主分类只提数据和标签，不区分是否是告警。这更符合ATT&CK行为识别而非攻击识别的中性数据假设。 |
A visual notification occurred in a dashboard and/or alert queue that “Lateral Movement” occurred.
A recognizable identifier populated to a dashboard so that an analyst recognizes that a high severity event may have occurred. |
| Correlated | Data is presented as being descendant of events previously identified as suspicious/malicious based on an alert or high suspicion activity that was brought to the attention of an analyst. Examples of correlation evidence include annotated process trees or tags applied to chains of events showing the relationship between the suspicious/malicious event and data from the technique under test.
可概括为：基于前序检测结果，通过某种机制传播（如关联）形成的新的检测结果，可见示例。基本对应APT3的Tainted类型，语义表达更明确了。 |
A process tree or chain of events is annotated showing the relationship between a net.exe process and a prior alert on “Credential Dumping”.
Telemetry in a dashboard shows a relationship between a process launch for ipconfig.exe and a prior alert on an IOC to show the linage of activity. |
|
| Delayed | Detection (alerts, telemetry, tactic, technique, etc.) is unavailable due to some factor that slows or defers its presentation to the user, for example subsequent or additional processing produce a detection for the activity. The Delayed category is not applied for normal automated data ingestion and routine processing taking minimal time for data to appear to the user, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. The Delayed modifier will always be applied with modifiers describing more detail about the nature of the delay.
Delayed is subdivided into: Manual — Processing was triggered by human action and not initiated automatically. In the case of detections provided by a MSSP, human analysts reviewed and produced the outputs that were later presented to an analyst. Processing – Detection incurred a delay based on additional data processing to apply complex logic to the events where the results were later available to an analyst. 可概括为：相关检测数据未能及时触发和展示，例如云端检测和MSSP分析等。 基本对应APT3的Delayed子类型，增加了分类。 |
The capability’s cloud service component uses machine learning algorithms that trigger a detection on credential dumping hours after the red team performs it. This detection would receive the Main detection category of Technique with a Modifier detection category of Delayed-Processing.
The capability sends data back to an analyst team, who manually analyze the raw data and create an alert called “Malicious Behavior Detected” that appears in the interface three hours after the red team performs the procedure. This detection will receive a Main detection category of MSSP and a Modifier detection category of Delayed-Manual. |
|
| Host Interrogation | Data is manually pulled from an endpoint via the capability for analysis. This category represents possible behavior identified through manual analysis from data that is not automatically ingested and analyzed by the capability to show an analyst that event(s) occurred specific to the behavior under test. Though useful as a capability to some security teams, capturing data through these means may be difficult and/or depend on the skill level of the analyst to derive actionable information. Host Interrogation is a modifier that will only apply to the None category. It will be marked as delayed to remain consistent with other delayed detections.
可概括为：通过非自动化手段直接拉去目标主机数据进行分析，只针对None这一主类型。 相对APT3为新增子类型。 |
There is a remote shell component to the capability that can be used to pull native OS logs from a system suspected of being compromised for further analysis. | |
| Residual Artifact | Data, such as a binary or process memory, that requires additional analysis to determine what capabilities or behaviors may have been used. This category represents possible behavior identified through manual analysis from data that is not automatically ingested and analyzed by the capability to show an analyst that event(s) occurred specific to the behavior under test. The collected data is more a byproduct of adversary actions and less indicative of adversary behavior. Though useful as a capability to some security teams, capturing data through these means may be difficult and/or depend on the skill level of the analyst to derive actionable information. Residual Artifact is a modifier that will only apply to the None category. It will be marked as delayed to remain consistent with other delayed detections.
可概括为：通过非自动化的手段（通过专家）对特定数据进行再分析，以得出进一步的检测结果和信息。只针对None这一主类型。 相对APT3为新增子类型。 |
Process memory of svchost.exe was collected for later analysis because it was identified as a suspicious process. Later strings analysis showed that there may have been a keylogger present on the system.
PowerShell scripts are collected automatically upon execution. Later analysis of a script shows it contains the functionality to capture a screenshot of the user’s desktop. |
|
| Configuration Change | The configuration of the capability was changed since the start of the evaluation. This may be done to show additional data can be collected and/or processed. The Configuration Change modifier may be applied with additional modifiers describing the nature of the change.
Configuration Change is subdivided into: UX – Change was to the user experience and not to the capability’s ability to detect behavior. Changes could include display of a certain type of data that was already collected but not visible to the user. Detection – Change was to the capability’s ability to capture or process information that impacts its ability to detect adversary behavior. Changes could include collecting a new type of information by the sensor or new processing logic that was deployed. 可概括为：厂商经过检测机制、配置文件中的配置参数修改后，才产生或展现出来的检测结果。 基本对应APT3的Configuration Change子类型，增加了分类。 |
Data showing account creation is collected on the backend but not displayed to the end user by default. The vendor changes a backend setting to allow Telemetry on account creation to be displayed in the user interface, so a detection of Telemetry and Configuration Change-UX would be given for the Create Account technique.
The vendor toggles a setting that would display an additional label of “Discovery” when the foo, foo1, and foo2 discovery commands are executed. A detection of Tactic and Configuration Change-Detection would be given (as opposed to a detection of Telemetry that would have been given before the change). A rule or detection logic is created and applied retroactively or is later retested to show functionality that exists in the capability. This would be labeled with a modifier Configuration Change-Detection. |
|
| Innovative | Designation applied to innovative and useful ways to detect a technique under test. Not all techniques will have or can get this designation applied to vendor solutions. It is meant to highlight accurate and robust approaches that bring value and deeper insight to consumers. This modifier will be applied at the Evaluation Team’s discretion and will take into account data collected, method of detection, accuracy of detection, context provided to the end user, and display of information.
可概括为：经评估团队认定的鲁棒、准确的分析方法产生的检测结果。 相对APT3为新增子类型。 |
/ |
表2 APT29 Enterprise Evaluation 2019检测能力分类
3. Carbanak+FIN7 Enterprise Evaluation 2020
本次评估中，基本的检测能力分类策略与前两次是一致的。基于APT29轮次的策略，Carbanak+FIN7项目中对归类方法进行了简化，剔除了MSSP主类型（猜测测试中只测试自动化部分，不涉及MSSP业务接口能力测试），剔除了多种子类型，只保留了Configuration Change和Delayed两个子类型。此外，Carbanak+FIN7增加了防护（Protection）方面的能力归类策略。
评估项目的检测（Detection）能力归类机制主要结构如下：
图3 Carbanak+FIN7评估项目的检测能力归类机制
评估项目的防护能力归类机制主要结构如下：
图4 Carbanak+FIN7评估项目的防护能力归类机制
本轮次的检测能力归类机制总结如下：
| 主类别 | 次类别 | 含义 | 举例 |
| Main Detection Types | Not Applicable | Vendor did not have visibility on the system under test. The vendor must state before the evaluation what systems they did not deploy a sensor on to enable Not Applicable to be in scope for relevant steps.
可概括为：无检测数据采集能力，无法收集相关数据。相对APT29为新增。 |
No sensor was deployed in the Linux systems within the environment to capture command-line activity, which would have been required to satisfy the detection criteria of the technique under test. |
| None | No data was made available within the capability related to the behavior under test that satisfies the assigned detection criteria. There are no modifiers, notes, or screenshots included with a None.
可概括为：无检测能力，无对应实际攻击行为的可观测数据。与APT29基本一致。 |
/ | |
| Telemetry | Minimally processed data collected by the capability showing that event(s) occurred specific to the behavior under test that satisfies the assigned detection criteria. Evidence must show definitively that behavior occurred and be related to the execution mechanism (did happen vs may have happened). This data must be visible natively within the tool and can include data retrieved from the endpoint.
可概括为：基本对应原始采集日志，且日志中明确包含攻击行为线索。与APT29基本一致。 |
Command-line output is produced that shows a certain command was run on a workstation by a given username.
There is a remote shell component within the capability that can be used to pull native OS logs from a system suspected of being compromised for further analysis. |
|
| General | Processed data specifying that malicious/abnormal event(s) occurred, with relation to the behavior under test. No or limited details are provided as to why the action was performed (tactic), or details for how the action was performed (technique).
可概括为：一般类型检测数据，如异常行为等，不包含对技战术的详细映射和识别。与APT29基本一致。 |
A detection describing “cmd.exe /c copy cmd.exe sethc.exe” as abnormal/malicious activity, but not stating it’s related to Accessibility Features or a more specific description of what occurred. A “Suspicious File” detection triggered upon initial execution of the executable file. A detection stating that “suspicious activity occurred” related to an action but did not provide detail regarding the technique under test. |
|
| Tactic | Processed data specifying ATT&CK Tactic or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on the potential intent of the activity or helps answer the question “why this would be done”. To qualify as a detection, there must be more than a label on the event identifying the ATT&CK Tactic, and it must clearly connect a tactic-level description with the technique under-test.
可概括为：该数据映射到ATT&CK Tactic并包含了ATT&CK Tactic战术层级的信息，能够给出并解释告警对应行为的“Why”问题。与APT29基本一致。 |
A detection called “Malicious Discovery” is triggered on a series of discovery techniques. The detection does not identify the specific type of discovery performed.
A detection describing that persistence occurred but not specifying how persistence was achieved. |
|
| Technique | Processed data specifying ATT&CK Technique, Sub-Technique or equivalent level of enrichment to the data collected by the capability. Gives the analyst information on how the action was performed or helps answer the question “what was done” (i.e. Accessibility Features or Credential Dumping). To qualify as a detection, there must be more than a label on the event identifying the ATT&CK Technique ID (TID), and it must clearly connect a technique-level description with the technique under-test.
可概括为：该数据映射到ATT&CK Technique并包含了ATT&CK Technique技术层级的信息，能够给出并解释告警对应行为的“How”问题。与APT29基本一致。 |
A detection called “Credential Dumping” is triggered with enough detail to show what process originated the behavior against lsass.exe and/or provides detail on what type of credential dumping occurred.
A detection for “Lateral Movement with Service Execution” is triggered describing what service launched and what system was targeted. |
|
| Modifier Detection Types | Configuration Change | The configuration of the capability was changed since the start of the evaluation. This may be done to show additional data can be collected and/or processed. The Configuration Change modifier may be applied with additional modifiers describing the nature of the change, to include:
Data Sources – Changes made to collect new information by the sensor. Detection Logic – Changes made to data processing logic. UX – Changes related to the display of data that was already collected but not visible to the user. 可概括为：厂商经过数据源、检测机制、配置文件中的配置参数修改后，才产生或展现出来的检测结果。与APT29基本一致，增加了分类。 |
The sensor is reconfigured to is created to enables the capability to monitor file activity related to data collection. This would be labeled with a modifier for Configuration Change-Data Sources.
A new rule is created, a pre-existing rule enabled, or sensitivities (e.g., blacklists) changed to successfully trigger during a retest. These would be labeled with a modifier Configuration Change-Detection Logic. Data showing account creation is collected on the backend but not displayed to the end user by default. The vendor changes a backend setting to allow Telemetry on account creation to be displayed in the user interface, so a detection of Telemetry and Configuration Change-UX would be given for the Create Account technique. |
| Delayed | The detection is not immediately available to the analyst due to additional processing unavailable due to some factor that slows or defers its presentation to the user, for example subsequent or additional processing produce a detection for the activity. The Delayed category is not applied for normal automated data ingestion and routine processing taking minimal time for data to appear to the user, nor is it applied due to range or connectivity issues that are unrelated to the capability itself. The Delayed modifier will always be applied with modifiers describing more detail about the nature of the delay.
可概括为：相关检测数据未能及时触发和展示，例如云端检测等。 与APT29基本一致，删除了MSSP相关的示例和描述。 |
The capability uses machine learning algorithms that trigger a detection on credential dumping after the normal data ingestion period. This detection would receive a Modifier detection category of Delayed with a description of the additional processing time. |
表3 Carbanak+FIN7 Enterprise Evaluation 2020检测能力分类
本轮次的新增的防护能力归类机制较为简要，总结如下：
| 主类别 | 次类别 | 含义 | 举例 |
| Protection Categories | Not Applicable | Vendor did not deploy protection capabilities on the system under test. The vendor must state before the evaluation what systems they did not deploy a sensor on to enable Not Applicable to be in scope for relevant steps.
可概括为：无防护能力。 |
No sensor was deployed in the Linux systems within the environment to block red team activity. |
| None | The technique under test was not blocked and/or the technique was unsuccessful and there is no evidence provided to the user that the capability blocked the activity.
可概括为：未成功防护。 |
The technique under test was successful.
The technique under test was unsuccessful, but no evidence was displayed within the capability showing that the behavior was explicitly blocked by the tool. |
|
| Blocked | The technique under test was blocked and the user was explicitly informed that the capability blocked the activity.
可概括为：成功阻断攻击行为。 |
A detection was generated for “Potential Malicious Credential Dumping” specifying that the capability detected potential credential access activity and successfully blocked the behavior. | |
| Protection Modifier Categories | User Consent | The technique under test was blocked after confirmation/consent was manually provided by the user.
可概括为：通过人工介入实现攻击行为阻断。 |
A detection was generated for “Potential Malicious Credential Dumping” specifying that the capability detected potential credential access activity and successfully blocked the behavior only after the user accepted a prompt to confirm that the behavior should be blocked. This protection would receive a Modifier protection category of User Consent. |
表4 Carbanak+FIN7 Enterprise Evaluation 2020防护能力分类
4. Wizard Spider + Sandworm Enterprise Evaluation 2021
本次评估中，直接沿用了Carbanak+FIN7评估项目的检测能力归类方法，详见上一节。说明经过前三轮的能力覆盖度评估，MITRE Engenuity评估团队的检测能力归类策略趋于稳定。
二、MITRE ATT&CK Evaluation检测能力归类方法总结
从以上几轮评估中采用的检测能力归类方法来看，可以归纳两个重要的变化方向：
（1）整体上，各个评估轮次中的主分类归类原则保持相对稳定，同时归类方法趋于简约。可以看到APT29评估轮次中，相对于APT3，增加了多项分类子类别。可以猜想MITRE公司与参与评测的各个厂商进行了充分的沟通，以通过足够丰富的标签化方法全方位展现并归类厂商的检测能力。但是很显然，相对后续的两轮的简约归类方法，前两轮的丰富标签大大增加了评测分类的难度，尽管多种子类型标签给最终给参与厂商留了一些评估效果缓冲的余地。Carbanak+FIN7和Wizard Spider + Sandworm两轮评测保持了稳定的归类方法，说明MITRE的评估策略对重点能力更聚焦了，也反映出参与厂商对评估方案更加熟悉，能够在检测方面提供更聚焦的能力。
（2）检测能力归类原则，更加聚焦在ATT&CK矩阵所要求的行为粒度，主分类更关注相对中性的技战术行为观测与识别能力。主分类标签是以检测结果数据提供的上下文信息量来划分的。特别是APT29相对APT3，主分类只说明数据和标签，不再强调并区分是否是告警。是否是告警只在子分类标签中体现。在后两轮评估中，这个告警字标签也被剔除了。此外，最初的IOC主分类也只在第一轮次出现，后面几轮都删除了这一分类。这一点更符合ATT&CK行为粒度，并且是相对中性行为识别而非攻击识别的建模假设。即，ATT&CK是一个行为级别的抽象能力评估框架，尽管框架致力于描述APT等攻击的意图与技术实现，但在框架设计中，原子化的战术、技术类别只提供行为观测的描述，而不提供可供量化的行为威胁等级信息。
三、小节
通过MITRE的ATT&CK能力覆盖评测，我们看到了MITRE在检测能力归类方面的方法论，及其在评测实践中的演化过程。基于数据湖，特别是多源异构的数据湖进行XDR能力建设，需要有一套自成体系的检测能力分类策略，或称为数据分类分级方案。只有对数据进行细粒度、精细化的管理和分诊，才能缓解当前SOC大数据实践中遇到的信息爆炸问题。
当然，MITRE提供的检测能力归类，是面向数据分类与知识库映射的，主要是围绕覆盖率这一指标构建的归类策略。实际上，面向安全运营效率提升的目标，数据湖数据的分类分级分诊体系化方法需要更完备、更精细的方案，所考虑的指标也更加复杂。
更多相关内容，欢迎大家关注本系列后续文章。
参考文献
[1] https://attackevals.mitre-engenuity.org/enterprise
上述情形之外的任何使用形式，均需提前向绿盟科技（010-68438880-5462）申请版权授权。如擅自使用，绿盟科技保留追责权利。同时，如因擅自使用博客内容引发法律纠纷，由使用者自行承担全部法律责任，与绿盟科技无关。