Continuously improve your SOC through the analysis of security metrics. 

Introduction

Metrics are quantifiable measures and assessment results. They empower organizations to describe and measure controls and processes, and make rational decisions driven by data for improved performance. They provide knowledge regarding how well an organization is performing and can help uncover insufficient performance and their causes, or ensure that policies are being adhered to. A measurement program can provide structure to collecting measurements and apply decision making to the assessment of measurements. There are considerations when selecting metrics to collect. Effectiveness and efficiency measures for incident response may prove beneficial to improving a Security Operations Center (SOC). Hurricane Labs can help make your metric journey easier.

1. Information Security Measurement Program Implementation

Implementing an information security measurement program can make utilizing metrics easy, structured, and repeatable. The broad steps into implementing a program include the following:

1. Evaluation of the organization’s existing security maturity
2. Identification and evaluation of measures
3. Data collection and analysis
4. Identification of applicable actions
5. Application of actions

1. Choosing metrics

There are some careful considerations to keep in mind when choosing metrics. The metrics an organization selects should depend on their needs. Think about “motivation”. Organizations should think about what they require and what would benefit them most. This may depend on their vertical market, compliance requirements, or security program maturity. Gathering stakeholder input may help make this more clear. Metrics should fulfill your needs and benefit your organization. 

An organization should set a target or goal for what they want to achieve. Organizations should evaluate where they are currently with a security control or process, and where they want to be in the future. This can help progress your security maturity and enhance effectiveness.

The scope of the metrics defines the bounds of what is being measured. Organizations may want to limit their measurements to focus on specific areas that are relevant to their needs. Unnecessary measurements may lead to confusion or wasted effort.

Metrics depend on data being available. Organizations should evaluate potential metrics to determine if the data is available, or if it would be feasible to make the data available. Doing this evaluation early in the selection process will help with assessments later on.

Experimenting with relative metrics and models with reference to time is the great way to identify effective performance indicators.

1. Metric attributes

High-performing and high-value metrics commonly can be described with certain attributes. Metrics should be relevant, precise, consistent, simple, and actionable. Relevant metrics take the organization’s goals into account. This ensures that the metrics can be utilized to help fulfill your organization’s goals. Metrics should be precise, meaning that they are numerical and free of bias. This can help make decision making easier. Consistent metrics ensures  trustworthiness and allows for long-term trend analysis. Metrics should be simple and easily understandable. Lastly, metrics should be actionable and drive assessments and changes. Additionally, NIST (National Institute of Standards and Technologies) provides a great list of characteristics and common fields of metrics in NIST SP 800-55v1 if more detail is needed.

1.  Incident Measures and Metrics:

Effectiveness measures ensure processes and controls are implemented and operating successfully as designed. Efficiency measures refer to the speed with which processes and controls operate. There are some common effectiveness and efficiency measures and metrics used in incident response. These include the following:

• Dwell time
• Mean Time to Detect (MTTD)
• Mean Time to Acknowledge (MTTA)
• Mean Time to Recovery (MTTR)
• Classification of true positives, false positives, true negatives, and false negatives
• Performance of a service level agreements (SLA)

As an example, if you notice that some service is consistently violating a SLA, then it would warrant some action to rectify it.

1.  Splunk and HL application

Splunk has some measures that can be collected from their case and incident management systems. For example, Splunk has some options for disposition classifications for findings (formerly notables) within their Mission Control dashboard (formerly Incident Review), which is a part of their Enterprise Security Splunk app. These disposition options include:

This information is queryable within Splunk with the `notable` macro or via the lookup incident_review_lookup. Splunk dashboards or saved searches can be made around these measures. 

Hurricane Labs similarly utilizes close codes with our alerts. Our analysts submit these closure codes after their investigations. An example of how we utilize this information is that we have some internal search logic that queries over the information and brings tuning opportunities to our attention. This encourages continuous improvement of our deployed rules. Our analyst closure codes include:

Hurricane Labs also provides customers with a metrics app that contains executive dashboards that help show the value our SOC services provide.

Conclusion

Metrics are an invaluable tool that can greatly benefit any organization. Hurricane Labs uses metrics for continuous improvement. NIST provides some great guidelines regarding the selection of metrics and the implementation of a program for them in NIST SP 800-55v1 and SP 800-55v2. I cannot understate the thoroughness and value of these publications. I am happy to have gotten the opportunity to review and share this information with you all.

References:

https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-55v1.pdf 

https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-55v2.pdf