In Part 1, we introduced Detection-as-Code, covering its core concepts and benefits and the Detection Development Life Cycle (DDLC) essential for modern threat detection practices.

In the second part of the Practicing Detection-as-Code series, we will cover some basic elements of designing a repository to develop, store, and deploy detections from. We’ll go through several different aspects of the setup like the Git platform, branch strategy, repository structure, detections structure, taxonomies, and content packs. These are core elements that we will inevitably encounter when designing our repository and it is important to put some thought into them early, so that we avoid unnecessary refactoring down the road and ensure our work is as organized and structured as possible.

However it is important to note beforehand that the principles, methodologies, and design elements are intended to be used as a blueprint that can be adjusted to meet the individual needs of your team. What may work for a larger team focused exclusively on detection engineering may not fit a smaller one that may have mixed roles and responsibilities. Your ultimate goal should be to adapt these elements to your internal processes, to provide structure and improve efficiency in your detection engineering day-to-day tasks. Be cautious of over-engineering, as it can drain your resources without adding significant value.

Selecting the Git Platform

The first step in implementing a Detection-as-Code approach is selecting the Git platform. You can choose GitHub, Gitlab, AWS CodeCommit, Azure DevOps or another platform, depending on your organization’s preferred technology stack. For the purposes of this blog series, we will provide examples from a repository hosted on Azure DevOps.

Standardizing Detection Structure

The next step is to standardize the detections stored in the repository. In the context of Detection-as-Code, standardization refers to enforcing uniform structures and formats across the detections library. Adhering to a predefined set of conventions will help us maintain and search the detection library with less effort.

Using YAML or JSON to store your detections is a battle-tested approach. These data-serialization formats offer the advantage of being structured, human-readable, and integrating well with CI/CD. Furthermore, many platforms support exporting rules directly in JSON, YAML, or even XML format.

The metadata fields available for detection rules vary depending on the platform. Some platforms will allow you to store tags and notes while others are limited to only a title and description field. If you are also supporting multiple platforms, this inconsistency of available fields could be problematic. In that case, maintaining a dedicated metadata file alongside the detection could be a beneficial approach. For inspiration on what kind of information you should include in your metadata file, so that the detection is sufficiently documented, you can consult the Sigma Specification [1], which is a vendor agnostic detection format, or Palantir’s Alerting and Detection Strategy Framework [2]. You can either adopt and extend one of those or create your own. Whichever your choose, we recommend to include at least the following fields:

id:                     # A unique identifier for the detection.
title:                  # A brief title for the detection.
description:            # A comprehensive description of the detection.
level:                  # Severity of the detection (low, medium, high).
version:                # The version of the detection that we are on.
references:             # References to the source(s) that the detection was derived from e.g. blogs, papers, presentations, tweets.
  -
data_sources:           # The data source on which the detection relies.
  - category:           # The category of the log source e.g. dns, network, os.
    vendor:             # The vendor e.g. cisco, microsoft.
    product:            # The product e.g. asa, windows.
    service:            # The service e.g. traffic_logs, security_event, antivirus.
    event_id:           # The event ID used e.g. ASA-3-201008, 4688
blindspots:             # Recognized shortcomings of the detection.
  -
known_false_positives:  # A list of known false positives that may occur.
  -
investigation_steps:    # Investigation steps that the analyst can follow to investigate an alert generated by this detection.
  - 
tags:                   # Taxonomies or notable things about the detection.
  - tactic.$tactic      # MITRE ATT&CK tactics.
  - technique.tXXXX.XXX # MITRE ATT&CK (sub)techniques.
  - group.gxxx          # MITRE ATT&CK adversary group name.
  - software.sxxx       # MITRE ATT&CK software name.
  - car.XXXX-XX-XXX     # MITRE Cyber Analytics IDs.
  - cve.xxxx-xxxxxx     # CVE IDs.
  - notable.$entity     # Notables about the detection e.g. name of lolbin.

An example would be:

Utilizing Taxonomies

Taxonomies provide a structured way to classify and organize detections into logical groups. This enhances the searchability of the detection library and provides context to the goal and scope of each detection by mapping it to known frameworks such as MITRE ATT&CK or CAR. Some examples of taxonomies that can be used are:

• Categories: application, cloud, dns, network, ids, os, proxy, …
• MITRE ATT&CK: tactic.defense_evasion, technique.t1548, group.g1046, software.s002, …
• MITRE CARs: car.2013-01-002, …
• CVEs: cve.2021-44228, …
• Notables: notable.rundll32, notable.regsvr32, notable.registry_run, notable.registry_run_once …

In our example above taxonomies are used either as tags or as part of the data_sources fields.

Leveraging Content Packs

Content packs are collections of related detection rules bundled together. They help the detection engineering team maintain and deploy a set of detection capabilities for specific threats or technologies to the appropriate environment. Content packs are particularly useful for MSSPs, as different detection capabilities are required based on the technologies available in each environment or the SLA between the provider and the customer. An in-house SOC can also benefit from using content packs when greater control is needed over the deployment of new detections or updates to existing ones.

A content pack may include a title, a description of the focus of the content pack, a version number and the relative paths to the detections in our detection library which we will describe in the next section.

{
    "name": "Detections for Azure",
    "description": "A collection of detections for Azure",
    "version": "1.0.0",
    "detections": [
        "cloud/azure/azure_ad_azure_discovery_using_offensive_tools",
        "cloud/azure/azure_ad_mfa_denied_phone_app_reported_fraud",
        ...
    ]
}

Structuring the Repository

The structure of the repository is important, as it provides consistency and simplifies navigation within your detection library. The Sigma repository [3] is an excellent example of structured organization, which you can adopt directly or adapt and extend it to better fit your needs. Regardless, we are going to describe what that structure may look like in our case.

detections/
│   ├── cloud/
│   ├── dns/
│   ├── endpoint/
│   ├── ids/
│   ├── os/
│   ├── network/
│   ├── proxy/
│   └── vpn/

Retired detections, meaning detections that have been taken out of commission and should not be deployed in the monitored environment, should be stored in their own directory as well (e.g. _retired). Keeping retired detections in a separate directory, rather than deleting them, is encouraged, even though it might seem counterintuitive to Git’s intended use and best practices, because understanding what hasn’t worked can be just as valuable as knowing what does, both for current team members and those who join in the future.

detections/
│   ├── _retired/
│   │   └── ... (detections that are out of commission)

Each detection rule should be stored as a separate file, using a consistent naming convention to facilitate tracking and version control. If you are maintaining detection logic across multiple platforms, it is helpful to group them under a parent directory within detections/ as follows: /network/vendor_agnostic/<detection_1>/<detection_1>_sentinel.json,
/network/vendor_agnostic/<detection_1>/<detection_1>_elastic.json.

detections/
│   ├── network/
│   │   └── vendor_agnostic/
│   │       └── detection_1/
│   │           ├── detection_1_meta.yml
│   │           ├── detection_1_sentinel.json
│   │           ├── detection_1_elastic.json

content_packs/
│   ├── content_pack_1.json
│   ├── content_pack_2.json
│   └── ... (grouped detection bundles for delivery)
detections/
│   ├── _retired/
│   │   └── ... (detections that are out of commission)
│   ├── application/
│   ├── cloud/
│   ├── dns/
│   ├── endpoint/
│   ├── ids/
│   ├── os/
│   │   ├── windows/
│   │   │   └── ... (detections for windows)
│   │   └── linux/
|   |   |   └── ... (detections for linux)
│   ├── network/
│   │   ├── palo_alto/
│   │   └── vendor_agnostic/
│   │       └── detection_1/
│   │           ├── detection_1_meta.yml
│   │           ├── detection_1_sentinel.json
│   │           ├── detection_1_elastic.json
│   ├── proxy/
│   └── vpn/
parsers/
│   └── ... (parser components)
filters/
│   └── ... (filter components)
pipelines/
│   ├── scripts/
│   │   └── ... (scripts for pipelines)
|   ├── schemas/
|   |   └── ... (schemas for detections)
│   ├── pipeline1.yaml
|   └── pipeline2.yaml
tests/
│   └── ... (validation tests for detections)
README.md

Choosing a Branch Strategy

Once we have selected the repository platform and designed the structure and format of the detections, the last step is to decide on your branch strategy. In software engineering, the branch strategy is crucial for effective collaboration and version control, allowing team members to work on different tasks simultaneously without conflicts. It separates code that is under development from stable code and ensures that only tested and reviewed code will reach the production. Those same principles are beneficial and can be applied when we practice Detection-as-Code.

In this case the branch strategy involves:

• When and why you create branches – Will you create a new branch for each detection you develop or for each bug you need to fix?
• Branch naming conventions – What patterns or prefixes will you use when creating branches, such as ‘dev-‘, ‘detection-‘, ‘bugfix-‘, or ticket numbers?
• How and when to merge branches – When will you merge your detections back into the main branch? After each detection development? After passing QA?
• Branch mapping to your environment – How will different branches correspond to different deployment environments in your organization? Will you have dedicated testing or development environments?”

Several models exist, that you can choose from, based on factors such as, team size, experience level of team members, dedication to the role, available resources, deployment needs, development pacing and type of your organization. There is no definitive right and wrong choice – you should select the strategy that best suits your needs. Each one however comes with its own pros and cons and may be more suitable for certain teams than others.

At NVISO we have been using GitHub flow with great success, but we’ll talk a bit about other available options so you can make a choice that best fits to the development style and needs of your team.

Trunk-Based

Trunk-based [4][5] development is a strategy where engineers commit directly to the main branch, often referred to as ‘trunk,’ or merge smaller, short-lived branches more frequently (e.g. multiple times per day). The branches are almost always the product of one person. The release can either happen from the main branch or from a short-lived release branch that is not merged into the main branch. Branch naming conventions should be used and typically include prefixes such as ‘detection-‘ or ‘bug-‘.

Trunk-based development is simple and low in complexity, making it easy for teams to adopt. It enables fast delivery of detections and minimizes merge conflicts through frequent commits to the main branch. However, it requires strong validation and testing pipelines to ensure quality, as the direct commit approach can increase the risk of production errors. This model is best suited for small or experienced teams and especially when robust automated testing and CI/CD pipelines are in place.

GitHub Flow

GitHub Flow [6][7] is a strategy focused on continuous delivery. The main branch serves as the central point for the latest, stable detection code and it should always be deployable. Branches can have a long lifespan (living for weeks) and a branch should be created for every change. The branches should be named after the task being worked on like “detection/brute-force-rules” or “bug-fix/threshold-tuning”. Pull requests are reviewed by peers and then merged into main while automated validation pipelines should also be used. Deployment can occur automatically upon merging the changes into the main branch.

This approach offers simplicity and low complexity, making it easy to adopt and ideal for fast delivery of detections in CI/CD environments. However, it requires effort to set up validation pipelines and lacks dedicated staging or testing environments, increasing the need for thorough reviews. It’s best suited for small, experienced teams with strong automated testing and CI/CD practices operating in fast-paced development cycles.

Although Trunk-Based and GitHub Flow might seem a little bit to similar at first glance, they have some differences [8]. Branching (short-lived vs long-lived) and merge frequency (frequent vs longer) are the core differentiators, which also makes them suitable for different development styles.

Gitflow

Gitflow [9] is one of the most structured branching strategies that utilizes multiple branches like main, hotfix, release, dev and feature. Branch naming conventions must be used for this one for consistency, otherwise it can become very easy to lose track of all the changes. With regards to detection engineering this is how you would adapt it:

This strategy provides a clear and structured workflow for development, testing, and release, ensuring stability by separating production and development code and allowing changes to be grouped into planned releases. It’s mostly suited for content delivered by vendors, where reliability is key. Its complexity can be challenging for less experienced users, and managing multiple branches can slow detection delivery, making it less suited for fast-paced CI/CD environments.

Environment Branching

Environment branching [10] is an option for teams with a dedicated staging environment where mirrored production or test data is available. This approach is often chosen because it feels more natural in that case and provides a safe way to test detections using real production data. While there are various adaptations, it typically includes dev, staging, and main branches. New features, or in our case detections are developed in branches derived from dev.

This approach is simple to adopt and provides a clear separation between development, staging, and production environments, ensuring thorough testing before deployment. It’s well-suited for organizations with strict procedures on alert handling and teams with dedicated staging environments. However, it is also prone to branch drifting and complex merges over time, which increase maintenance effort and slow down the release cycle.

Custom

As mentioned earlier, there is no right and wrong choice. Select a branch strategy that fits the needs of your team. You can define your own branch strategy, according to your processes, your way of working, your teams experience and ability to collaborate and segregate tasks.

Wrapping Up

In this second part of the Practicing Detection-as-Code series we discussed about designing a repository for detections and went through different aspects like, selecting a Git platform, suggesting a repository structure, standardizing the detection format, introducing the concept of the content packs and deciding on a branch strategy.

The next blog of this series will introduce the usage of CI workflows to validate the detections and repository structure.

References

About the Author

Stamatis Chatzimangou

Stamatis is a member of the Threat Detection Engineering team at NVISO’s CSIRT & SOC and is mainly involved in Use Case research and development.