Introduction:

Splunk SOAR (Security, Orchestration, Automation, and Response) is a very useful tool that can super charge your security operations by giving your security team a relatively easy, low code, automation capability that has great integrations with tools you already use, straight out of the box. 

One of the things that makes SOAR a favorite tool of mine, is that it is relatively easy to start learning with little-to-no coding experience required. You can develop playbooks that immediately bring you and your security team value and start making everyone’s lives easier. However, while SOAR is relatively easy to pick up, it does not leave you disappointed when you begin to dive deeper. Once you develop some experience and learn some of SOAR’s quirks and niches; combine that with some Python and creativity, and you are left with a very flexible product that you can mold to take powerful actions that can truly revolutionize your security posture. As someone who loves to be both technical and creative, SOAR has become an awesome playground for me to develop new solutions that ultimately help our clients better reach their security automation goals.

While Splunk SOAR has proven to be a very powerful tool, there is one area that I have found to be lacking, and that is the lack of functionality to sort ingested containers pulled via the “On-Poll” action. In this blog I want to talk about why this is an issue and the custom function I developed to fix it!

As a bonus, I included a GitHub link to the custom function that can be downloaded completely for free to help you solve this problem in your Splunk SOAR environment – found at the end of the blog!

What Is a Container?

Data comes into SOAR as “containers” which essentially are JSON objects that can contain more nested JSON objects (Artifacts). When a container is created in SOAR, two main things happen. 

1. It is assigned a label.
2. Active playbooks (automation) associated with that label, run on the container and can access the data in the container.

Containers contain all the data from the event that generated it. For example, a Splunk alert ingested as a container in SOAR will contain all the data from the alert, again, in a JSON format.

Pictured in the screenshot above is a container in Splunk SOAR titled “This Is A Container” assigned the label “gavris_test”

What Is On-Poll?

You can get data/containers into Splunk SOAR through two methods. You can send it to SOAR from somewhere (pushing) – or SOAR can pull it. The latter uses what is called “On-Poll” because SOAR (or, more specifically, an App in SOAR) is “polling” for updates on a set schedule or interval.

The “Generator” app in SOAR allows you to generate containers via On-Poll to simulate dummy Splunk alerts. This allows you to create playbooks and test on simulated events with ease (we will be using this more in just a little bit!)

The Ideal Scenario & The Problem:

So we established what containers are and the two main things that happen when a container is created in Splunk SOAR – they have a label and run playbooks active on that label. We also established the different methods of creating containers in Splunk SOAR – sending or polling. Great! Before we dive into the problem, I want to explain the ideal scenario.

In this scenario we are sending data from Splunk to SOAR (not polling). Say we created 3 separate alerts in Splunk to detect malicious activity. You would likely be using the Splunk App for SOAR Export to send these alerts to SOAR, where you can define the SOAR instance, container severity, sensitivity and most importantly the container label. 

Remember, the label you assign to the container defines what playbook(s) run on it! If you have different alerts, you ideally want them to have different labels to run different automation, right? That’s because you would likely treat an alert regarding a Phishing email, for example, differently than you would an alert regarding Malware, or an IDS detection – so your playbooks that run on these alerts will also likely be different since they are taking different actions and doing different things!

This is the best-case scenario. You define the labels you want to assign to the alerts when they get created as containers in SOAR, and you’re in business. You can have any different playbooks running on any different labels all day long and SOAR is happily automating your alerts away.

Now, we go to the other side of the coin, our friend, On-Poll. In some cases, your only option to get data into SOAR is to use the On-Poll action in whichever SOAR app you are trying to configure. This is not a bad option, but it is natively limited, and herein lies the root of our problem. Let’s take a closer look at the Ingest Settings page for the Generator app, containing the On-Poll configuration.

Notice the limitation of the On-Poll action; you can only specify ONE label to assign to the containers ingested by this app – in this case, a label called “generated_events.”

When running the Generator app, you can see all the containers being ingested under one label. Now the Generator app is meant to simulate real alerts being ingested as containers; and it does a great job doing so! In fact, it shows that we have several different alert ‘types’ being ingested – in this screenshot we see containers named “Malicious URL Request Attempt”, “ASN Transaction”, and a couple Malware alerts. 

These containers would realistically require different playbooks to run on them, which would require different labels. However, due to the limitation of On-Poll, only 1 label can be defined or configured, which means we are limited only to playbooks active on that label.

For some use cases, this may not be an issue, but you run into some serious limitations if you are ingesting several ‘types’ of containers, using the On-Poll action, that need to run different playbooks!

To elaborate, if I need to run a playbook to enrich my “Malicious URL Request Attempt” alerts and another playbook to detonate malware in a sandbox for my Malware alerts, I must have both of those playbooks active on the “generated_events” label – not ideal (more on this in the next section).

Using the actual Splunk app on SOAR as another example, you can see that it is no different:

If you are resorted to using the On-Poll action to ingest all your Splunk alerts or notables from Splunk Enterprise Security, how do you run different playbooks on all the different alerts you have? 

Thinking even bigger picture here, if you are ingesting emails, tickets, or any other data via On-Poll, how do those run different playbooks depending on the content of those containers?  This was the main issue that I was confronted with and got me thinking about a creative way to solve it!

Some Possible Workarounds:

What are some ways to deal with this dilemma? While brainstorming for a good solution to this problem, I mentally went through a check list and initially thought about these potential workarounds:

The first workaround is to simply have all your active playbooks run on the label your containers are ingested under and have some logic in them to stop the playbook run when it realizes that the automation will not work for the container it is running on.

For instance, you can have a filter block at the top of the playbook. If you want the specific playbook to only continue its run if “Phishing Alert” is in the container title, for example, then any other container that this playbook runs on will result in the playbook stopping very early on in the run. SOAR will then go on to the next playbook if there are multiple playbooks set active on the label.

The problem with this approach is that it does not scale well at all. Using the Splunk alert example again, if you have 5 different alerts/notables being ingested into SOAR as containers, it may take up to 5 playbook runs until it gets to the right playbook. You might get away with this at a small scale, but if you are ingesting 20, 30, 50, or 100 (etc.) different alerts that all need different playbooks to run on them; letting that many playbooks run on 1 container until it gets to the right one is wildly inefficient and will likely cause resource issues in SOAR itself.

The second workaround, if applicable, is to have separate assets or instances of your app polling your different data to different labels. For example, you can set up different instances of the Splunk app in SOAR and adjust your On-Poll query that pulls your notables, to look for notables with a specific name. You can have an asset pull in Phishing alerts, another pulling in Malware alerts, another pulling in IDS alerts, etc. 

But again, this gets very tedious and inefficient at scale, leaving you with possibly dozens of app assets/instances at best, and this may not even be possible with certain apps at worst. For example, if you are On-Polling an email inbox to ingest emails into SOAR, you may not have permissions or be able to make different inboxes or be able to direct specific emails to new inboxes to create new assets for those inboxes in SOAR. This becomes a potential headache to solve this problem outside of SOAR, and it is also not efficient or ideal to do so either.

My Solution – Introducing: The SOARting List

My first thought when initially confronted with this problem, was that we need to solve this in SOAR itself. Since we can’t set different labels when configuring the On-Poll action, maybe I can make a playbook to do it for me. Here were my thoughts:

There needs to be only 1 main playbook running on containers that are ingested under the initial label, using some sort of dictionary or key/value pair to look at the container and determine what label it needs to switch to. This will eliminate the scaling issue and ensure that no matter how many container ‘types’ are ingested under the one label, only 1 playbook will be initially running on the containers. 

My second thought was that this main playbook must do 2 things:

1. • 1. The playbook needs to be able to “identify” the container to know what type of container it is, based on some data in the container; whether that’s the container’s name or some field in one of the artifacts.
   • 2. The playbook needs to change the label of the container based on the container ‘type’; effectively sorting it from its initial label it’s ingested under, to its destination label that will then trigger the actual playbook(s) intended for this container.

Using the Splunk alert example again, this would make my ideal outcome be the following:

If all my alerts come ingested under 1 label called “notables”, my master playbook will run on the container, determine if it is a “Phishing”, “Malware”, or “IDS” alert/container – then switch the label on the container from “notables” to “phishing”, “malware”, or “ids” so that when the label on the container changes, it runs ONLY the intended playbook next on the container. Excellent! 

This is exactly what I created in my custom function appropriately called “The SOARting List”, because it runs in an active playbook on the label configured at On-Poll and sorts your containers to their appropriate labels triggering the desired automation to be run, swiftly and precisely.

The SOARting List – Deep Dive:

As a gift to the Splunk community, I am giving the SOARting List custom function away at the end of this blog to help you fix this issue in your Splunk SOAR environments and hopefully get you more value from your SOAR!

Here is a deep dive on how to use it:

The SOARting List has 2 components to be fully functional – the custom function itself and a custom list you must create in SOAR that will essentially be a dictionary of Key/Value pairs in two columns. 

• 1. Column A, contains a unique string that the SOARting List will look for in the container (your Key).
• 2. Column B, the label you desire the container to switch to (your Value).

Here is what the custom function looks like in a playbook, without it being populated:

There are 3 necessary fields: 

• 1. The name of the custom list containing your dictionary.
• 2. Your “Key Comparator” which is the data path of the field that the unique string in the custom list will compare against to check if it should output the corresponding label.
• 3. A default label that the SOARting List will output if sorting fails. This can be the original label it was ingested under, or a completely new label to trigger any automation you want.

Here is what the custom function looks like populated: 

And here is the “Alert_Sorting” custom list that the custom function is referencing:

As you can see, the custom list in SOAR is split into two columns which get read by the SOARting List as a dictionary (Key/Value pair). It takes the string in Column A and checks it against the string in the data path you define in the “key_comparator” field in the custom function. In this case the container’s name. If the string in the custom list is found IN the container name, it outputs the corresponding label!

I want to emphasize this; the string in the custom list does not need to FULLY match string in the data path, to output the label. It takes the string in Column A of the custom list and checks to see if it is IN the string in the data path.

This is because sometimes there are variables in container names (or artifact fields) that change, but you can still have most of the name be consistent. For example, your “key_comparator” data path might be a container name from an IDS alert that contains different IP addresses; for example, a container titled “IDS Alert from xxx.xxx.xxx.xxx” (some IP address). You can match on the string “IDS Alert” or even “IDS Alert from” omitting the variable portion, to still sort these containers to the same destination label to run the same playbook(s)! 

You can then use the output label of the SOARting List and input that in the community custom function called “container_update” to update the container to its new label.

However, if you want the SOARting List to handle updating the container itself, you can uncomment out line 66 of the Custom Function to allow it to do it for you:

This option makes your sorting playbook look even cleaner by having the SOARting list custom function as your only block in the playbook:

After all is said and done, you set your playbook to be active on whichever label you are ingesting events under and watch your containers sort themselves, on ingest!

Using my previous example from above, I set my SOARting List to be active on the “generated_events” label which sorted my containers to the following:

I also like to add a comment to my containers when they switch labels so I can briefly see, at a glance, that it worked:

You can see here that this container sorted from “generated_events” to “malicious_url” because the SOARting List compared the string “Malicious URL Request” found in the custom list, to the key comparator – the containers name – which was “Malicious URL Request Attempt” and found that the container name included that string. Because the container name included “Malicious URL Request”, it output the new label “malicious_url.” Woo!

Conclusion:

This has been a fun ride in SOAR-land and I hope that you can find value in this custom function to help you with your container sorting needs! If you have any feedback or ideas to make this custom function even better, please feel free to reach out to me at [email protected] 

Keep on SOARing!

SOARting List Github Link: https://github.com/GavriSec/SOARting_List 

You can watch the full video tutorial on our Hurricane Labs youtube channel! https://youtu.be/d6oVcK2XWBY