Do you ever wish you could get more done in less time? One of the promises of automation is not just doing things faster, it’s doing things better, more consistently, and then scaling to become a force multiplier for both yourself and your team. By freeing yourself up to work on more creative or strategic challenges, you can embrace a more interesting, enjoyable, and satisfying time at work. By automating aspects of threat hunting and associated remediation activities, EDR (Endpoint Detection and Response) and productivity platforms such as SentinelOne and Atlassian Jira Service Desk can be wrangled to drive ever more value and productivity across organizations.
The Google SRE handbook refers to toil as work that is overhead, specifically that of manual, repetitive, automatable, and tactical, with no enduring value. This type of work tends to scale linearly and as fast as your service footprint (or the underlying complexity) grows resulting in O(n) or greater amounts of work to be done. Preferentially, a service or services footprint could scale and there would be zero or negligible impact on your team’s workload. This is why automation is so vital for addressing low-value, interrupt-driven, and reactive tasks. It is especially true in the security domain where inconsistencies and alert fatigue can result in serious loss, damage, or even injury. Security teams build on the bedrocks of observability and repeatability while attempting to scale their defenses amidst ever-growing attack surfaces. To quickly find and deal with thousands of nano-sized needles in all manner of giant haystacks, a simple yet powerful and extensible approach is required. So, what’s next?
Let’s automate a simple security Story that happens every day in modern security teams where suspicious traffic to a domain is investigated using an endpoint tool. This time, rather than a singular flow like that of the last blog, Insider Threat Hunting with Datadog and CrowdStrike, we will introduce modularization to our Story to enable its reuse across teams and future workflows. Automation doesn’t have to be difficult and once you have a few “verbs” to work with, you can add your own “nouns”. Fluency is then just a matter of discovery, connecting concepts, and periods of practice.
In the Story above we will use the Tines Storyboard to simply drag-and-drop Actions to construct our high-level desired outcome. You can reorganize, add to, or modify the steps to suit your own needs. This basic Story demonstrates only a few of the Tines’ automation “verbs”, however, the key to abstracting and modularizing stories are the purple “Send To Story” Actions. They allow us to break up and chain multiple different stories and workflows together.
We will construct a simple threat hunting query looking for an IoC (Indicator of Compromise) and issue it to our main Story via a webhook Action. This means the main Story itself becomes modular as you can initiate the workflow from a range of initial sources. We will ask SentinelOne’s Deep Visibility platform to search for events across a specific window of time, looking at our installed Windows fleet to try and find any host or process that made DNS requests to the domain “www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com”. This domain is a high confidence IoC due to its use as a Command and Control (C&C) host for ransomware. We could just as easily look for a SHA256 file hash or destination IP addresses. If we get back any results we format them and move on to quickly raise a Jira Service Desk ticket. This is to ensure remediation is tracked and completed while we also email the responsible team leads (or department heads).
If we dive into the first part of the Story i.e. the SentinelOne Deep Visibility sub-story, we can unwrap how to deal with everything from a background search job (via API) to different vendors pagination approaches by using only a few more automation “verbs”. Let’s take a look at how our first sub-story unfolds:
There are four main parts to this threat hunting sub-story (and we embrace a defensive stance in relation to the API as we go):
we test our connection and create a query in SentinelOne Deep Visibility
we wait for the query status to complete by looping with a delay (on the left-hand side)
once complete, we request the relevant events and deal with any pagination of results
finally, we extract, deduplicate, and summarize the information to return it to the main Story
Once we arrive back at the main Story thread we are careful to ensure we receive valid results and that the sub-story has run to completion. Just as a careful programmer embraces defensive programming, we too embrace defensive automation.
Now let’s take a look at the brief Jira Service Desk sub-story which, depending upon your organization or workflow, could equally be something like Service Now, Freshdesk, or any other ITSM / helpdesk platform with a REST API.
If Jira Service Desk is your authoritative data source for where work gets created, allocated, and tracked, then having the ability to create an audit trail, update records, and initiate tasks in the system is crucial. For security teams, time is of the essence, but just as important is a healthy collaboration with system owners, operations teams, and developers to ensure Action is taken. This means interfacing with the tools and platforms other teams use to be as efficient as possible (rather than bolting on your own case management and expecting other teams to use yours). Here we will create a simple linear sub-story to create a Jira issue, comment on it and return the status.
The above sub-story results in the populated ticket below. We can embellish this sub-story over time to use additional Tines widgets to prompt for manual authorizations to spark further automation workflows. As we’re dealing with servers, we require more information before arbitrarily quarantining a host or blocking primary processes (but it’s obvious we already have a problem when someone or something is using browsers on servers to access malicious domains!).
We now have an almost complete parent Story run. It just remains to let the team leads (or managers) know that there is a ticket of the highest priority that requires an immediate assignee and requisite action.
We will include the Jira ticket key and direct link in the email for ease of use and to expedite clear communication and action. We could just as easily replace this email step with HTTP calls or dedicated sub-stories targeting other platforms or services depending upon the team’s preferences.
Automating toil has many benefits from functional to cultural, but it helps if you already have a plan or playbook where a process or workflow can be quickly mapped to automated steps. But rather than trying to automate your whole process from the beginning (especially if it’s not well understood or hasn’t been codified fully yet), the next best thing is to automate parts of your process (usually the most tedious!). By modularizing your stories early and starting with the low-hanging fruit you begin to free yourself up from being in a perpetual state of reactivity. You become more able to adopt a proactive stance, one that delivers more value, faster and better than before. You can download the Stories above: main Story, Sentinel One sub-Story, Jira sub-story.