Counting the seconds: a method of showing automation value

Written by John TucknerHead of Research at Tines Labs, Tines

Published on April 24, 2023

This article was posted more than 18 months ago.

Automation has revolutionized the way cybersecurity functions. Not only has it led to significant time savings, but it has also improved the consistency and accuracy of various processes. Here, we will discuss how to effectively record the time saved from automation to demonstrate its value. Tines offers great utilities to easily record the time an analyst has saved by automating manual, repetitive tasks over the course of a day, right down to the second.

What is time saved? 

Tines offers the feature of tracking the time saved for each individual Action. To do this, click on an Action and navigate to the Status tab. There, you will find the option to set "Time saved" by specifying the duration the activity would typically take. This feature is most useful when applied to Actions that involve external services or user feedback, such as HTTP Request Actions, or complex Send to Story or Group Actions containing multiple modular steps.

As the time saved accumulates, the reporting dashboard populates with graphs that illustrate the impact of automation over time. You can apply filters for specific teams, time periods, and particular Stories to gain a deeper understanding of your workflows' performance. In addition to the time-saved data, the dashboard also includes sections that review the most active Stories and Actions.

Beginning with time saved 

Where should you begin when recording time saved? Start by using the Reporting dashboard mentioned earlier, and check the "Most Active Stories" section to identify the workflows that run most frequently. Prioritize these Stories first when assigning time saved, as they will be used the most.

Often, a highly active Story contains a set of frequently used Actions, which can also be found in the Reporting dashboard. These Actions can serve as an excellent starting point when assigning time saved. However, keep in mind that not all Actions carry equal weight. For example, a Webhook that receives thousands of events and quickly filters them out might be useful for your team, but it might not present a strong value proposition. Instead, focus on assigning time saved to significant and time-consuming Actions, such as HTTP requests that "Run a search in Elastic" or "Analyze a file in VirusTotal." As time accumulates through these Actions, the impact of automation on tasks that were once done manually becomes evident.

Another excellent opportunity to record time saved in Tines is within Stories designed for use with Send to Story and Group Actions. Similar to individual requests for time-intensive services like malware sandboxes or SIEM searches, these modular workflows can include various Actions and be used across multiple Stories (Send to Story) or easily duplicated (Group). Setting time saved within these components can exponentially increase the value of those modular workflows based on how many other Stories utilize them.

Determining how long an activity would take can be challenging, but don't let that discourage you. Even a basic and small estimate of a task's duration can be helpful in understanding how automation is helping you. Consider the following aspects when estimating time saved:

  • Can the task vary in duration?

  • If the task takes different amounts of time, is multitasking likely while waiting? Could that increase the time required to complete the task manually?

  • How is someone notified when the task is complete, if at all?

When answering these questions, try to be somewhat conservative in your estimate. For example, if a task usually takes two minutes but occasionally takes 10 without a completion notification, it might be better to estimate a time savings of four minutes.

When analyzing a full Story workflow for time saved, reviewing recent Story runs for the duration it took to complete the Story through automation can be helpful. The minimum amount of manual time the workflow could conceivably take can be determined by comparing the start and end times of the Story.

Understanding the value 

With a dashboard now displaying the time saved, the next step is to understand the data collected. First, verify whether the amount of time saved makes reasonable sense for the Actions you've configured. If you find that you've saved an unrealistic amount of manual work effort, such as hundreds of years after just one day, it's likely a good idea to reconfigure some Actions. This could occur due to unintended loops or test events, so be careful!

A significantly large amount of time saved is not necessarily a negative thing, but it should be scrutinized carefully. If the time saved seems reasonable, it might indicate that there's too much work for humans to handle, making automation crucial. Regardless of automation, it may be worthwhile to examine the processes created to eliminate any low-impact activities. In fact, automating a low-value process could be even worse than performing it manually!

An unusually high amount of time saved could also suggest that specific activities or processes were being skipped before automation. While time saved is an important aspect of understanding the value of automation, the consistency and accuracy of activities carried out can be equally significant.

Effectively showcasing the time saved through automation can help your organization recognize its value and encourage its continued adoption for future success.

To get started reporting on time saved, try any of our Stories tagged with ReportingEnabled in our Story Library.