Awalin Sopan

Sophos

Improving Analyst Workflow using Event Clustering (pdf)

A Security Operation Center (SOC) receives thousands of alerts every day. The systems backing up the SOCs often use handwritten custom rules or Indicators of Compromise (IoCs) to detect potential malicious events, generate alerts based on these detections and pas these alerts to the SOC analysts. SOC analysts monitor alerts from various organizations and host machines. Sometimes alerts occurring in the same machines are aggregated over time and near-match alerts are deduplicated to give the analysts more context of the situation. But this simple near-identical deduplication scheme leaves a lot of room for further improvement through more aggressive deduplication and clustering. Similar alerts may have occurred before in the same organization in the past, or similar alerts may occur concurrently in different organizations. The information from the previous or current similar alerts is rarely used to resolve new alerts. Among the millions of potential malicious alerts, only a few thousands are labeled truly malicious by the security analysts. So, if a system can identify the bulk of the false positive alerts by observing their similarity to prior false positive detections, it can filter out the noise of the false positive data from the analyst’s workflow and expedite their process of alert triage.

In this talk, we present an improved analysts’ workflow that utilizes the knowledge from similar alerts across machines. To demonstrate the proposed workflow, we have developed a prototype web-based application illustrating how we can cluster similar alerts and present the cluster-level statistics to the analysts to help them 1. make quick decisions on new alerts based on similar prior alerts, 2. identify patterns and inconsistencies of decisions across analysts, and 3. provide them an option to apply group level decision on new alerts. The system utilizes concepts from supervised and unsupervised learning to cluster similar alert-data into groups and score them based on their probability of maliciousness. Using the tool, analysts can quickly glance though a list of alerts and their related information, check details to get more context if necessary, and make decisions on the alerts with the help of metrics calculated from similar alerts.

Our system uses a nearest-neighbor algorithm to generate clusters of similar alerts from a data warehouse of alerts received by our managed threat response system. This system observes on average 1.5 million total security events and on average 3,500 of these events generates alerts as they are matched with an IOC or signature. Both new unresolved alerts and previously resolved alerts go into the clustering mechanism as input dataset. A new alert gets priority based on its neighboring similar prior alerts that are already resolved by analysts. If the similar prior alerts are all benign or false positives, then the new alert is de-prioritized. This is a work in progress, and we are iterating over system to find the best prioritization scheme. The system also calculates aggregate metrics on the cluster of similar alerts. For example, the total number of alerts in an alert-cluster, the average probability of maliciousness of the alerts in the cluster, the diversity of labels of alerts in the cluster, etc. It also derives the timelines of the detection-events on the alerts present in the cluster. Finally, these priority scores, the aggregate metrics, and timelines are presented in the user-interface (UI). Our motivation behind this work is to reduce analyst workload. For example, when analysts, look at individual alerts, it will take a long time to solve one alert. But if they are presented with a group of similar alerts, along with a user interface that allows them to resolve all of them together, it will reduce the time and workload to a great extent. Here, our assumption is similar alerts should have similar resolution. For example, if we find that 20 new alerts are very similar to each other, and they are similar to 100 old alerts, then these 20 new alerts can be solved in a similar manner to the old 100 ones. Now, we need to distill the knowledge from the prior alerts, and need to show the analysts, why and how the similarity matters to the new alerts. It also shows if there are both malicious and benign alerts are presents in the clusters; in that case the cluster itself contains diverse decisions from the analysts. Analysts can make their decision to each individual alerts, or they have the option to apply the resolution to all the new alerts belonging to the same cluster at the same time. The web-based interface allows the analysts to see the overview of the alerts, filter and sort them based on various criteria, select and see more details, and finally perform their main task, which is to decide whether an alert is malicious and should be escalated, or benign and should be suppressed. The UI prototype uses sample data from alerts generated by potential malicious PowerShell detections collected over 5 months. The labels are generated by our algorithm, but the final action will be taken by the analysts based on their insights, enabled by the aggregate stats presented in the UI. If accepted, the authors would present a demo of the UI with the full workflow.