Post-incident Review and the Big Data Problem

Across the board, security teams of every industry, organization size, and maturity level share at least one goal: they need to manage risk. Managing risk is not the same as solving the problem of cybersecurity once and for all, because there is simply no way to solve the problem once and for all. Attackers are constantly adapting, developing new and advanced attacks, and discovering new vulnerabilities. Security teams that have accepted the post-breach mindset understand that cybersecurity is an ongoing chess match with no end. They focus on reducing risk as much as possible through visibility and automation, instead of searching for a one-size-fits-all solution.

Incident response plays a key role in effectively reducing risk. In a breach, the average cost per lost or stolen record is $148, and having an incident response team reduces this cost by almost 10%. Because of the human component of critical thinking that goes hand-in-hand with response and resolution, incident response is not something you can totally automate. But that doesn’t change the fact that it is something organizations absolutely need in the event of a breach. Despite this, 77% of IT professionals say their organization does not have a formal cybersecurity incident response plan. Instead, organizations respond in an ad-hoc fashion to threats without digging for the root cause of the incident and resolving it. Incident response is an under-utilized asset that has organizational and defensive, immediate and long-term benefits.

An incident response team is accountable for having a plan to handle an incident and implementing it. They’re prepared to mitigate damage, identify the root cause of an incident, and communicate with the proper channels. But they are also responsible for another crucial part of incident response: the post-incident review.

Post-incident review is about identifying every aspect of an incident down to its true root cause. It answers critical questions like what happened before, during, and after the attack. By answering these questions, organizations can ensure the same attack doesn’t happen twice. They review the attack, and identify and close all gaps in their defense that the attacker leveraged.

However, this leaves post-incident review with a major problem.

It takes organizations an average of 191 days to identify a data breach. For a post-incident review that does its due diligence, this means potentially going all the way back in time through at least 191 days’ worth of data to find the root cause of the attack. Consider all of the data in your environment that has come and gone over the course of 191 days. How many investigations have your analysts performed in that time?

To put this into perspective, 27% of IT professionals have reported receiving more than one million threats daily. 55% reported more than ten thousand. A Fortune 500 enterprise’s infrastructure can generate 10 terabytes of plain text log data per month; imagine trying to sort through over six months of logs when evaluating an incident.

Post-incident review is a big data problem that requires a big data solution. Incident response teams need to be able to easily query months’ worth of data, but until now the industry just hadn’t reached the point where that was an option. Network forensics is limited to two to three weeks of raw data, while log management solutions are difficult to access and correlate across large data sets.

As an industry, we need to be incorporating the principles of big data into our security solutions. Legacy solutions do not address dwell time or the need to see every phase of the attack properly correlated, which means security analysts miss out on important data. The bottom line: if you’re a security company, you need to be using data science principles in your development. If you are in a SOC, you need to be thinking not only about incident response, but also about how your tools handle massive amounts of incoming data and make it easily consumable.