Post-Incident Review

Examining the importance of post-incident review for security teams

WHAT IS POST-INCIDENT REVIEW?

Post-incident review is a detailed retrospective that allows an enterprise to carefully understand each part of an incident, from start to finish. It is one step in the incident response process that requires a cross-functional effort from all individuals and technologies connected to the incident to truly understand the root cause and full scope of the attack.

Use a post-incident review to assess all the processes and people that were impacted by the attack so that it never happens again.

See Weaknesses

By completing a post-incident review, security teams are able to uncover network vulnerabilities and weaknesses that have been compromised by a threat actor. Further, they should identify all IOCs and TTPs associated with the attack, as well as internal issues that may have been manipulated by the threat actor to gain entry.

What does good look like?

A good post-incident review results in a list of practical actions that address each of the issues that allowed the threat actor to succeed. These actions should minimize the impact of an attack and teach the security team, the security tools, and the wider enterprise how to prevent, detect, and respond to a similar attack in the future.

“Just like architecture reviews in the R&D world or debriefs and after-action reports in the military world, we too need a process for improvement in incident management, response, containment and remediation.”

– Sam Curry, CSO

 

Read His Column in Forbes →

GOALS OF A POST-INCIDENT REVIEW

remediate-01
Find the Root Cause

Address the problem from the very beginning, not just the end.

damage-01-01
Scope the Damage

Iteratively assess the complete picture of damage to prevent future incidents.

resilience-01
Improve Resilience

Mature your security program by investigating attacks from start to finish.

WHY IS POST-INCIDENT REVIEW IMPORTANT?

Many Hollywood movies conclude with a “replay” in the last third of the film: going back in time, adding details not shown during the movie, and pointing out the ones a viewer may have missed. These details fill the holes in the plot and resolve the who, why, and how - the stealthy information viewers try to identify for the first two thirds of the movie. These make up the hook, the point, and the solution for the complexity of the plot.

For Security

In a similar way, a cybersecurity incident is a whole story, with its own structure, characters, methods, complexity, and unnoticed details. However, many defenders tend to be satisfied with the rapid remediation of an attack before they move on immediately to the next threat.

With commoditized attacks, this approach is enough.

But with more sophisticated attacks, especially low and slow or targeted attacks, the initial incident is just a battle in the bigger war.

- Tweet this quote.

Low and Slow Attacks

With low and slow attacks, the attacker’s progress across the kill chain may be scattered across a long timeline involving different network segments or third-party channels to the network. Attackers may use different tools and techniques in each part of the attack, access the data they want, and then remove any trace of activity. These attacks pass under the radar of many security products where data retention time is short and event correlation over time is weak.

SEE THE FULL SCOPE OF THE ATTACK

low-slow-graph

In these instances, defenders are unable to get the full picture of an attack and are only able to remediate the final, most recent stages. This leaves exposed areas for attackers to target.

The Nation State

For example, nation-state threat actors may begin their attack with a commodity cyber attack framework. They scan the perimeter for vulnerabilities, gain access to a specific endpoint or server, steal local admin credentials, and wait. They wait to use the credentials weeks later to regain access to the network. Once in, they use different tools and techniques to accomplish their main attack objectives. In this case, the attack is only detected in its last stages. If the retention time of the SIEM or EDR is smaller than the length of the attack, the defender is going to miss the initial compromise in their review. They will only respond to the final stages and be totally oblivious to the bigger picture. This leaves the network exposed to those same exploits and entry points the attacker used at stage 1.

Learn More →

Advantages of Post-incident Review

  • Improve continuously to face the nature of evolving attacks and TTPs.
  • Don't fight cyber alone. Work as part of the community to help make the unknown known for the industry.
  • Cultivate your security program and reevaluate what you're protecting and how you're protecting it.
  • Develop your incident response procedures.
  • Reduce risk by going back in time to prevent a recurring attack by the same method.

PAINT THE COMPLETE PICTURE

Low and slow attacks easily bypass traditional security defenses by incremental actions that on their own are too small to detect, but put together can devastate an organization.

The majority of organizations lack the in-depth visibility to identify the entire lifecycle of a sophisticated attack like this. Answering the question of, "when did this attack begin?" is increasingly difficult.

A single, low-priority alert in an ocean of incidents is able to operate below the radar, and may even look like legitimate activity.

- Tweet this quote.

These attacks are only becoming more common, especially among nation state threat actors that want detailed information on an enterprise or government.

Read About Low and Slow Attacks From DarkTrace →

197 Days

Attackers spend an average of 197 days of dwell time in your network before being detected.

Read the Ponemon Study →

By understanding their behavior from start to finish.

image (11)

OPERATION SOFT CELL

In 2018, the Cybereason Nocturnus team identified an advanced, persistent attack targeting global telecommunications providers carried out by a threat actor using tools and techniques commonly associated with the Chinese-affiliated threat actor APT10. This multi-wave attack focused on obtaining data of specific, high-value targets and resulted in a complete takeover of the network.

 

Read About the Attack →

HOW TO PERFORM A POST-INCIDENT REVIEW

Follow Post-incident Review Best Practices

STEP 1: State the Goal

Goals for a post-incident review should cover four tiers and revolve around learning and improving.

  1. Learn how to detect and respond to similar attacks, then improve the existing defense.

  2. Learn how to prevent similar attacks in the future, then improve preventative actions.

  3. Evaluate the cost-effectiveness of the current defense given its performance in this incident.

  4. Assess the damage the incident caused and confirm no further damage will occur.

STEP 2: Assess Roles

Identify each individual affected by the incident, whether they were involved or should have been involved. Were they able to successfully defend? Were they impacted by the incident?

STEP 3: Ask Questions

Create a list of questions derived from the goals that concern at least one stakeholder. Group the questions by the stakeholders identified when assessing roles.

Assign each group of questions to a single dominant stakeholder. This stakeholder must have access to the appropriate systems and resources to be able to accurately form a detailed answer for each question. Allocate appropriate time and resources so they are able to do a thorough job. This may involve internal or external resources depending on the skillset of the stakeholder.

Bear one thing in mind: Sometimes it is important to leave the “who” and focus on the “what”, “why”, and “how”.

STEP 4: The Technical Stakeholder

The technical stakeholder must simulate the chain of events from before the incident began until the very end. This should be meticulously crafted in a very high resolution, down to the last detail. It is critical to scope out missing parts and leave no stone unturned. Some areas to consider include:

    1. Vulnerabilities

    2. Weaknesses

    3. System Configurations

    4. Network Operations

By the end, the technical stakeholder must be able to explain how the threat actor was able to infiltrate, as well as where the attacker succeeded and failed. This must include all relevant data for the scope of the attack, no matter how far back in time the attack started.

STEP 5: Build a Timeline

Once each stakeholder has answered their questions, every member of the team should come together to connect the dots. Build a complete timeline using the perspectives of every stakeholder.

STEP 6: Establish Counteractions

Assign counteractions, remediations, improvements to the defense, architecture changes, and any other actions that will help prevent this type of attack in the future. Be practical, clear, and detailed.

If there are issues that cannot be addressed immediately, try to find a workaround. For example, if a certain server cannot be patched, make sure the detection system covers it and that its connections are limited and secured.

Assign each task to the appropriate stakeholder: the one that can accomplish the task in full. Set follow-up meetings for accountability and to address any lingering issues.

STEP 7: Help the Community

If this type of attack has the potential to impact other enterprises, find a way to inform them of the threat. Give them the much-needed details to prevent these incidents and be part of a greater community of defenders.

ASK THE RIGHT QUESTIONS

What Happened Before the Attack?

Look back in time to before the incident started. Are there any indications the attacker is already in your environment?

EXACTLY WHAT HAPPENED, AND AT WHAT TIME?

Walk through the incident step by step. Identify what happened before detection, during investigation, during response, and after remediation.

Review AT&T Incident Response Processes and Procedures →

HOW WELL DID THE TEAM HANDLE THE INCIDENT?

Talk about if the SOC was well-prepared for the incident. Were they able to resolve it quickly? Was someone on call to handle it?

Learn How to Build an Effective IR Team from IT Governance →

WHAT INFORMATION WAS NEEDED SOONER?

Consider the information your team was able to gather about the incident. Were they well-prepared to react? How much time passed before remediation?

Read about reducing MTTD and MTTR from Siemplify →

WERE ANY STEPS TAKEN THAT INHIBITED RECOVERY?

Consider whether the processes followed was as efficient and effective as it could be. What can be optimized? What can be removed?

Review Exabeam's Recommendations for Plans, Teams, and Tools →

WHAT COULD THE TEAM DO DIFFERENTLY THE NEXT TIME AN INCIDENT OCCURS?

Consider if each member of the team contributed as effectively as possible. Does the team understand what they need to do? Do they need training?

WHAT CORRECTIVE ACTIONS CAN PREVENT SIMILAR INCIDENTS?

Retrospectively analyze every piece of the attack. How far back can you find the attacker in your system?

WHAT ADDITIONAL TOOLS DOES THE TEAM NEED TO PREVENT THIS?

Did your team have the visibility it needed to resolve the incident completely? Think of good wish list items that you want your team to have to succeed.

How did management handle this incident?

Consider if management was well-prepared for an incident. Were lines of communication open? Did management understand the gravity of the situation?

EXISTING SECURITY TOOLS ARE NOT ENOUGH

indicatorofcompromise-01
Network Forensics

Network forensics is useful, but is limited to 2-4 weeks worth of raw data, with a few additional weeks for metadata. Further, capturing and storing packet capture is expensive.

Take the SANS Course 

siem-01
LOG MANAGEMENT

The main driver for longer data retention is compliance. Vendors offer a way to efficiently compress and archive data, but no easy way to access and correlate it.

Read Exabeam's Tips 

cloud-01
Cloud-delivered Platforms

Cloud-delivered platforms are a step in the right direction, but the cost of data shifts to the vendors, who have little incentive to store and pay for all that data.

Read the CISA Incident Response Report 

ENTER RETROSPECTIVE HUNTING

When a customer suspects a breach and deploys an incident response team, their first task is to obtain all data available about the incident. Incident responders are very good at sifting through data to uncover the needle in the haystack. However, this is a unique ability that takes a long time to hone, and one the majority in the industry does not have.

Data Retention

Compounding this, most organizations do not have all data available as part of their regular investigation because of compliance reasons or, alternatively, the inability to store it. Further, the initial point of compromise may have happened weeks or months before detection, which could easily go well beyond an organization’s data retention limits.

Chain of Attack

With retrospective hunting, enterprises are able to understand how widespread a campaign is by accessing data as far back as their team needs to go. They can identify all users and entities involved in the incident, and graph that behavior as a detailed chain of events.

Answer “What If”

Retrospective hunting lets security teams answer the critical question of, “Are we impacted by this world-wide breach?”. The second a large attack on another organization is reported, an enterprise can use retrospective hunting to search for the new zero day and confirm they are not vulnerable.

BEFORE HISTORICAL HUNTING

NO CERTAINTY OF WHERE THE ATTACK BEGAN
LITTLE CLARITY ON GAPS IN YOUR SECURITY PROGRAM
DOUBTS ABOUT ATTACKERS IN THE ENVIRONMENT
RISK OF REMAINING BACKDOORS, MALICIOUS USERS, AND MORE

AFTER HISTORICAL HUNTING

THE ABILITY TO FIND THE ENTIRE ATTACK LIFECYCLE
TOTAL INSIGHT INTO GAPS IN YOUR SECURITY PROGRAM FROM THE START
USE TODAYS INTELLIGENCE TO UNCOVER REMAINING ATTACKS
THE FREEDOM TO FIND ATTACKERS THAT ARE DUG IN FOR YEARS
TALK TO US ABOUT INCIDENT RESPONSE
GET EXPERT HELP