Later this week MITRE Engenuity will be releasing the results from their fourth round of the ATT&CK Evaluations. This round focused on threat actors Wizard Spider and Sandworm. In this article, we’ll review why MITRE is the preeminent organization providing third-party evaluations of vendor solutions, and the key metrics to look for when evaluating the effectiveness of a solution.
The ATT&CK emulation tests are performed by MITRE Engenuity, a non-profit organization that provides transparency and publicly available test data that measures the efficacy of leading cybersecurity products and solutions. These evaluations are based on the ATT&CK framework, which maps adversarial tools, tactics, and procedures (TTPs) used by malicious actors and cybercriminals.
MITRE Engenuity ATT&CK Evaluations objectively determine the effectiveness of participating vendors’ abilities to prevent and detect these TTPs. With MITRE Engenuity serving as an unbiased evaluator of a cybersecurity solution’s efficacy, these yearly ATT&CK evaluations provide end-users with a window into expected performance against real-world cyber attacks.
This year’s emulation by MITRE Engenuity focused on Data Encryption For Impact (T1486)—which in common parlance we refer to as ransomware.
MITRE’s lab tests simulated the attacks of two common ransomware operators Wizard Spider and Sandworm.
Wizard Spider is a financially motivated criminal group that has been conducting ransomware campaigns using the Ryuk malware against a variety of organizations, ranging from major corporations to hospitals.
Sandworm Team is a destructive Russian threat group that has been attributed to Russian GRU Unit 74455 by the U.S. Department of Justice and the U.K. National Cyber Security Centre. Sandworm Team's most notable attacks include the 2015 and 2016 targeting of Ukrainian electrical companies and 2017's NotPetya ransomware attacks.
Organizations evaluating cybersecurity solutions that have been tested and proven against ransomware threats, like those propagated by Wizard Spider and Sandworm, will find this MITRE evaluation to be an invaluable guide as they navigate the vendor solutions available today.
The key metrics to consider in the MITRE results will vary depending on the unique needs of each security team, but there are four metrics that are especially impactful: Protection, Visibility, Real-Time Response, and Analytical Coverage.
This category of testing focuses on prevention—stopping threats from executing before a foothold can be gained or damage can be inflicted.
When it comes to improving your security posture, it all starts here. A solution that cannot provide comprehensive protection to prevent threats sends the rest of the security operation into a tailspin.
Solutions with comprehensive protection scores can be relied upon to prevent threats (ransomware and otherwise) before they even start. Solutions tested and proven to be effective at providing preventative protection will drastically reduce the number of alerts that the security team has to triage downstream, improving the team’s ability to uncover sophisticated threats and reducing burnout on the security team.
Visibility matters, especially when dealing with threats like ransomware, which has grown more challenging to detect as threat actors try new and complex techniques. That is why this round of MITRE Engenuity ATT&CK Evaluations is a powerful gauge of how vendor solutions performed in spotting and stopping ransomware in some of its most complex forms.
Another important aspect of visibility is having the business context to improve mean-time-to-respond and give you the full breadth of the attack story. An operation-centric approach to security means the full context of an attack, where it originated, what was affected, the timeline of events, as well as the granular details of the attack chain are displayed in a simple to understand and use platform.
During the simulated attacks in MITRE Enginuity’s lab, some solutions may miss the attack at first, but after additional processing time and having a human analyst confirm malicious activity, a missed detection may be uncovered. These missed threats that are eventually caught are referred to as “delayed detections.”
When it comes to defending against ransomware threats the single most important factor is time. Attackers that are afforded the opportunity to dwell in environments deepen their footholds and move laterally. As they engrain themselves, they exfiltrate more and more data (to make double extortion a credible threat in the future) and stage as many machines as possible for future encryption (so they can ensure devastating business interruptions). This means effective ransomware protection must be able to uncover ransomware activity, respond, and recover from the threat faster than we ever have before.
A solution that’s delayed in its detection of threats isn’t going to cut it against today’s modern ransomware threats.
The analytical coverage evaluation is a powerful way to measure which vendors provide enriched context when it comes to attacks. The vendors that perform well in this category demonstrate an effective mapping to the MITRE Framework and present detections that are mapped to key ATT&CK techniques.
In essence, analytic detections are the pinnacle of all detections because they encompass a combination of tactic and technique detections. Vendors who perform well in this category are providing greater insight into an attack from more subtle signals and indicators.
MITRE evaluations shine a spotlight on vendors’ true capabilities and their ability to effectively secure customers from known, real-world threats that are used to attack the public and private industry every day. Solutions that perform exceptionally well in Protection, Visibility, Real-Time Response, and Analytical Coverage provide customers peace of mind knowing that they are protected from real-world, advanced threat actors.
Cybereason values the work of the MITRE organization and the benefits this brings to customers. That is why Cybereason is a research sponsor for MITRE’s Center for Threat-Informed Defense (CTID), a non-profit, privately funded research and development organization operated by MITRE Engenuity with a mission to advance the state of the art and the state of the practice in threat-informed defense globally.
Cybereason is dedicated to teaming with defenders to end cyber attacks from endpoints to the enterprise to everywhere. Join us on April 7th, 2022 for a webinar where we will take a deep dive into the MITRE Wizard Spider and Sandworm Enterprise ATT&CK Evaluation results with Cybereason experts.