MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (MITRE ATT&CK) is an open and transparent methodology that can be used to evaluate security vendors capabilities. It is a knowledge base and complex framework of more than 200 techniques that adversaries may use over the course of an attack. These include specific and general techniques, as well as concepts and background information on well-known adversary groups and their campaigns. Since its inception in 2015, ATT&CK has become one of the most respected and most referenced resources in cybersecurity. This post explains everything you need to know about the latest MITRE ATT&CK evaluation.
One of the most valuable things MITRE ATT&CK has given the cybersecurity industry is a common language and framework to discuss and analyze an attacker’s tactics, techniques, and procedures (TTPs). The ATT&CK framework gives organizations a better understanding of the behaviors of an adversary so they don't have to analyze endless malware. Another exciting component of MITRE ATT&CK is their ATT&CK-based product evaluations, as the product evaluations use the ATT&CK framework to give insight into how security vendors approach threat detection. Last year, the MITRE ATT&CK team evaluated several security vendors to report on how effectively they identify the techniques used by threat actors for a specific APT (APT3) without scores, rankings, or comparisons. Defensive teams – whether tactical, strategic or operational – can make good use of this information in actionable ways like using it to create prevention and detection rules or to guide architectural and policy decisions to protect an organization.
While the MITRE ATT&CK Evaluations Round 1 was based on APT3 (Gothic Panda), MITRE ATT&CK Round 2 focuses on TTPs associated with APT29 (Cozy Bear), a hacker group believed to be linked to the Russian government. The group has been tied to cyber attacks on the Pentagon, on other Western governments, and on organizations and agencies around the world, but it has become most infamous for its hack on the Democratic National Committee in the run-up to the 2016 US Presidential elections.
“While APT3 has focused on noisier, process-level techniques—relying on pre-installed system tools that hide malicious activity within legitimate processes—APT29 offers the chance to measure against an adversary that uses more sophisticated implementations of techniques through custom malware and alternate execution methods.”
- Frank Duff, Lead Engineer of the MITRE ATT&CK Evaluations
APT29 is distinguished by its stealth and the sophisticated implementations of techniques via an arsenal of custom malware. Their tactics vary significantly depending on the target and method of exploitation used to gain access – ranging anywhere from low and slow targeted techniques to full-on smash-and-grab. They regularly leverage custom malware and living-off-the-land binaries and scripts, with heavy use of PowerShell. During the evaluation, the focus is on two scenarios that emulate publicly reported APT29/Cozy Bear/The Dukes tradecraft and operational flows. The evaluation is spread out over two days to test 58 Enterprise Windows ATT&CK techniques across 10 ATT&CK tactics.
APT29 Evaluation: Technique scope. Credit: MITRE ATT&CK
Similar to last year, all security vendors are evaluated using a subset of six detection types. When a vendor detects adversary behavior, the MITRE ATT&CK team records the detection based on how much relevant information the vendor is able to gather. A technique may be recorded as detected more than once if the vendor’s product detects the technique in multiple ways. Some of the main changes for this round include:
Credit: MITRE ATT&CK
As attackers continue to find ways to avoid detection by traditional security tools, defenders find themselves having to change how they approach defense. MITRE ATT&CK shifts the perception of how to detect from low-level indicators like IP addresses and domain names to behaviors. The MITRE ATT&CK team recognizes that real-world threats are constantly advancing, and existing evaluation methods simply do not give buyers a clear understanding of how vendors will protect them. In order to give buyers the complete picture, MITRE ATT&CK maps events as well as stand-alone IoC’s. By themselves, these events may appear benign. However, when correlated with other events, they give security analysts the necessary context to identify advanced persistent threats (APT) beyond IOCs.
Compared to other third-party evaluations, which tend to focus on prevention capabilities, MITRE ATT&CK measures the number and types of detections based on real-world adversaries. Ultimately, you can’t prevent or respond to what you can’t see. Visibility is a key criteria in ATT&CK evaluations. However, keep in mind, no one vendor can or should aim to have 100% ATT&CK coverage. Some portions of the evaluation may be irrelevant to specific organizations needs.