There is a fundamental problem with many cyber-security products involving detection.
Bob Klein and Ryan Peters did a brilliant job outlining this problem in their Black Hat presentation, “Defeating Machine Learning - What Your Security Vendor is Not Telling You.”
To borrow Mr. Klein’s and Mr. Peters’ analogy, while introducing a twist:
Imagine a town with only one locksmith. Everyone buys the same lock from this locksmith. The lock looks strong and has good reviews, so why not? And the lock really is strong - all the latest technologies are inside, ensuring cracking it is as difficult as possible.
Then along comes a thief, intent on robbing a house. He approaches the first home and sees the lock on the door. No sense in sticking around and picking the lock where he can be seen; that’s risky. Instead, he makes his way down the street and one-by-one, he notices every house has the same lock. He realizes, “If I crack this lock, I can rob every house in town." So the thief goes to the locksmith and buys the same lock. He then heads to his basement with a two-liter bottle of Mountain Dew and gets to work on breaking the lock.
Days go by. Since the thief is determined and sophisticated, he finally cracks the lock. The effort was worth it. His next question is, “Which house do I rob first?”
The point is not that the lock was too easy to break. Better locks can always be built. The point is that if we accept the premise that an unbreakable, silver-bullet security product does not exist, and frankly, we’d be naive to believe otherwise, a sophisticated adversary obtaining an identical lock to ours makes our lock utterly useless against him.
It doesn’t take much imagination to apply this idea to cybersecurity.
The problem is everyone has an identical lock. This stems from two distinct issues:
- If an adversary obtains an identical lock to yours, he'll crack it in a safe place, away from your environment, then hit you hard and fast.
- If everyone has the same lock, the reward for cracking it is immensely high, motivating an adversary to dedicate more time and resources to cracking it.
As cyber-security defenders, how might we approach this problem?
Don’t use the same products everyone else is using
So you’re going to settle for second- or third-rate products? I don’t think so. A better idea is to develop your own, in-house tools. If you’re capable of this, do it. Unfortunately, it’s out of reach of most organizations. Why? Lack of time, money, and resources are the reasons first cited, but equally important is lack of data. Getting detection right is hard. Really hard. To do it properly, a significant quantity of diverse data is required for analysis. This is an area vendors with many customers and partnerships have an advantage. Also, there really are vendors out there with new, innovative ideas with real-world experience to back it up.
Frequent updates
This doesn't refer to new feature updates (those are nice though.) It means updates to the signatures / rules / algorithms / models the product uses for detection. By updating frequently, you cause the attackers some pain because time is no longer on their side. Think The Imitation Game, where Alan Turing and team attempt to break the Enigma code before the Germans changed their encryption key. Once the key changes, they must start over from scratch. Talk about frustration.
Updating frequently is good because we cause the adversary pain; however, frequent updates do not solve the problem: there is no guarantee the crack an attacker finds will be addressed in an update. In fact, unless you are talking about an extreme example like changing encryption keys, it probably won’t be.
Ultimately, frequent updates in this context are akin to taking aspirin when you have a backache; you mask the pain, but don’t fix the root cause. Not to mention, security products rarely focus on ease-of-use and intuitiveness, often resulting in painful upgrades. Instead of looking for a product that doesn’t upgrade their detection frequently, look for a product where upgrades are frequent, easy, and seamless.
Hire smart, creative analysts to hunt for attacks
Every human mind is different. Creativity spurs differentiation. Therefore, this definitively solves the identical lock problem. The challenge here is that it is not possible for any organization to find and retain the talent necessary to effectively protect an entire environment. You should still strive to hire as many smart, creative analysts as possible because they are gold in this industry. And like gold, great analysts are difficult to find, expensive, and everybody else wants them too.
People are good at certain things; machines are good at others. People cannot continuously hunt 24x365 even in small environments. Machines can. There is simply too much data, and we are dealing with a big data problem in cyber security. People aren’t good at shifting through vast amounts of data, but we are good at building, managing, and improving machines to handle that data. Machines are exactly the opposite of people in this regard. For this reason, arming analysts with state-of-the art, automated hunting capability is essential.
Write your own rules
By writing unique detection rules, no one will have the same lock that you do. The identical lock problem is solved, but at what cost? Misconfiguration and improper tuning are concerns, and so is the ongoing maintenance of continuously adding new rules to stay ahead of adversaries. Writing a good rule is incredibly challenging. A good rule is one that detects malicious behavior (painful for adversaries to adapt to) instead of IOCs and artifacts (easy for adversaries to adapt to) while keeping false positives tolerable. When seen this way, the word ‘rule’ gives the wrong connotation; in order to detect behaviors like this, complex algorithms and data models are better suited for the job.
The Solution
The options above have downsides, don’t actually solve the problem, or are impossible practically.
The best solution is for vendors to automatically and seamlessly introduce enough differentiation in their product's detection capability so every deployment is unique. If an adversary cannot get an identical copy of the product you use, he must interact with your environment to find a crack. Furthermore, he is less motivated because success doesn’t grant him the keys to the kingdom.
It sounds counter-initiative at first, but having the attacker lurking around your environment isn’t a bad thing when you have the tools to properly detect, monitor, understand, and respond to that attacker. Consider the alternative: the adversary prepares the attack safely in his basement, where you have no visibility into what he’s doing, how he operates, or where he’s going to hit you. The more we understand our adversary, the more we gain the upper hand.
Unfortunately, dealing with the Identical Lock Problem is often not part of vendors’ methodologies. We can fix this, though, by investing in products that can solve this problem (yes, they’re out there) and by asking the right questions to encourage the market to fix it.
Here are a few questions to ask your security vendor (and be sure to drill in):
- How will my deployment of your product/service be different from my neighbors with respect to your detection capability?
- Does your detection adapt automatically over time? If so, how? Does it adapt continuously or in a batch mode?
- How often are you updating your detection signatures / rules / algorithms / models?
- What effort is involved in your upgrades?
- What automated hunting capability does your product have? How does it scale to handle enormous quantities of data?
