Bypassing Cylance: Part 5 – Looking Forward
We just finished up a walk through of how we bypassed Cylance in a previous engagement. To conclude this exciting week, I want to share a few comments and notes with everyone.
First, we are a penetration testing firm. We test what customers give us to test. We do not test what the vendor wants us to test. We test where the rubber meets the road.
Some have pointed out that the techniques we used were basic. Yes, many of the techniques we used were extremely rudimentary. But they worked! This, in and of itself, calls into question the validity of the marketing claims AV companies sometimes make.
Let’s take a few moments and talk about those marketing claims and the NSS reports. Why is it that companies like Cylance and CrowdStrike are so insane when it comes to people publicly testing their products? Why is it that a small testing firm in South Dakota would be one of the only sources of information people have about a product’s limitations.
I want to apologize to Cylance for putting them in the same category as CrowdStrike. Cylance has not threatened litigation yet. Like CrowdStrike, they have insane EULA restrictions regarding the discussion, review, or independent analysis of their products. This would be like Chevy/Ford/Toyota threatening to sue Car & Driver or Consumer Reports anytime there was an unbiased investigative article written about one of their vehicles. Would these same car companies hunt down customers of vehicles who wrote blogs and posted on bulletin boards with negative experiences? Can you imagine if Cylance/Symantec/Oracle/Microsoft were hunting down customers who said bad things about them? That would be insane – Orwellian!
Cylance is far better than most traditional blacklist AV products. When we test a company, we try multiple types of malware and Command and Control (C2). This is so we can properly identify what the endpoint security tool can and cannot detect. There are a number of cases we expect to be detected, but seldom are with traditional blacklist AV. With this test, there were a number of cases where Cylance shined as compared to traditional AV. There were a number of cool things that Cylance did detect. It’s far harder to redact that information. But trust me… They are better than most traditional AV.
There’s always people who rightfully point out that whitelisting was not enabled and it would stop most (if not all) of the techniques we used. We can’t say enough positive things about whitelisting. It’s one of the single greatest defenses an organization can deploy. But it’s not magic that is held exclusively by Cylance or any other vendor. It’s something that can be deployed via Applocker or SRP via group policy – albeit not without significant administrative overhead. Thus, claiming a security product didn’t do well because whitelisting wasn’t enabled is invalid.
The point is, whitelisting is free. It is not, cannot, and will never will be a “feature” in one product alone. All vendors have special options and configurations which are great, but rarely implemented in the real world.
There are features in many endpoint security products which would shut down most attacks. Typically, these features are never fully enabled/deployed. Is it because the product’s flawed? Not necessarily. Often these features are disabled because they can decrease work productivity or have unexpected IT administrative costs, consequently reducing the effectiveness of the security solution.
The industry wants security products to simply work with little to no interaction. They want the easy button.
Cylance makes fantastic claims about Artificial Intelligence and how they’re going to render all other AV obsolete. Even being able to predict future attacks! I want to believe this, I really do… but, from what we’ve seen those claims are still a ways down the road. I will say, this product has a brilliant concept, it is a step in the right direction; though perhaps rushed to market before reaching butcher weight.
So what can we learn from this? A couple of things.
First, this highlights a need for more comprehensive, unbiased testing of security products. We’ve seen reports, like the recent NSS Labs reports, which leave substantial room for improvement.
Secondly, there is still no silver bullet. This industry’s marketing capitalizes on silver bullets to solve our fear of vampires but there’s no such simple solution. There’s no way even Cylance is the be-all, end-all solution that marketing claims, at least for now – there are many brilliant minds hard at work in the back room.
In conclusion, if there isn’t a silver bullet, what do we need? Architecture. We should be looking at whitelisting for applications and egress internet access.
Do not, under any circumstances, believe that moving to whitelisting or an advanced endpoint product is going to be an easy fix. You will need to work to properly implement it. You will need to have more staff on hand to keep these products fed and happy.
Every time you hit the easy button, God deploys another bot on your network.
Stop. Hitting. The. Easy. Button.
From South Dakota with love,
John
______
Much of what we did was based on the existing research of Casey Smith and @_TacoRocket. Read Colby Farley’s blog here:
*and the previous blog of Casey Smith
They’re truly fantastic and should be mined for tips, tricks and hints anytime you’re testing an advanced endpoint security product.
Ryan
March 31, 2017 @ 7:10 am
I’m interested to hear if you guys have done similar tests to bypass CrowdStrike on a customer environment? (Not just thier next-gen AV) but thier full suite of tools.
Great work!
tangent
April 1, 2017 @ 12:33 am
We are currently testing 6 endpoint solutions including Cylance and Crowdstrike.
After testing 1,000’s of known, personally mutated files, scripts, obfuscating malware and now moving into fileless testing, I agree 100% with John comments in this piece.
Every single solution that we have tested has great strengths and weakness. The reality of this testing was not which solution was better or worst, as honestly it started off that way. In fact our testing helped us identify our own security deficiencies in our security architecture. To Johns point, which we discovered, there is no “silver bullet” and each solution works differently to solve different problems.
Whatever solution we choose to help solve our business problem, one thing is for sure, these products are just another compensating control, the last defensive layer in our security archecture when other hardening controls can’t be implemented.
John, I appreciate the respect you have shown these vendors and their solutions in this last blog post. As security professionals we can all throw the first stone and give our option on what’s the “best” product but take a look in your own own backyard. At the end of day the problem always comes back to architecture and the implemented controls on your network.
Rony Michaely
April 1, 2017 @ 4:32 pm
Vendors who call themselves NextGen never heard of System & Application hygiene. The term BHO is not in their dictionary, neither malicious browser plugins. They present funny ‘Advanced Threats’ demos showing Web Born Attacks of malicious obfuscated JS they cannot understand, emulate nor prevent only to show they got lucky with the PE payload detection. Of course there are other PE payloads they do not detect and Advanced attacks do not necessarily need PE for a complete attack chain. How to change market awareness? maybe start with Campaigns of “Dear CISO, we are sorry we thought you are a dumb security purchase manager, Sincerely yours, NexGen Proclaimer ”.
https://www.linkedin.com/pulse/marketing-vs-reality-cylance-smoke-mirrors-rony-michaely
JOHN STRAND
April 3, 2017 @ 9:22 am
Wow… Just wow.
There have been two or three people the last two days with additional bypass techniques.
And, some stories about some of these companies getting all legal on testing firms. Or worse yet, their own customers.
We have not seen this from Cylance.. Yet.
John