Researchers Trick Cylance Into Giving Malware a PassWannaCry, SamSam Skirt Past Cylance's Protect Product, Skylight Cyber Reports
An Australian cybersecurity company says it tricked BlackBerry's Cylance Protect anti-virus product into believing that some of the most pernicious types of malware, including WannaCry and the SamSam ransomware, were benign programs.
Skylight Cyber says it examined how Cylance's Protect product evaluates malware, giving it a score to determine whether an executable is likely to be malicious.
Researchers at Skylight Cyber say they discovered that appending strings from the executable of a gaming application to files such as WannaCry would fool Cylance Protect's detection engine into thinking the file was not malware. The findings were first reported by Vice's Motherboard. The specific gaming application was not revealed.
"This method proved successful for 100 percent of the top 10 malware for May 2019, and close to 90 percent for a larger sample of 384 malware," Skylight writes in a blog post.
Skylight's CEO, Adi Ashkenazy, tells Information Security Media Group that the issue researchers found with the gaming strings is essentially a "bias" that is baked into Cylance Protect's detection mechanism. Other vendors may have the same issue, he says.
In a statement, Blackberry Cylance says it's "aware that a bypass has been publicly disclosed by security researchers. We have verified there is an issue with Cylance Protect, which can be leveraged to bypass the anti-malware component of the product. Our research and development teams have identified a solution, and will release a hotfix automatically to all customers running current versions in the next few days."
Blackberry Cylance declined to comment further beyond its posted statement.
Bypassing anti-virus programs by creating malware that looks legitimate is nothing new, and it's not terribly surprising that products that rely on artificial intelligence and machine learning are also prone to error, writes Martijn Grooten, the editor of Virus Bulletin, a security product testing and research organization.
"Of course, it's kind of funny that in Cylance's case, you could bypass it by just adding strings, but that's how AI/ML works: It discovered that the output of strings is a strong indicator of whether something being malware, so that's what it used," Grooten writes on Twitter.
Blackberry Cylance is one of many security vendors that have heralded the use of artificial intelligence and machine learning to catch malware. Those vendors contend their approach is more reliable that relying on signatures, or descriptions of already known malicious files that are updated regularly in endpoint security products.
Relying on signatures has a weakness in that slightly modified versions of the same malware may not be caught. The security industry has moved toward other detection methods that are used in parallel, such as observing the behavior of a file.
The Cylance Protect product is intended to catch never-before-seen malware based on an analysis of millions of characteristics. Blackberry Cylance has said that the advantage of using a well-trained algorithm is that it rarely has to be updated and is likely to trigger on unknown malicious files.
By all measures, the Cylance Protect product is a competitive and effective one. But the company has drawn some scrutiny over its marketing practices and response to testing (see: Anti-Virus Wars: Sophos vs. Cylance).
BlackBerry, which shifted its business from the mobile into the internet-of-things security space, completed its $1.4 billion acquisition of Cylance in February.
Reversing the Score
Skylight Cyber reverse engineered the Cylance Protect application to figure how it scores executables to determine whether they're likely malicious. The worst score a file can get is -1000, with the best 1000.
The researchers figured out that Cylance Protect had incorporated exceptions into its classification model, similar to whitelists and blacklists. Their first idea was to create a malicious file that looked close enough to something on the whitelist.
But, they write, "we quickly realized that this technique has little chance to work as this mechanism relies on thousands of features, some of which are extremely hard to modify."
But they noticed that Cylance Protect also uses a method of analyzing the strings in an executable and pondered whether there was a bias toward certain kinds of strings. "The string features are easier to manipulate, and adding them to an executable should not be too difficult," the researchers write.
They found that Cylance Protect had whitelisted a gaming application, perhaps because it was generating false positives. They bought the game and extracted the strings in the main executable.
"That is when we had our second eureka/lazy moment: Let's try the most naive solution and slap the strings onto the end of the file," they write. "That wouldn't possibly ever work, would it?"
It did. In tests, appending those strings to Mimikatz, for example, changed its score from a -852 to 999, nearly a perfect score. They ran other tests with WannaCry and SamSam, with the same results, according to a video.
Blackberry Cylance hasn't much time to react to Skylight's findings.
Ashkenazy says Skylight Cyber approached Vice's Kim Zetter about a week ago, who then contacted Blackberry Cylance. Skylight Cyber provided Blackberry Cylance with the technical details on Thursday, he says.
Although Cylance Protect would be vulnerable until it is fixed, Ahskenazy says the company didn't provide enough information in its blog post to reproduce its research.
He says Skylight Cyber follows the Australian CERT disclosure policy, which says vulnerabilities can be revealed publicly after 45 days regardless of whether a patch is available.
"However, we did not consider this to be a software vulnerability," Ashkenazy says. "We were interested in showing the basic flaw and attack surface provided by pure AI-based protection."
One of Cylance's founders, Chief Scientist Ryan Permeh, tells Vice that it likely won't take the company long to retrain its detection algorithm with full details on the bypass from the researchers. Ashkenazy says if Blackberry Cylance tries to fix this issue with, say, a blacklist "then it may be a fast fix but a very poor one."
"Fixing the bias in the model itself is going to be substantially more difficult, as it is not a patch, rather a full retrain of the model," he says.
In an update on Sunday, Cylance says it has added anti-tampering controls to its parser, which pulls artifacts - also called features - from an unknown file. Cylance says it analyzes those features using its algorithm.
It says it has also strengthened its model to prevent certain features "become proportionally overweight." Cylance also "removed the features in the model that were most susceptible to tampering."
"We appreciate the efforts of security researchers who responsibly disclose vulnerabilities and move the industry forward," the company says.
Grooten tweets that Skylight's technical finding is interesting, but it shouldn't necessarily stop people from using Cylance Protect. Malware writers are in the business of tricking AV products. But the findings are a warning of how artificial intelligence and machine learning techniques could be vulnerable to bias, Grooten says.