No security or AV company is equipped with a procedure, independent of hardware or personnel requirements, that can easily keep up with the daily barrage of newborn threats. Shadowserver shows they receive daily unique binaries numbering in the tens of thousands. With the mass amount of malware being created and distributed across the internet, each security company is left with the burden of being unable to “catch ’em all.”

They must then employ a prioritization method of analysis, often leaving data too long in the queue, some collecting dust. Some security companies concentrate on searching for malicious domains and IPs while others concentrate on binary identification, many using a hybrid approach. All, however, are in search of a way to efficiently label these variables as malicious or benign, trying desperately to keep pace with the release of new malware.

AV companies have of course felt the strain of keeping up with the Joneses and for fear of looking inferior have made the choice to often “borrow” the conclusions made by other AV groups.

According to this “Analyst’s Diary” entry at Kaspersky Lab, an experiment was used to show just how often AV groups rely on one another to categorize samples as malicious in order to appear up to date. From the blog:

“We created 20 clean files and added a fake detection for 10 of them. Over the next few days we re-uploaded all twenty files to VirusTotal to see what would happen. After ten days, all of our detected (but not actually malicious) files were detected by up to 14 other AV companies…”

I can’t exactly blame those copycat AV companies for trying to stay on par with others. There is constant pressure, of which all security groups are aware, to try and balance reputation, integrity, and effectiveness. Trying to avoid false positives means evil may slip by unnoticed, while avoiding false negatives means sacrifices in accuracy. A series of check systems could be put in place but often there is insufficient detail or time for quality assurance, and delays in the conviction process detracts from the goal of real-time protection.

Security researchers often collaborate in some way, perhaps only in certain circles, but we do so because each performs their own independent analysis in their own area of expertise, bringing unique input to the table. Our products should behave no differently. Only shared information that meets certain quality requirements should be used, according to the individual company’s ruleset. If a company or security product has nothing to contribute and only relies on the work of others then it has little purpose in this industry, (yet may find success with the right marketing). However, a company will struggle greatly if they dismiss or completely separate themselves from the security zeitgeist.

In recognition of this need for both dependence and originality, Defence Intelligence is working to bring security and internet architecture groups together to create something new and more complete. We want to make a product that takes a more global approach to the threats we’re facing, but also bring a confidence and purpose back to our industry that seems to have waned. A strong offence may rely on a good defence but we need both if we’re ever going to make real advancement on this battleground.

Matt Sully
Director
Threat Research & Analysis