Antivirus Testing Agencies
Whether free or purchased, the single most important factor in the decision for antivirus software is the scanner's ability to detect viruses. While this may appear obvious, many purchase antivirus software based on feature sets and price, overlooking detection rates in the process. Couple this with the fact that determining just which product has the best detection can seem downright impossible, and it is easy to understand why such an obvious approach is often undertaken by only the most determined.
Considering there are only a handful of certifying agencies, why is there so much confusion? To better answer that question, let's look at the who and how of certifications.
What is the Wildlist?
The WildList is the only standardized testing used across the board, even though other tests will be used by the various testing agencies. For WildList inclusion, designated reporters are tasked with the responsibility of reporting any qualifying malware that actively infects in any given month. Additionally, a minimum of two reporters must notify in the same month for the malware to be considered in-the-wild. Once placed on the WildList, the malware will remain listed for up to a year and a half after the last report, unless the original reporters request earlier removal.
Since the Wildlist is cumulative, it cannot be considered a prevalency or even a commonality gauge. Additionally, since the WildList is published monthly, testing is based on at least month-old virus reports. In other words, the Wildlist cannot be considered a measure of what is presently in-the-wild, but rather what was considered in-the-wild for a given month.
It's worth noting that without such an effort, there would be no standardized testing mechanism available.
Who certifies antivirus?
Well known testing agencies (in alphabetical order) include:
How should results be viewed?
Test results vary and are often confusing. For example, an antivirus scanner may fare very well in the VB100% Awards, but do poorly in the real-world testing of Hamburg and AV-Test.org. Indeed, such disparities are not uncommon, though there are products that maintain a respectable showing in all tests.
Consistency is a good measure to consider. Has the antivirus scanner performed well on each of the various tests? Note that this doesn't mean they necessarily have the highest score - a solid performer across the board is better than a scanner that gets high marks on one test but low marks on several others.
Don't overlook the fact that testing costs money. If an antivirus vendor wants testing and certification, chances are they will have to pay not only per platform (operating system tested), but also for each type of test performed. For this reason, a very large vendor (in terms of spending capital) may have several products listed as certified, while a smaller company may have only one or two listed. In such cases, it might be useful to remember that the same core scanning technology can be expected to be in other platforms the vendor supports, even though those particular platforms may not be listed as certified.
Considering there are only a handful of certifying agencies, why is there so much confusion? To better answer that question, let's look at the who and how of certifications.
What is the Wildlist?
The WildList is the only standardized testing used across the board, even though other tests will be used by the various testing agencies. For WildList inclusion, designated reporters are tasked with the responsibility of reporting any qualifying malware that actively infects in any given month. Additionally, a minimum of two reporters must notify in the same month for the malware to be considered in-the-wild. Once placed on the WildList, the malware will remain listed for up to a year and a half after the last report, unless the original reporters request earlier removal.
Since the Wildlist is cumulative, it cannot be considered a prevalency or even a commonality gauge. Additionally, since the WildList is published monthly, testing is based on at least month-old virus reports. In other words, the Wildlist cannot be considered a measure of what is presently in-the-wild, but rather what was considered in-the-wild for a given month.
It's worth noting that without such an effort, there would be no standardized testing mechanism available.
Who certifies antivirus?
Well known testing agencies (in alphabetical order) include:
- AV Comparatives
- AV-Test.org
- ICSA Labs
- NSS Labs
- OESIS OK (interoperability certification)
- West Coast Labs
- Virus Bulletin
How should results be viewed?
Test results vary and are often confusing. For example, an antivirus scanner may fare very well in the VB100% Awards, but do poorly in the real-world testing of Hamburg and AV-Test.org. Indeed, such disparities are not uncommon, though there are products that maintain a respectable showing in all tests.
Consistency is a good measure to consider. Has the antivirus scanner performed well on each of the various tests? Note that this doesn't mean they necessarily have the highest score - a solid performer across the board is better than a scanner that gets high marks on one test but low marks on several others.
Don't overlook the fact that testing costs money. If an antivirus vendor wants testing and certification, chances are they will have to pay not only per platform (operating system tested), but also for each type of test performed. For this reason, a very large vendor (in terms of spending capital) may have several products listed as certified, while a smaller company may have only one or two listed. In such cases, it might be useful to remember that the same core scanning technology can be expected to be in other platforms the vendor supports, even though those particular platforms may not be listed as certified.