How Good Are Website-Reputation Services?

Websites have now become the standard modus operandi for spreading malicious software to infect personal and corporate environments. A large number of benign and well-meaning websites are compromised everyday by hackers inserting malicious code to, in turn, infect the computers used by visitors to the hacked site.
One of the ways to combat this is to develop a website reputation mechanism which can warn of potential threats before visiting a compromised site.

A report by StoptheHacker.com shows results of a test they conducted where 350 websites known to be malicious where checked by a number of the leading website-reputation service.

Here’s an excerpt of what they reported:

“At StopTheHacker.com (Jaal LLC) we have conducted tests with 350 domain names, all of which have been reported as malicious by volunteers of various blacklists.

The aim of the test is to:

1. Identify how accurate the website reputation services are
2. What is the overlap in terms of safe/unsafe websites

We have found some interesting results which we present in this article. First we detail the parameters of the testing procedure to provide an idea of how the test was set up.

Website-reputation services vary wildly in their opinions.

Note that all 350 domains, were reported as malicious, and were collected from malware.com.br on December 18, 2009. The blue column (maximum 350) indicates the number of sites that the website-reputation service correctly identified reported bad sites. The orange column (maximum 350) indicates the number of sites that the website-reputation services incorrectly identified reported malicious sites as safe.

We compared the reputation provided by each website-reputation service and observe how many websites are marked as unsafe, safe, untested, maybe-unsafe/caution/potentially-unsafe, unreachable.

* McAfee Siteadvisor
* Norton Safeweb
* Google SafeBrowsing
* Comodo Siteinspector

The date that we conducted this test was: December 21, 2009.

We have identified some of the most interesting results below:

1. McAfee Siteadvisor marked 32.5% of Domains as Unsafe, 22% as Safe, 43% as Untested and 1.7% as Potentially-unsafe.
2. Norton Safeweb marked 50.86% of Domains as Unsafe, 43.71% as Safe, 2.29% as Untested and 3.14% as Potentially-unsafe.
3. Google SafeBrowsing marked 10.86% of Domains as Unsafe, 89.14% as Safe. Note: the presence of the hash of the domain name being tested, on the google malware hash list, is interpreted as “unsafe” while the absence in interpreted as “safe”.
4. Comodo Siteinspector marked 0.29% of Domains as Unsafe, 98.86% as Safe and 0.86% as Unreachable. Note: after feedback from Comodo, a retest was conducted, accuracy changed from 0.29% -> 1.2%.”

The full report is here:http://www.stopthehacker.com/2009/12/21/how-good-are-website-reputation-services/

So this test shows that you cannot rely solely on these website reputation services to provide accurate security against malicious websites. THey do add an extra layer for sure, but need to used in conjuntion with other anti-malware security tools.