An interesting study into the comparative performance of leading search engines was published a couple of weeks ago and provides some interesting reading. The report (pdf) is available here.
The study was undertaken in Dec 2005 by students at the University of Provence in France and used Google, Yahoo, MSN, Exalead, Voila and Dir.com as the search engines to test.
The report is well worth a read and highlights that search engines are still a long way from providing the perfect search results. Key points of interest include the following:
- the degree of overlap between the search results returned is low (a maximum of 25% between Google and Yahoo)
- the overall level of satisfaction is very low, with up to half the results being judged as ‘off topic’
- Google and Yahoo enjoy the best results, although these are still low at an average score of 2.3 out of a possible 5.
This can only mean that search engines are struggling to deliver results that achieve satisfactory levels of relevancy. There is some good news (the ability to filter out pornographic websites has improved over the years) but the lasting message is that no single search engine provides reliably relevant results.
An interesting question that also arises from the survey is why Google enjoys such a large market share (compared to Yahoo) when the results are judged to be very comparable?
