The Retrieval Effectiveness of Web Search Engines: Considering Results Descriptions

Lewandowski, Dirk: Retrieval Effectiveness of Web Search Engines: Considering Result Descriptions. Journal of Documentation 64(2008)6, 915-937. DOI: 10.1108/00220410810912451

Download: Preprint / Publisher Version


Purpose: To compare five major Web search engines (Google, Yahoo, MSN,, and Seekport) for their retrieval effectiveness, taking into account not only the results but also the results descriptions.

Design/Methodology/Approach: The study uses real-life queries. Results are made anonymous and are randomised. Results
are judged by the persons posing the original queries.

Findings: The two major search engines, Google and Yahoo, perform best, and there are no significant differences between them. Google delivers significantly more relevant result descriptions than any other search engine. This could be one reason
for users perceiving this engine as superior.

Research Limitations: The study is based on a user model where the user takes into account a certain amount of results rather
systematically. This may not be the case in real life.

Practical Implications: Implies that search engines should focus on relevant descriptions. Searchers are advised to use other search engines in addition to Google.
Originality/Value: This is the first major study comparing results and descriptions systematically and proposes new retrieval measures to take into account results descriptions.