Challenges for search engine retrieval effectiveness evaluations: Universal search, user intents, and results presentation

Lewandowski, D. (2013). Challenges for Search Engine Retrieval Effectiveness Evaluations: Universal Search, User Intents, and Results Presentation. In G. Pasi, G. Bordogna, & L. Jain (Hrsg.), Advanced Techniques in Web Intelligence 3: Quality Issues in the Management of Web Information (S. 179–196). Heidelberg: Springer.

Download: Preprint / Publisher Version


This chapter discusses evaluating the quality of Web search engines to effectively retrieve information. It identifies three factors that lead to a need for new evaluation methods: (1) the changed results presentation in Web search engines, called Universal Search, (2) the different query types that represent different user intentions, and (3) the presentation of individual results. It discusses implications for evaluation methodology and provides some suggestions about measures.