| ASIS&T 2005
Toward a Unified Retrieval Outcome Analysis Framework for Cross Language Information Retrieval
Sparking Synergies: Bringing Research and Practice Together @ ASIST '05 (ASIS&T 2005)
Westin Charlotte, Charlotte, North Carolina, October 28 - November 2, 2005
This paper introduces a concept called Retrieval Outcome Analysis (ROA) to stand for a systematic approach not only to assess overall information retrieval system performance, but also to discover solutions for individual queries, especially hard queries. A ROA framework for cross language information retrieval (CLIR) is then proposed to demonstrate the usefulness of the concept. The framework is comprised of four interrelated components: Overall System Performance Evaluation, Query Categorization, Translation Analysis, and Individual Query Analysis. An example of applying the framework is discussed in detail. We believe the proposed framework would be especially useful for the development of real-world CLIR and multilingual systems since the evaluation guided by the framework could provide richer evidence on how to further improve those retrieval systems under investigation.
this on the ASIS&T 2005 Annual Meeting wiki!