Annual Meeting Contributed Papers 2009 START Conference Manager    

Evaluating a Metadata-based Term Suggestion Interface for PubMed with Real Users with Real Requests

Muh-Chyun Tang, Wan-Ching Wu and Bang-Woei Hung

(Submission #38)


This paper reports a continuing evaluation of MAP (Multi-faceted Access to PubMed), a metadata induced query suggestion interface for PubMed bibliographic search. A novel evaluation methodology was used to address the challenges involved in evaluating an interactive information retrieval system such as the term suggestion interface proposed here. The most significant aspect of this methodology is that it uses real users searching for real search requests on real systems, instead of using assigned tasks common in traditional IR evaluation. Several performance criteria were used to demonstrate the potential benefits of the term suggestion function, including the comparison between users’ initial and revised query, user satisfaction, user assessment of the records, and relevant records. Use of genuine requests also allowed us to investigate whether the effectiveness of MAP differs in different search situations.

START Conference Manager (V2.54.6)