ASIS&T 2006 START Conference Manager    

The Effects of Conceptual Description and Search Practice on Users’ Mental Models and Information Seeking in a Case Library with a Best Match Search Mechanism

Wu He & Sanda Erdelez

ASIS&T Annual Meeting - 2006 (ASIS&T 2006)
Austin, Texas, November 3-9, 2006


Abstract

Traditional searching for information relies on “exact match” search mechanism that encourages the use of Boolean queries and keywords. However, in the recent years novel retrieval systems have started to incorporate search engines based on a “best match” search mechanism.This mechanism relies on unstructured user queries and ranks search results according to their probability of relevance to the query (Cool & al., 1996).

A best match search engine often expands or transforms a user’s query during the retrieval process. This transformation process may have a profound influence of both the search results and the users’ ability to understand the relationship between the query and the returned results (Muramatsu & Pratt, 2001). For instance, users may find mysterious search results and get confused. Thus, for users accustomed to traditional keyword match with Boolean operators using a system with a best match search mechanism can be challenging.

Information retrieval literature (e.g., Campagnoni & Ehrlich, 1989; Hsieh-Yee, 1993; Savage, 2001), suggests that the knowledge of an information system (i.e., system knowledge) is best acquired during the actual use of the system, and that prolonged use increases proficiency.Constructivism learning theory (Jonassen, 1999) also suggests that knowledge and meaningful learning are best built by doing and experiencing.

A detailed literature search revealed no published research on how presence of a mental model affects the users’ search performance and satisfaction in a case library with a best match search mechanism. Our study examined the following research questions:

• How do conceptual description and search practice differ from each other in terms of their role on the construction of the user’s mental model? • Is there a statistically significant difference between conceptual description and search practice with respect to search satisfaction? • Is there a statistically significant difference between conceptual description and search practice with respect to search correctness? • Is there a statistically significant difference between conceptual description and search practice with respect to search time? • What are the relationships among user’s mental models,search satisfaction, search time, and search correctness?

This study collected data from users performing information seeking queries and tasks using a combination of both qualitative (structural interview) and quantitative (questionnaire and transaction logs) data collection methods.

While detailed analysis is still in progress, we will describe some preliminary findings: • There is no statistically significant difference between conceptual description and search practice in terms of the quality of the user’s best match mental model, search correctness, and search satisfaction. There is significant difference between conceptual description and search practice with respect to search time. • Qualitative analysis for the subjects’ post mental models revealed that subjects in the conceptual description group seem to have more complete mental models of the best match system than those in the search practice group. • Subjects with best match models have significantly higher search success and search result satisfaction than subjects without the best match models. However, there is no significant difference in terms of search time. • We did not find significant correlation among search time, search success and search satisfaction. • Comparing the subjects’ attitudes toward the system, we found that subjects with best match models felt it was easier to use the system than subjects without mental models.

In conclusion, as an exploratory research at the intersection of Information Science and Learning Technology this research revealed the role and importance of training approaches on user’s understanding,information seeking and acceptance on best match systems.


  
START Conference Manager (V2.52.6)
Maintainer: rrgerber@softconf.com