Bulletin, June/July 2006


Toward Human-Computer Information Retrieval

 

by Gary Marchionini

Gary Marchionini is with the University of North Carolina at Chapel Hill . He can be reached at march<at>ils.unc.edu. This article is based on the author’s Samuel Lazerow Memorial Lecture, given at The Information School, University of Washington , on October 17, 2005.

This is a propitious time for information science. The WWW has propelled information services into the public eye as never before, and information professionals are sought out in all walks of life to assist people with work, learning and play in the information environment. Classical information retrieval has yielded novel techniques for applying computers to retrieval problems, including WWW search engines. The classical model of retrieval is one of matching queries to documents and ranking these matches. It is apparent, however, that a new model of retrieval is needed as people access large-scale digital libraries of multimedia content and vast collections of unstructured data in the WWW. What is needed are ways to bring human intelligence and attention more actively into the search process.

            To this end, researchers are beginning to combine the lessons from designing highly interactive user interfaces with the lessons from human information behavior to create new kinds of search systems that depend on continuous human control of the search process. I call this hybrid approach to the challenges of information seeking, human-computer information retrieval (HCIR). Though human-computer information interaction is perhaps a more expansive and appropriate phrase, the HCIR phrase unites two well-known fields/communities of practice and is thus adopted here.

            HCIR aims to empower people to explore large-scale information bases but demands that people also take responsibility for this control by expending cognitive and physical energy. This paper outlines the basic motivations and concepts of HCIR and presents design goals and challenges that are informed by two ongoing HCIR projects.

The Basis for HCIR

The basis for HCIR lies in three classes of observations. First, information retrieval (IR) and human-computer interaction (HCI) are related fields that have strong traditions that have been challenged and energized by the WWW. The intersection of these fields offers interesting new opportunities for high-impact research and development, especially in WebIR and digital libraries. I suggest that integrating the human and system interaction is the main design challenge to realizing these opportunities and that what is required is design that recognizes a kind of syminforosis – people as organic information processors continuously engaged with information in the emerging cyberinfrastructure.

Second, content has moved beyond text to include statistics, images, music, video, computer code, sensor streams and biochemical sequences. These data are multimedia and multilingual and this expansion of data types challenges extant models of retrieval and interaction. More importantly, content has become dynamic. On the one hand, content changes quickly and continually, as blogs, wikis and real-time sensor streams illustrate. Additionally, content is increasingly conditional, as computed links, explicit or implicit recommendations, user-generated tags and click stream analytics illustrate. If an indexing algorithm must run overnight to create a new index for a corpus, the corpus will have changed by the time the algorithm completes. Furthermore, new kinds of content relationships are emerging that themselves represent new kinds of content. Hyperlinks and automatically generated metadata are obvious examples with aggregations and virtual collections from digital libraries as more subtle examples. The bottom line is that content now acquires history, and this history is important to retrieval and eventual use. This new reality gives rise to context-based retrieval.

The IR community has responded to some of these content changes. The most vigorous response is link analysis that aims to leverage the relationships among hyperlinks in the WWW. IR researchers now have multiple sources of evidence, such as authors’ words (for example, full text IR), indexer/abstractor words (OPACs), authors’ citations/links (ISI, CiteSeer, Google Scholar), readers’ search paths (recommenders, opinion miners) and machine generated features and relationships. This state of affairs leads to two key challenges for information scientists: What new kinds of relationships can we leverage (human and machine) and how can we integrate multiple sources of evidence (data fusion)?

Third, the installed base of users and their levels of information literacy continue to evolve. Television remote controls and website hypertexts have legitimized browsing as a form of human-controlled information seeking. Search engine business models have trained people to expect ultra-high precision query results for simply expressed (one- to two-word) queries. Today’s search engines offer services that are just good enough to whet desires. As people become accustomed to these low hanging fruits they will require understanding rather than retrieval. Achieving this understanding will require people to leverage their intelligence and effort – to assume responsibilities beyond the two-word, single query effort so typical today.

From the system design point of view this requirement implies that, rather than solely depending on matching algorithms, systems must focus on the flow of representations and actions in situ as people concurrently think and act with these new tools and information resources. HCI designers respond to these new kinds of expectations and skills by adapting classical IR techniques, such as relevance feedback and query expansion, to Web environments and by investigating user modeling to anticipate needs and customize results. Additionally, they leverage community-based actions to make recommendations aimed at helping people find and understand what they need. Designers find themselves constantly tuning IR systems to respond to changing content and changing user characteristics and needs. Finally, designers have developed new kinds of user interfaces, such as dynamic queries and agile views, that engage people more continuously rather than viewing the design problem as a turn-taking dialogue of type/wait/read/type actions.

A New Conception of HCIR

These changes lead to a new conception of human information interaction and information retrieval embodied in HCIR. In this model we think of information interaction from the perspective of an active human with information needs, information skills, powerful digital library resources (that include other humans) situated in global and local connected communities – all of which evolve over time.

This conception of HCIR suggests systems with a number of desiderata:

·        Systems should aim to get people closer to the information they need, especially to the meaning; that is, systems can no longer only deliver the relevant documents, but must also provide facilities for making meaning with those documents.

·        Systems should increase user responsibility as well as control; that is, information systems require human intellectual effort, and good effort is rewarded.

·        Systems should have flexible architectures so they may evolve and adapt to increasingly more demanding and knowledgeable installed bases of users over time.

·        Systems should aim to be part of information ecology of personal and shared memories and tools rather than discrete standalone services.

·        Systems should support the entire information life cycle (from creation to preservation) rather than only the dissemination or use phase.

·        Systems should support tuning by end users and especially by information professionals who add value to information resources.

·        Systems should be engaging and fun to use.

Challenges for HCIR Design

            This wish list for information systems reflects the mutual dependence of information seekers and systems and the evolving nature of both. The resulting complexities raise significant challenges for design. Most significantly, designers must find ways to closely couple the user interface to the system backend so that people interact with information rather than systems. Close coupling makes the overall system components less discrete and more transparent. For example, by including multiple indexing organizations in the backend to support alternative views in the front end, people can focus on examining pertinent information in different contexts that help understanding as well as retrieval. This general challenge spawns several more specific challenges such as designing and supporting alternative representations (for example, organizational layouts) and control mechanisms (clicks, hovers, scrolls, utterances) and generating rich metadata that will support a multiplicity of meaningful views. The grand challenge of HCIR design is thus making the many layers of UI, middleware, database and data acquisition transparent to the information seeker so that they may focus on how information matches their needs and how this information is applied or transformed.

            A second key challenge for design is raising levels of user literacy and involvement without insulting or annoying people. On-demand, multi-layered help that is under the control of people is the key solution trajectory in this regard. However, such help depends on extensive context-sensitive metadata as well as reliable models of human information-seeking behavior.

            Finally, there are significant evaluation challenges for HCIR. The IR measures of recall and precision and the HCI measures of time to completion and general satisfaction are not sufficient to assess information interaction except in the narrowest senses. We need ways to measure series of physical actions (such as click, buy, print. save, read, email) and ways to measure learning and behavior or attitude change. For example, complex constructs like satisfaction must be broken out into assessments of usefulness, learnability, usability, engagement and enjoyment, to list a few factors.

            Our work in the Interaction Design Laboratory aims to take incremental steps to meet these desiderata and confront these challenges. Two projects demonstrate some first steps in this direction. The systems have been well documented in the literature and can be seen at the respective websites (Relation Browser examples are at www.idl.ils.unc.edu/rave and the Open Video digital library is at www.open-video.org), so here I will only summarize the design principles that underlie these systems.

Relation Browser:  The Relation Browser aims to facilitate exploration of the relationships between and among different data facets such as topic, date, data type and geography. The interaction paradigm is to rapidly display alternative juxtapositioned partitions of the database with mouse actions. The backend structure is a relational database (thus requiring systematic metadata), and limited string searching is supported within partitions. The interface is meant to serve as an alternative to existing search and navigation tools and has inherent scaling limitations due to screen real estate, such as two or more facets with 10-15 categories per facet, and the need to have all possible query results on the client side to support the interaction dynamics. The metadata requirement has led us to investigate machine learning techniques to automatically populate the underlying database with mixed results (see www.ils.unc.edu/govstat/ for papers).

Open Video Digital Library.             The Open Video Digital Library provides a set of views for video content. These views are agile in that simple mouse actions control several alternative overviews for subsets of the collection and previews based on novel visual surrogates.

            Both of these examples illustrate the agile view design framework that gives users control over slicing and dicing the corpus and various partitions. The framework aims to do the following:

  • Provide people with look ahead without penalty
  • Minimize scrolling and clicking
  • Closely couple search, browse and examine functions
  • Offer useful attractors that stimulate continuous engagement
  • Bring information treasures to the surface rather than forcing many discrete actions to get to them.

Our experience is that user studies can inform specific design decisions and that a long-term iterative approach yields progress toward the more ambitious HCIR goals. These efforts represent a start and are ongoing.

Hopes that we can create systems (solutions) that do IR for us are unreasonable. Expectations that people can find and understand information without thinking and investing effort are unreasonable. Therefore, we aim to develop systems that involve people and machines continuously learning and changing together. Google would not work as well next month if there were not a large group of employees tuning the system, adding new spam filters and crawlers checking out pages and links continuously. Thus, the more theoretical aim of our HCIR research is rooted in the view that information interaction is a core life process. It is as important to life in an informated age as food and personal relationships. Our examples demonstrate some early ways to get the information seeker more involved in the information seeking process. However, plenty more must be done. As with eating, we have varying expectations, we invest different levels of effort, and we use diverse and ubiquitous infrastructures. We should aim to create flexible systems to span this variety without burdening people with choice overload.

Acknowledgements

            The ideas, user studies and systems upon which this lecture is based were supported by NSF Grants EIA 0131824 and IIS 0099638 and inspired by the many students and colleagues at the Interaction Design Laboratory at UNC-Chapel Hill and the Human-Computer Interaction Laboratory at the University of Maryland .