Unmediated and Mediated Information Searching in the Public Library

Ragnar Nordlie
Oslo College
School of Journalism, Library and Information Studies
Oslo, Norway

Abstract

This paper reports a study of 40 audiotaped reference interactions between librarians and public library users. The data are analyzed to identify patterns f interaction, ith a focus on users' problem formulation, librarians' elicitation of information from the user, unsolicitated information provided by the users and the effect of these factors on the outcome of the interaction.

The results are compared with studies of public library user behavior in unassisted OPAC searches. Users' initial query formulations are very similar in the two situations. In the mediated searches, ambiguities are resolved and users' information needs are determined not primarily through extensive, direct elicitation by the intermediary, but hile interacting with the material on the shelf. Replication of this functionality in the OPAC's interaction with the user would solve many problems in unassisted end-user searches.

Introduction

The inadequacies of the online library catalog (OPAC) as a tool to aid end users in their search for information have been explored in a number of studies, both through observation of users in experimental settings and through analysis of unobtrusively recorded search logs. The results of these studies have been difficult to summarize, as methods of investigation and definitions of key concepts like "search", "success" etc. have varied, and the catalogs under study have had very different functionalities. The prevalent impression, however, is that of a high incidence of user failure, particularly in topical searches, where failure rates are sometimes reported to be 50% or higher (see e.g. Peters, 1989, Hunter, 1991, for reviews of studies see Peters, 1993, Tonta, 1992).

Few studies have been conducted in Public libraries, those that exist report even higher counts of failed searches than those who study Academic library catalog use. The author's own transaction log analysis of 350 searches in a Public library OPAC (unpublished) shows an overall failure rate of 30%, with 44% of topical searches failing, whereas of the 1000 separate queries which make up the 350 searches, 60% of the total and 70% of the topic-related queries failed.

In analyzing the causes of these failures it has been pointed out (see e.g. Norgard et al., 1993) that most OPACs in current use in libraries are still, both in content and functionality, rooted in the card catalogs they replaced. These catalogs were essentially designed for mediation; the rules for their production and ordering were designed for the librarian rather than the library user. At the same time technical development, particularly in the field of communication, has rapidly changed the catalogs' environment, so that for many users of electronic catalogs mediation will no longer be available, they will be accessing the catalog from their home or their desk at work. The current generation of OPACs provides poor support for this kind of unmediated search.

It is interesting to note that with the rapid growth of the Internet and the ubiquity of presentation media like the World Wide Web and access mechanisms like Netscape, a long tradition of technology-driven OPAC development continues; numerous libraries now seem to rush to make their catalogs available via the WWW based on the technological possibilities, rather than on an assessment of user needs and capabilities.

One approach to generating the background knowledge necessary for a user-centered process of OPAC improvement is to study the interaction between users and human intermediaries in the information search situation, to see whether there are patterns of interaction which can be replicated in the electronic catalog's interaction with the user. A number of such studies have been conducted, almost exclusively, however, in an Academic library setting, and on searches in bibliographic databases rather than OPACs. Belkin, Brooks and Daniels (e.g. in Belkin et al., 1987) arrive at a categorization of the elements of user-intermediary interaction through discourse analysis of a limited number of database searches by trained intermediaries. Saracevic et al. (1990) analyze 40 DIALOG searches by four professional searchers, data from this study have been further analyzed by other researches, most recently by Spink et al. (1995), who studied the elicitations made by intermediaries in the course of the search interaction.

In the Public library setting, a much-quoted study by Lynch (1977) categorize the librarians' questions through content analysis of some 300 unobtrusively recorded reference interviews. A similar technique was used by Dewdney (1986), but in this case to study the effect of a particular kind of training on librarians' question behavior. In Denmark, studies by Ingwersen and Kaae (e.g. Ingwersen and Kaae, 1980) investigated various aspects of librarians' problem-solving behavior, including analysis of a few recorded user-librarian interactions. Mark Pejtersen (1986) used recorded interactions between librarians and users searching for fiction to develop a classification scheme for fiction, adapted to users' search strategies, and to design the "Book- house" interface for a catalog based on this classification.

Existing studies directed towards the particular search situation which one encounters in the public library are few and largely old. There is need for further research, particularly in the light of what we now know about users' problems in the unmediated search situation. The main purpose of the study which is in part reported here has been to attempt to identify patterns of user-intermediary interaction, in particular to see how and to what extent the intermediary establishes a model of the user and the user's information need, and to investigate whether elements of this modeling process may be expected to have an impact on problems identified in unmediated OPAC search sessions.

Method

Interactions between users and librarians in a medium-sized Norwegian pubic library (serving a user population of approx. 50 000 people) were collected during five days over a 14-day period. The interactions took place at the library's combined reference and information desk in the adult department, i.e. the place where the library patrons bring all their questions, be they directional, bibliographic, topical or just appeals for help with the copier. Interactions were audiotaped and supplemented with unobtrusive observation in order to record both the verbal and the non-verbal communication between user and librarian. Librarians carried the recording equipment to record the (substantial) part of the interaction which took place away from the reference desk. Users were notified of the recording by posters prominently displayed on the desk and told that they could require the tape turned off at any time. Nobody in fact required this, indeed none of the users seemed inhibited by the recording in any way.

A total of 170 interactions, involving six different librarians, were recorded. They do not represent all the transactions of the days in question, as at times more than one librarian manned the information desk, and as the recorder was turned off in quiet periods and was not always turned on for a question which obviously would not involve negotiation. As the distribution of types of users and types of questions varies somewhat over the year, for instance with heavier use by students/schoolchildren at midterm than towards the end of term (when this data sample was collected) the data is not a true random sample. The goal of the project, however, was to investigate a representative, not a random, set of user- intermediary interactions. As a control of this, the sample was compared with an inventory of user questions collected during a week at a different time of the year. The themes and types of questions in this inventory match the recorded data quite well. The age and sex distribution of the users in the sample also approximately match the distribution of registered borrowers of the library, with a slant towards the younger age groups.

The 170 interactions were categorized in the following groups:

Help to find particular titles 52
Help to find books by particular authors 8
Concrete problems/directional inquiries 10
Help to reserve books 9
Topical (subject) inquiries 86
Incomplete or technically faulty recording 5

This study focuses on the topical inquiries, which cause the majority of problems for users when they perform unmediated online searches. 40 of the topical inquiries were selected for transcription and analysis. The criterion for transcription was that some interaction was taking place between user and intermediary, other than the initial question and the final negotiation of the retrieved item(s). This interaction might for instance take the form of one or more question/answer sessions, or of unsolicited additional information from the user and response from the intermediary. In addition the transcribed transactions were selected to give an approximately balanced representation to all the librarians involved in the study. A somewhat simplified version of the standard conversation analysis transcription technique developed by Gail Jefferson (Atkinson & Heritage, 1984) was used for the transcripts.

Through content analysis, several aspects of the transcribed interactions were examined:

Since the data consist only of the recorded interactions, with no follow-up questions posed by the investigator to either user or intermediary during or after the sessions, both the determination of success and the categorization of the elements in the interaction are open to misinterpretation. The results are so consistent both between the intermediaries and between different types of users, however, that it seems justifyable to draw conclusions from the material.

Findings

Types of queries

As shown above, about half of all user queries were topical. These topical queries again fall into two groups, searches for factual information ("do you have books which show me how people were dressed in 1945"; "I want to see the colors of a sunflower") and searches for more comprehensive information about a subject ("books about recent developments in Russia"; "I am writing an article on various sexual minorities"). Though the distinction is not always easy to draw, about 1/3 of the queries were of the factual information type. There is no generally discernible difference in either length or pattern of interaction between the two query types, however.

General pattern of interaction

Interactions were of fairly long duration, the average length was 4.5 minutes, none were shorter than a minute and the two longest in the sample lasted 10 minutes. The interaction style of the individual librarians influenced the interaction length somewhat, the average varying from 3.75 minutes to 6 minutes between the librarians. The "traffic density" at the inquiry desk did little to influence interaction length, there is no significant difference in average length between busy and quiet times of the day.

The interactions follow a remarkably consistent pattern. Three phases can be identified: an initial problem presentation and clarification phase, a catalog consultation phase (during which problem clarification may continue) and a problem solution phase which in all but one of the 40 cases consists of the librarian going with the user to the shelves to negotiate a search result. The pre-search problem presentation phase is brief, lasting 15 seconds on the average and in only one case lasting longer than 30 seconds. In about 1/3 of the cases the interaction moves directly from the problem presentation phase to the shelf. In the remaining 2/3, where the problem presentation is followed by a catalog search, the duration of this search varies with the librarian's knowledge of the collection and the catalog, but it is rarely a question of entering more than one search term. In any case, the shelf negotiation phase is by far the longest lasting. On the average, it occupies approx. 75% of each interaction, and only in 4 cases does the time spent at the shelves make up less than 50% of the total interaction time.

User's problem statement

The users' initial problem statement show a remarkable similarity to the kinds of initial query terms which are input by Online catalog users. They are normally brief, most often consisting of one sentence in which the problem is stated as a single term or expression. They are mostly less spesific than the user's real information need ("I am looking for something written about people from Telemark [a county in Norway]" turns out to be a search for a humorous quotation to be used on the invitations for an anniversary celebration for a couple who hail from a spesific village in Telemark), and they often need disambiguation ("..something on insects, trees and flowers" is a request from a kindergarten teacher wanting to know what he will find when he takes the children out on a nature trail walk).

Intermediary elicitation

The focus of the information elicitation by the intermediary changes during the course of the interaction. In the first phase, the problem presentation, the main purpose of the librarian's question is to understand, disambiguate and focus the user's problem. Very little elicitation take place during the catalog search, while in the resolution phase at the shelf the questions mainly serve to specify, select and verify the answers to the information need.

In the light of the characterization above, it is remarkable that in only 18 out of 40 cases did the intermediary elicit additional information after the user's first problem statement before turning to the shelf or to the catalog, and in half of these cases again only a single question was asked. As has been shown, this does not mean that the users' initial problem statements present a clear and complete representation of the problem at hand, on the contrary, by the end of the interaction the problem as presented in the initial statement had been changed or modified in 60% of the cases. It is rather a reflection of the role of the catalog search and of the search at the shelf in the total interaction. This will be further discussed below.

When all three interaction phases are considered, the intermediary asked 2 questions on the average during an interaction, or 2.5 if follow-up questions which just echo or reiterate the user's statement are counted. The questions were distributed as follows:

Subject definition (specification, fact gathering or instanciation through particular titles): 50%
User objective or motivation: 5%
Personal characteristics of users (language restriction, level of difficulty, urgency, amount of material needed etc.): 12.5%
User status in relation to system (sources checked, catalog use, previous browsing): 10%
Verification of acceptable answer: 22.5%

The main part of the subject definition elicitations serve a disambiguation purpose ("Is it something in connection with genealogy you are working on?" is the question triggered by the Telemark problem statement quoted above); questions pertaining to the librarian's factual knowledge (of the type "What are martial arts?") are less frequent. More than 80 % of the subject definition elicitations have the form of closed questions.

Less than 20% of the intermediaries' elicitation are directed towards establishing a model of the user, leaving aside the "verification of acceptable answer" category of elicitation, which might in a difficultly definable way contribute to such a model ("is this what you are thinking of?"), but which in most cases takes place at the very end of the interaction.

Information provided by the user

In addition to answers to intermediary elicitations, users contribute direct, unsolicited information relevant to the intermediary's model of the user or the problem approximately once in each interaction. These items of information refer (in almost equal parts) to subject definition/specification, desired form/medium, purpose or use of materials. In most cases these unsolicited contributions seem to be triggered by titles the user is shown, either in the catalog search or during the shelf browsing. Neither the triggering conditions nor the use the intermediaries make of these contributions have been fully investigated, however.

Intermediary catalog search

In the 2/3 of the cases where the problem elicitation phase continues in a catalog search, the search terms used are almost invariably general rather than specific. Users tend to initially present their query in general terms and then specify it either through intermediary elicitation or unsolicited comments, but in most cases, the intermediaries use the users' initial formulation as search term. Even in cases where the user's initial formulation is fairly specific ("Do you have books on repair of fiberglass boats") and a subject heading of matching specificity exists in the system ("Boats and boatbuilding - fiberglass") the librarian chooses to search on a general term ("Boats"). This might, of course, be due to the librarian knowing that the system is inconsistent (in the above example, books on fiberglass boats are also found under the subject heading "Boats and boatbuilding - plastic"), but the pattern is so consistent that the reason must rather lie in the librarian's percieved purpose for the search. This purpose seem to be to provide one or more entry points for shelf browsing rather than to find explicit titles. In all but two cases the search is abandoned without any attempt at specification, and the interaction continues at the shelf.

User satisfaction

Without access to direct user assessments it is impossible to make definite statements about the success of the interactions. The final statements from the users vary from "well - I've got something" via "I guess I'll look at these two" to "great - thank you!". In only one case did the librarian fail to retrieve any material when there was actually relevant documents in the library, and in one case the library did not hold the requested material. In all the remaining cases the users found at least some material relevant to their query, and in more than 80% of the cases the final statement may be interpreted as indicating that the user was either "fully" or "quite" satisfied. This is in marked contrast to the author's inquiry of users' online search behavior, where more than 30% of all subject searches (and more than 60% of all individual subject queries to the system) ended with no documents retrieved.

In so far as the terms precision and recall are meaningful in this connection, the users without exception ask for precision rather than recall - they are interested in something about their subject of interest rather than everything about it. Most frquently the user terminates the transaction, with comments like "then I think this will do, thank you". On the other hand they generally show quite a lot of patience during the browsing phase and accept being taken from shelf to shelf to judge titles as the librarian's chain of associations develop.

Discussion

The purpose of this study was to investigate how patterns of interaction between public library intermediaries and users may provide insights applicable to the improvement of online catalogs for unmediated search. A related, unpublished study by the author has sought to gain insight in interaction patterns in unmediated search. A comparison between these two studies show that the distribution of types of user query is quite similar, and that the initial query formulation of the users also show similarities, with a tendency towards a rather general initial formulation where, at least in the case of the mediated search, the underlying information need can be shown to be more spesific. The great difference between the two is the rate of success of particularly the topical searches.

It was assumed at the outset of this study that a process of model-building on the part of the intermediary would be identified, and that the elements of this model-building might be isolated. Such model-building is obviously taking place, it is impossible for the intermediary to engage in problem- solving interaction without having and developing a model both of the user and the user's problem. The nature and frequency of intermediary elicitations show that the main emphasis in the interaction is on the modeling and understanding of the user's problem.

The study indicates, however, that the intermediary's model of the system, represented by the organization of the catalog and of the documents on the shelf, and the intermediary's model of the world, represented by general knowledge and in particular by knowledge generated through queries encountered in the past, play a large role in the i nteraction. A relatively modest amount of information elicitation takes place, both pertaining to the subject of the query and the situation of the user. The catalog search seems to serve the purpose of matching the query to the intermediary's world and collection model and provide a focus for the further search rather than to develop a detailed understanding of the query. The overwhelming predominance of closed questions from the intermediary seem to point in the same direction. The greater part of the interaction takes place at the shelf, where the user is exposed to the system model. It seems to be as much through participating in this exposure and observing the user's reactions to it as through direct elicitation that the intermediary establishes the user and problem model and is able to assist the user to a successful completion of the interaction.

More research is needed to establish if this is indeed the case. The next sep might be to interview intermediaries to gain insight in their perception of the process and the models they posess and establish. It may be that the nature of the queries and the collection in question is conductive to the interaction process the intermediaries are observed to chose - the degree of specificity and constraints on the queries ("must have illustrations","must not be too difficult" etc.) in many cases demand that the documents are examined to determine their value to the user. An interesting follow-up study would be to examine in more detail how the user and the intermediaries work with the documents on the shelf.

The findings indicate some fruitful directions for online catalog improvement. The author's and other OPAC user studies show that OPAC users have difficulty in applying any but the most simple search functions. Their query formulations are often ambiguous, do not convey the real information need and do not match the language of the system. OPAC users also seem to have difficulties in applying information from the document descriptions they do retrieve in a search to improve their search strategy, for instance by choosing new subject headings or keywords. These problems might be alleviated if the OPAC could:

Acknowledgement

The research reported here was conducted while the athor was visiting Fulbright Scholar at the Alexandria Project Laboratory, School of Communication, Information and Library Studies, Rutgers University.

References

Atkinson, J. M. And Heritage, J., eds. (1984). Structures of social action. Cambridge, Cambridge University Press.

Belkin, N. J. Et al. (1987). Knowledge elicitation using discourse analysis. International Journal of Man-Machine Studies, 27, 127-144.

Dewdney, P. (1992). Recording the reference interview: a field experiment. In Glazier, J. D. And Powell, R. R., eds. Qualitative research in information management, Englewood, Co., Libraries Unlimited, 122-150.

Hunter, R. N. (1991). Successes and failures of patrons searching the online catalog at a large academic library: A transaction log analysis. RQ, 30,395-402.

Ingwersen, P. And Kaae, S. (1980). User-librarian negotiations and information search procedures in public libraries. Analysis of verbal protocols. Final report. Copenhagen, Royal School of Librarianship.

Lynch, M. J. (1977). Reference interviews in public libraries. Ph. D. Dissertation, Rutgers, The State University of New Jersey.

Mark Pejtersen, A. (1986). Design and test of a database for fiction based on an analysis of children's search behvior. IN Ingwersen, P. Et al., eds. Information technology and information use: toward a unified view of information technology. London, Taylor Graham, 125-145.

Norgard, B. A. Et al. (1993). The online catalog: From technical services to access service. Advances in librarianship,17, 111-148.

Peters, T. A. (1989). When smart people fail: An analysis of the transaction log of an online public access catalog. Journal of Academic Librarianship, 15(5), 267-273.

Peters, T. A. (1993). The history and development of transaction log analysis. Library Hi-Tech, 11(2), 41-66.

Saracevic, T. Et al. (1990). Nature of interaction between users and intermediaries in online searching: A qualitative analysis. Proceedings of the 53rd Annual Meeting of the American Society for Information Science, 27, 47-54.

Spink, A. Et al. (1995). Search intermediary elicitations during mediated online searching. Proceedings of the 58th Annual Meeting of the American Society for Information Science, 32, 97-102.

Taylor, R. (1968). Question negotiation and information seeking in libraries. College and Research Libraries, 29, 178-194.

Tonta, Y. (1992). Analysis of search failures in document retrieval systems: A review. Public-Access Computer Systems Review, 3(1), 4-53.

© 1996, American Society for Information Science. Permission to copy and distribute this document is hereby granted provided that this copyright notice is retained on all copies and that copies are not altered.