Shifts of Focus in Information Retrieval Interaction


David Robins
Louisiana State University, Baton Rouge, LA


Abstract

During the course of information retrieval interactions, users may change the focus of their attention to various aspects of their information problem. These changes in focus, or interaction shifts are the subject of this paper. This phenomenon is readily observed in mediated database searching, from which careful analysis of the dialog among users, search intermediaries, and information retrieval (IR) systems reveals changes of search/interview focus. Transcripts of mediated information retrieval interactions are analyzed to describe interaction shifts. This paper presents preliminary results of the author's dissertation that investigates information retrieval interaction.

INTRODUCTION

During the course of information retrieval interactions, users may change the focus of their attention to various aspects of their information problem. These changes in focus, or interaction shifts are the subject of this paper. This phenomenon may be readily observed in mediated database searching, from which careful analysis of the interaction among users, search intermediaries, and information retrieval (IR) systems reveals changes of search/interview focus. This paper presents preliminary, descriptive results of the author's dissertation that, in part, examines shifts in information retrieval interaction.

A shift of focus (hereafter, shift) is defined as any change in the topic of discussion (however brief or lengthy) in a conversation between a user and a search intermediary during the time they work together seeking information. Shifts are manifested in a variety of types and for various functions. This paper will describe the types of shifts and the functions as derived from observation. As a guide for observation, shifts are characterized by changes of focus such as:

The importance of this problem lies in a need for more detailed understanding of the process of IR interaction. An analysis of interaction shifts should provide data detailing the various topics of conversation between user and search intermediary, and how the participants shift their focus among these topics.

RELATED LITERATURE

The theoretical framework of the paper is based on the notion of rational/intentionalist information seeking behavior on the part of users with goals (Robins, 1996). The framework derives from work done in the field of cognitive psychology by J. R. Anders on (1990). This rationalist approach is characterized by the analysis of goals and environment in order to explain behavior. Intentionalist theories of mind hold that systems may be "explained and predicted by relying on ascriptions to the system of belie fs and desires (and hopes, fears, intentions, hunches...)" (Dennett, 1971, p. 191). Put together and applied to studies of information behavior, these conceptions of human cognition form a theoretical framework helping to explain the paths taken by humans seeking information.

Although a user's behavior may be intentional and rational, it may not be linear or appear logical. Newell (recounted in Anderson, 1990) states that even amateur chess players have perfect knowledge of how each piece on the board may move. Even so, people play less than perfect games. Why? Because, cognitively speaking, the costs of calculating all possible moves (to determine the most efficient checkmate), even in the short term, is too great for most if not all players. Players exhibit rational behavior by optimizing the resources of time, attention, and energy.

According to this framework, users (and human search intermediaries) tend to optimize scarce resources such as attention (Cooper, 1973), time, and money (in the case of online searching). Those seeking information are faced with ill-defined cognitive demands. A chess player has the constraints of the board and the rules governing the movement of pieces. A person seeking information has constraints but they are far more nebulous. Ingwersen (1996) has presented a range of possible information seeking behaviors based on two factors: (1) clarity of information need definition; and (2) the stability of the information need. At one extreme of the matrix, a well-defined, stable information need produces information seeking that is focused on specific resources, and contains limited uncertainty. The other extreme is an information need that is ill-defined and varies over time. Such a state generates behavior characterized by high uncertainty and browsing. This hypothesized variance in behavior has parallels in Anomalous States of Knowledge (Belkin, Oddy and Brooks, 1982).

Given the discussion to this point, one might hypothesize the number of shifts to vary during any single interaction as a user's information need varies within Ingwersen's (1996) matrix. In other words, participants may need to address more facets of an i ll-defined need that varies over time than they would at the other extreme. This hypothesis would be consistent with a rational view of information behavior. A user attempting to optimize return on his/her time/attention/energy investment would probably b e driven by uncertainty to consider more options (hence, shift from one discussion facet to another) if his/her information need is less stable and more ill-defined. However, it is not yet know if this hypothesis is true.

Researchers in the past have attempted to perform basic research on IR interaction through dialog analysis. Such studies attempt to explain information behavior through observation of transcripted dialogs between users and search intermediaries as they interact with an IR system. Saracevic (1989) proposed a triadic model of IR interaction made up of: (1) a user; (2) a search intermediary; and (3) an IR system. Although since 1989 a revolution in end-user searching has occurred (primarily on Web-based systems and CD-ROM databases), there are three arguments for maintaining this triadic model. First, it has not been determined that increases in end-user searching has produced better searching. Time and research will tell whether such notions are true. Second, mediated searching on proprietary (and thus, expensive) databases continues. Although natural language processing has improved and is a viable alternative to traditional Boolean searching (thus making end-user searching more viable), end-users working on their own may still wonder if they have gotten adequate search results (especially where money rides on not missing something important). Third, it may be argued that all searching is mediated in some way. That is, interfaces and algorithms are created with certain assumptions (founded or unfounded) about users. End-users, therefore, use an IR system within the inherent design constraints. These constraints, such as ease of access (or access at all) to a controlled vocabulary constitute intermediary functions, and therefore fit Saracevic's triadic model. It is within this triadic model that the present study aligns itself.

With respect to interaction in general, Grosz and Sidner (1986) present a comprehensive theory of discourse structure. They show that segmentation of discourse is occurs naturally in any discourse. Such segments are bounded by "cue phrases" that indicate a change in a conversation's focus. These cue phrases vary from the subtle to the obvious, and therefore discourse must be studied carefully to perceive such cues. Grosz and Sidner argue the existence of three components of discourse structure: (1) linguistic structure; (2) intentional structure; and (3) attentional state. Linguistic structure may be thought of as naturally occurring segmentation of discourse (shifts of focus). Linguistic structure has syntactic qualities, as opposed to intentional structure, or the meaning/purpose of each discourse segment. The attentional state is a device for displaying the properties and objects of discourse segments, and their relationship to other segments.

Grosz and Sidner (1986) hold that discourse segments are not necessarily related to one another linearly as they occur temporally. Any discourse segment may be predicated on (i.e., related to) any prior segment (not necessarily the one immediately preceding it). The reasons for such nonlinearity of communication are varied. For example, Interruptions may break the flow of discourse. Similarly, one of the speakers may take an aside, and return subsequently to his/her previous focus. For this paper, I chose to show only linear flow of shifts as they naturally occurred during the interactions. However, future studies will include such an analysis.

Belkin (1984) showed how intermediary functions varied within ten utterance groups. By displaying ten utterance intervals on the "X" axis of a graph, and the intermediary functions on the "Y" axis, he was able to show what functions were invoked and when they were invoked. However, the 10 utterance blocks were an arbitrary division of the interactions. Brooks (1986) attempted to describe shifts in a more meaningful way by looking for cues, or "frame words" (Sinclair & Coulthard, 1975), that indicate chang es of focus in dialog. Frame words are used by a participant in a conversation to set apart all subsequent dialog as belonging to a different focus (until another frame is established by a participant). Examples of frame words are "okay," "now," "right," "good," or "so." Once a new frame had been established, Brooks would then classify each frame by its function in the dialog. One of the limitations of Belkin's (1984) and Brooks' (1986) work is that it focuses exclusively on interaction occurring before actual searches are performed, i.e., the presearch interview. None of their data shows what happens once the search begins. Other studies (Spink, Goodrum, Robins & Wu, 1996) have shown that significant modeling and rethinking of tactics, etc. occurs after the search begins.

It is somewhat surprising that interaction shifts are so neglected in IR research. Since Brooks' (1986) dissertation, there have been no direct studies on such phenomena. This study represents a continuation of such efforts. The ultimate goal is to better understand the nature of IR processes.

RESEARCH QUESTIONS

This study was guided by the following research questions:

  1. How many shifts occur during various IR interactions?
  2. When do such shifts occur in the interaction?
  3. What are the general types of shifts that occur during IR iterations?
  4. What are the specific functions of such shifts?
  5. How do the types and functions of shifts relate to participants' (user and search intermediary) cognitive states?
RESEARCH DESIGN

The aim of research in this paper is to describe the nature of interaction shifts between search intermediaries and users. The following research design is used to illuminate such shifts.

Data Corpus

The data were collected during a previous study by Saracevic and Su (1989). The study was funded by a grant from the Library Research and Demonstration Program, United States Department of Education (ref. no. R039A80026), with additional funding by DIALOG, and entitled, Nature and Improvement of Librarian-User Interaction and Online Searching for Information Delivery in Libraries. The data consists of transcribed discourse (originally videotaped) between real users and professional search intermediaries during authentic information retrieval interactions. All of the users were either graduate students or faculty in pursuit of some particular research goal. In summary, 40 searches were taped, amounting to over 46 hours of video. Each search averaged nearly 70 minutes (13 minutes presearch; 56 minutes online). Four search intermediaries participated in the study, each averaging over 8 years of search experience. The searches covered a wide variety of topics (46 different databases were used).

For this study, six of the 40 transcripts are analyzed. Both user and search intermediary utterances are included in the analysis. The transcripts are drafted as utterances. That is, they are written in the form of:

Speaker A: speaks until interrupted...
Speaker B: interruption
Speaker A: ...continued statement after interruptions.

The above interaction consists of 3 utterances.

Methodology

Six of the above mentioned interactions were analyzed in detail for this preliminary study. The goals prescribed by the research questions require that the methodology do the following:

  1. Identify shifts within each dialog;
  2. Classify each shift according to its type and function;
  3. Quantify the number of shifts and utterances in each shift;
  4. Each of these facets of the methodology is discussed in the following sections.
Identification of shifts

Interaction shifts are defined as any change in focus of the conversation between the user and search intermediary with respect to the user's information problem. Interaction shifts are any change in focus of the interaction between a user and a search intermediary with respect to the user's information problem. Change of focus is denoted by a change in some topical aspect of the conversation (by shifting to a different aspect of the topic, or by broadening or narrowing the topic, etc.), or by some shift to a non-topical aspect of the information problem (again by narrowing or changing). In other words, a shift may occur between or within topical or nontopical discussions, or, shifts may occur laterally or horizontally within a topical or non-topical discussion.

Brooks (1986) identified "focus shifts" (p. 85) in her analysis of user modeling functions of intermediaries in pre-search interviews. She found that intermediaries initiated most coding shifts, a notion consistent with other discourse analysis literature which suggests that shifts in dialog are initiated by participants with higher status (Grosz, 1981). Brooks used, in part, certain dialog cues to identify the points at which shifts took place. These cues have been referred to as "frame words" (Sinclair & Coulthard, 1975). For example, such cues might be utterances which contain frame words such as "well," "now," "right," "ok," or "good." Such words in (particularly at the beginning of) an utterance may indicate that the speaker has begun to think about changing the focus of the discussion.

Classification of shifts

Shifts are to be classified according to their type and function. Shifts types refer to a broad class of shifts to be determined as they emerge from the data. Types of shifts concern the source of motivation behind the shift. Shift functions refer to the specific purpose of each shift. The exact nature of types and functions are determined as they emerge from the data consistent with principles of qualitative, grounded theory research (Glaser & Strauss, 1967). However, another facet of the author's dissertation deals with the classification of individual utterances (Robins, 1996). It is assumed that the categories assigned to the individual utterances within a shift give evidence of the shift's function. Yet, each shift is analyzed as a whole, which may or may not be the sum of its parts. In order for this analysis to be considered reliable, it is necessary for the transcripts to be analyzed by more than one coder. At the time this paper was being prepared, four other coders are working on transcripts to determine overlap rates for the application of shift function and type codes.

Quantification of shifts and utterances within shifts

In order to show a clearer picture of the nature of shifts in IR interaction, it is necessary to provide an ac#ount of the number of shifts that occur in each interaction, and in aggregate. Accordingly, shifts are to be tabulated by presearch, online and total, for types and functions of shifts.

RESULTS

Results of the study are presented in the following order:

  1. Frequency of shifts in each interaction (i.e., transcript);
  2. Classification scheme for shifts;
  3. Frequency of shifts for each class;
  4. Intensity of focus on each shift.

Each of these is discussed in the following sections.

Frequency of Shifts in Each Interaction

As Table 1 shows, the majority of time in each interaction is spent after the participants go online. Therefore it is not surprising to find in Table 1 that in all cases, there are more shifts in the online phase of the interaction than in the presearch phase.

Phase/Transcript T002 T003 T004 T005 T006 T007 Totals
Presearch 20 9 11 17 11 14 82
Online 30 95 28 28 44 17 242
Totals 50 104 39 45 55 31 324
Table 1. Number of shifts in each interaction with total

Classification Scheme

The following codes were derived from an analysis of the data within each shift. There are two levels of coding for the analysis of shifts: (1) functions of shifts; and (2) types of shifts. Each is discussed in the next two sections.

Functions of Shifts


The functions of shifts in IR interaction describe the what the participants were trying to accomplish during any particular shift. Functional codes may be thought of as sub-goals (Belkin, 1984) of the entire interaction. These functions were derived directly from the interaction data. However they are based on a coding scheme (also derived from the same data) developed for coding individual utterances (Robins, 1996). Simply stated, functions are the "how" with respect to accomplishing the goals of the search. Each is described in Table 2. In all, 16 functions were identified in this study. Some are directly related to formulating search tactics while others focus on gathering background information. Still others are more concerned with search logistics. These meta-functions (types) are discussed in the next section.

Function Description
AVAIL Determine how a user will ultimately obtain full text versions of retrieved text; usually occur during printing
COST Discussion regarding the costs of the search, the full text of documents, or other related expenses
EVAL (+, -) Judgments regarding the relevance of system output (+ = overall positive; - = overall negative; none = unclear)
FORMAT Discussion regarding the format of document representations (e.g., full record, titles only, etc.), or full text (e.g., microfiche vs. paper)
I Indiscernable passage
LIT Discussion regarding the domain literature of the search
MAG Discussion regarding the number of items in a given retrieved set
PRINT Discussion pertaining to printing system output
PRIOR Discussion of prior searching on, or knowledge of, the topic at hand by either the user or the search intermediary
Q Discussion regarding the actual entry of the query into the system
R (<, >) Discussion of a query reformulation (< = narrowing; > = broadening; none = lateral)
SCOPE (<, >) Determine the boundaries and scope of the desired information (< = narrowing; > = broadening; none = lateral)
SNSR Discussion of social issues NOT related to the search
SSR Discussion of social issues related to the search
ST Discussion related to the experiment itself (e.g., videotaping)
STRAT (<, >) Concerned with the strategies leading to query formulation or reformulation (< = narrowing; > = broadening; none = lateral)
STRPR Discussion of problems associated with search strategy
STREVU
SYS Explanations, preparations, or problems with the IR system itself
TECH Discussion of technical issues related to the equipment (computers, etc.) associated with the search
TERM (<, >) Regarding choice of terms; definition/clarification/resolution of terms (< = narrowing; > = broadening; none = lateral)
TERR Technical errors such as typographic errors
TOP (<, >) Discussion of the specific subject area guiding the search (< = narrowing; > = broadening; none = lateral)
Table 2. Function codes and descriptions for interaction shifts

Types of Shifts

If functions are the "how" of accomplishing search goals, then search types are the "what" of the process. That is, types are the dimension of each shift that describes the deeper motivation of the participants (i.e., they describe what is going on at this deeper level). There are four types of shifts: (1) Conceptual; (2) Operational; (3) Administrative; and (4) Other. Each type is described below.

Conceptual. Shifts that relate to, or assume/require knowledge on the part of, participants are conceptual shifts. Examples of functions that tend be conceptual in nature are: EVAL (because of the need for some knowledge state in order to judge/evaluate retrieved text; PRIOR (because it requires participants to discuss existing knowledge states); TERM (if the context is one in which knowledge states are evoked to explain or understand terms); SCOPE (if the context is one in which knowledge states are evoked in order to explain or understand the parameters of the search); and STRAT (if participants discuss strategy from the standpoint of its relationship to the user's information problem, as opposed to an immediate strategic action). STRAT is more likely to be an operational type than a conceptual one.

Operational. Operational shifts are concerned with the procedural aspects of the search itself. Any functions related to the conduct of query planning, formulation, or reformulation, should be grouped as operational. Examples of functions of this type are: STRAT (if a discussion of what steps to take at a particular point in time); MAG (because of its relationship to narrowing/broadening of the search); Q (because of its direct relation as a query; R (because of its direct relation to search procedures); and TERM (if discussed as a means of strategic implementation, e.g., to add to a query).

Administrative. Shifts involving discussion of activities related to the search, but not conceptually or operationally linked to it are administrative shifts. Examples of administrative search functions are: SYS; AVAIL; PRINT; and COST.

Other. Non-search related shifts fall into the category, "other." Examples of other functions are: SOC (if non-search related); and ST.

Types are a higher level description of shift processes than are functions. Types represent a broader class of functions. In other words, functions tend to be of a certain type, but it is not necessary for them to be assigned to only one type. For example, the SCOPE function may be either conceptual or operational depending on the context of the shift within the interaction. Consider the following focus segment from transcript 002.

U: primarily I don't wa:nt to start research um (0.8) towards- towards my uh dissertation (0.5) and find out a year down the road that hh I'm reinventing the wheel hh
I: so you want just to be very comprehensive
U: uh hum also we don't think that anybody- I mentioned chicks when I had the request because I don't think anybody has used chicks as a model for this. chicks t-t- turned out to be a very good model for heart disease. really a much better model than rats, so we know that in a lot of ways they're- they're like humans for modeling but no one uses them everybody traditionally uses rats (0.3) but we think that this may be a novel approach to looking at things

This shift must be coded as functionally related to scope, and as a conceptual type. The user is explaining why s/he wants the search limited to chicks and to be very comprehensive. The reasons are spelled out in a way that directly relates to the conceptual basis for the search.

Conversely, the focus segment below (from transcript 005) illustrates how a discussion of scope may be operational in type. Notice that the search intermediary (I) is quizzing the user (U) in order to get enough information to structure the query without regard for the conceptual basis for the scope of the search.

I: good. hhhhhh okay why'nt you tell me a little bit about. okay 0let me see0 um- a couple of questions how um- how far back you know in time are you interested
U: about f[ifteen years ]=
I: [for this topic fifteen years]
U: =back to about 1970 on
I: okay so we might as well search the whole data base that will go back to 1966
U: oh great
I: and- are you only- do you wanna limit to English also
U: yes
I: okay uh how about uh. you want to limit to human also
U: yes
I: okay. okay. so we can do that from the very beginning let me just put that (typing 4.0))
I: cause that we can do right off the bat ((typing 3.0)))) okay. so the whole search will be limited to english and to human
U: >good<

The following example gives an indication how discourse may be broken down into segments representing shifts of focus. The point of shift from one topic to another is indicated by a horizontal line.

I: You will see. You will see. (Printing) This is going to showing you the titles of the articles and what are the index terms they are put under (Printing...) and as soon as it stops we will go back. (Printing, Intermediary scrolled back the screen.....) O.K. This is the first one, uh.. this is the title, current status of fall armyworm host strains.. and then these are the indexes, the index terms and key words and these are the different concept codes they put under .. put in. Uh.. can you tell by the title is this something that's relevant for you..
U: Uh....not from that particular title..
I: for you? O.K. Let's look at the next one.
U: Doesn't it have author's name in there?
I: O.K., that will be when we actually get to the right combination we will be printing out with the author, references and the abstracts.. oh, this is just a test format. Just to see if we are in the right area.
U: Um...
I: (Scrolling back the screen) How about this one?
U: It's hard to say (smiling).
I: O.K. Can you tell by the.. See, these are different.. oh here, they have something called phytopathology- parasitism and resistance..Will that.. that's not?.. that's not..
U: No, not necessarily what I am looking for.
I: O.K. (keying in on computer) Oh, this one is pasticide resistance.. Well, you are the one that has to tell me which items are good, which are not, you know, relevant, because that's how I can adjust it.
U: I am not.. (laughing) I don't know.
I: Do you .. do you have a known paper? I mean, do you have a specific paper..that on that's like what you want to get? because I can call that up and see how it's indexed. (keying in..)
U: O.K. I am not sure. You can do an author search on..
I: O.K.
U: Brattsten, that should be at least one paper that's is in that area. I just don't remember, because that's more than 10 years ago.
I: O.K. Let me try. What's the name of the author?
U: Brattsten.
I: How do you spell his name? B..
U: R A T T S T E N.
I: Do you have his first initial?
U: BRATT..
I: Oh, BRATTSTIN.
U: TEN.
I: ST?..
U: EN.
I: O.K. Do you have the first initial?
U: L
I: O.K.... O.K. Let me try that. (keying in..... computer printing) 15 items, let me see if there is any...None of his papers are in that.. this larger set, the 118. You said, you thought that he had something that has to do with cyanide? Let's try no.5..(typing, and computer printing..) O.K. Let's look at these two... O.K. This one.. see the index terms are under cyanide, glycoside, cyanogenic glycoside..linamarin.. These are the things that it's indexed under.
U: All right.
I: Concepts such as invertebrate, insect physiology.. uh.. animal ecology, general biochemistry, biochemical studies, proteins, peptides and amino acids...

Each of the segments in the example above are separated because one of the participants changed the focus of the discourse. For example, the transition from utterance 28 to 29 is a case in which the search intermediary shifts from talking about a specific search term (utterances 17-28), to reformulating a query and extracting new terms from the results (29-31).

Frequency of Shifts for Functions and Types

The coding scheme described above was applied to the shifts in the six transcripts. The resulting data are tabulated according to the total number of functions and types that occurred during the presearch and online phases of the search. Table 4 shows the number of functions and Table 5 show the number of types. These tables are arranged so that functions and types are ranked in descending order by totals. Note that the functions in Table 4 are not specified as broader, narrower, positive or negative. A more detailed graphic is shown in the next section. Obviously, certain shift functions are more likely to occur in either the presearch or online phases of the search. Notably, evaluations (EVAL) of documents, reformulations of queries (R), queries (Q), and output related shifts (AVAIL, FORM, PRINT, etc.) do not occur in the presearch phase of an interaction.

Frequency Percentage
Code Presearch Online Total Presearch Online Total
STRAT 22 56 78 7% 17% 24%
TERM 20 29 49 6% 9% 15%
EVAL 0 39 39 0% 12% 12%
TOP 14 14 28 4% 4% 9%
R 0 24 24 0% 7% 7%
SYS 6 17 23 2% 5% 7%
MAG 0 20 20 0% 6% 6%
SCOPE 10 1 11 3% 0% 3%
FORMAT 0 7 7 0% 2% 2%
AVAIL 0 6 6 0% 2% 2%
SNSR 2 4 6 1% 1% 2%
TECH 1 5 6 0% 2% 2%
ST 1 4 5 0% 1% 2%
LIT 3 1 4 1% 0% 1%
SSR 0 4 4 0% 1% 1%
STREVU 0 3 3 0% 1% 1%
COST 0 2 2 0% 1% 1%
PRIOR 2 0 2 1% 0% 1%
Q 0 2 2 0% 1% 1%
STRPR 0 2 2 0% 1% 1%
I 1 0 1 0% 0% 0%
PRINT 0 1 1 0% 0% 0%
TERR 0 1 1 0% 0% 0%
Total 82 242 324 25% 75% 100%
Table 3. Frequency and percentage of shift functions

Spink, Goodrum, Robins, and Wu (1996) found that a majority of elicitations in IR interaction dealt with search terms/strategy, and output relevance. Similarly, this study finds that shifts focus on evaluation judgments (EVAL), search strategy (STRAT) and terms (TERMS), and query reformulation (R) 63.3% of the time. The other 12 functions range from 7% to 1.3% occurrence.

Operational and conceptual shift types constitute 89% of all shifts (61% and 28% respectively). The participants did not turn their attention to administrative or other concerns with any frequency until the online portion of interactions.

Frequency Percentage
Type Presearch Online Total Presearch Online Total
Operational 48 150 98 15% 46% 61%
Conceptual 30 60 90 9% 19% 28%
Administrative 0 18 18 0% 6% 6%
Other 4 14 18 1% 4% 6%
Total 82 242 324 25% 75% 100%
Table 4. Frequency and percentage of shift types

Intensity of focus on each shift

Another comment about the data regards the rate of utterances per shift (Table 5). The importance of this ratio is that it gives an indication of the time spent by the participants discussing a given topic. There were roughly the same average intensity of discussion during the interactions.

A: No. Utterances per Interaction B: No. Shifts per Interaction A/B: Utterances per Shift
Presearch Online Total Presearch Online Total Presearch Online Total
T002 130 143 273 20 30 50 6.50 4.77 5.46
T003 64 663 727 9 95 104 7.11 6.98 6.99
T004 193 246 439 11 28 39 17.55 8.79 11.26
T005 84 309 393 17 28 45 4.94 11.04 8.73
T006 73 232 305 11 44 55 6.64 5.27 5.55
T007 59 106 165 14 17 31 4.21 6.24 5.32
TOTAL 603 1699 2302 82 242 324 7.35 7.02 7.10
Table 5. Utterances per Shift by Interaction (T)

DISCUSSION

It is not surprising that most of the participants' efforts went toward strategies, evaluation, term selection, and query reformulation. These activities are primary given the circumstances of the searches in this study. The searches were directed toward reasonably distinct goals. Each of the six users was a graduate student and five of them were working on dissertations. In general, the interactions would fit into Ingwersen's (1996) matrix as well-defined and stable.

What is somewhat surprising is the number of shifts of the conceptual type. Earlier research by Spink, Goodrum, Robins and Wu (1996) found little evidence of user modeling in an analysis of elicitations. Most elicitations in that study were involved with search terms and strategy, and output relevance. If we consider conceptual shifts to indicate some form of problem modeling by the participants, we can say that modeling occurred in parallel with procedural functions, thus conflicting with the earlier finding. However, the highest number of conceptual functions were evaluation, which cannot be considered strong evidence for modeling. Therefore, conceptual activity is not limited to modeling, but also makes use of cognitive states in order to accomplish certain subgoals, as noted by Belkin, Brooks and Daniels (1987).

Overall, however, one particular item of interest is readily available from this study. That is, the fact that on average, users and search intermediaries engage in little more than seven utterances on any given topic shows the fast paced, and often hurried nature of interactive IR, particularly online searching. Long, intensive discussion on conceptual matters is rare. This suggests that those engaged in such search behaviors do not have time to reflect on what they are seeing and doing, and therefore, are not always able to judge the value of their search in a rational manner.

Type categories present the possibility for further study. One of the many remaining questions in information behavior research is whether users conceptions of their information problem changes during information retrieval interaction. Future research should, without question, attempt to identify such changes. Intuitively, we may guess that some type of change probably occurs. However, identification and measurement of such change is problematic. Perhaps through closer analysis of conceptual shifts during interaction, we may better sense cognitive changes. That is not to say that evidence will not be found in operational shifts as well. Inferences from strategic may yield clues. Certainly some understanding of problem understanding with respect to time during IR interactions would be useful for IR system designers and human search intermediaries.

Future studies should look more closely at focus shifts as non-linear processes. Methods from chaos theory have suggested means of teasing pattern from such data. Similarly, methods of visualizing data of this nature must be established so that patterns may be seen. In these six interactions alone, over 2000 utterances were segmented into 324 shifts of focus. These data need to be displayed in a concise, informative manner.

CONCLUSION

The coding scheme presented in this study should be considered work in progress. The functional codes are based on deeper development that the four shift types. However, all codes are drawn from observation and analysis, rather than a priori. As more tran scripts of interactions are analyzed for shifts, it may become necessary to add or modify existing codes.

The implications for this study are twofold. First, expert system designers need more information about how users interact with IR systems. Improvements for any knowledge base for expert systems should include information about what, and how, interaction is undertaken. That is, a knowledge base should contain data relating to the types of topical and non-topical issues that are generally discussed. They should also reflect the manner in which these issues are discussed, be it trial and error, linear, or what have you. This research gives specific information regarding such issues.

Second, this research has implications for training and practice in both mediated searching, and in end-user searching. Both end-users and search intermediaries need ongoing training for the development of IR interaction skills. Since surprisingly little is known about the particular ways in which information is sought, more research is needed to augment training.

ACKNOWLEDGMENTS

I wish to thank Professor Tefko Saracevic of Rutgers University for providing the transcript data used in this study. I thank my reviewer for this paper who took the time and effort to make extensive, and valuable notes. I would also like to thank Dr. Amanda Spink of the University of North Texas who is my major professor, and whose guidance has been invaluable. I thank my wife, Teresa Franson, for her editing and valuable comments, and everything else.

REFERENCES

Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Erlbaum.

Belkin, N. J. (1984). Cognitive models and information transfer. Social Science Information Studies, 4, 111-129.

Belkin, N. J., Brooks, H. M., & Daniels, P. J. (1987). Knowledge elicitation using discourse analysis. International Journal of Man-Machine Studies, 27, 127-144.

Belkin, N. J., Oddy, R. N. & Brooks, H. M. (1982). ASK for information retrieval. Part I: Background and theory. Journal of Documentation, 38(2), 61-71.

Brooks, H. M. (1986). An intelligent interface for document retrieval systems: Developing the problem description and retrieval strategy components. Unpublished doctoral dissertation, City University, London, United Kingdom.

Cooper, W. S. (1973). On selecting a measure of retrieval effectiveness. Journal of the Association for Information Science, 24, 413-424.

Dennett, D. (1995). Intentional Systems. In W. Lyons (Ed.), Modern philosophy of mind (pp. 191-213). London: Everyman.

Glaser, B. G. & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.

Grosz, B. J. (1981). Focusing and description in natural language dialogs. In A. Joshi, B. Weber, & I. Sag, Elements of discourse understanding (pp. 229-346). Cambridge, UK: Cambridge University Press.

Grosz, B. J. & Sidner, C. L. (1986). Attention, intentions, and structure of discourse. Computational Linguistics, 12(3), 175-204.

Ingwersen, P. (1996). Cognitive perspectives of information retrieval interaction: Elements of a cognitive IR theory. Journal of Documentation, 52(1), 3-50.

Robins, D. (1996). Dimensions of Users' Information Problems as Represented in Interactive Information Retrieval. Unpublished doctoral dissertation proposal. Denton, TX: University of North Texas.

Robins, D. (1996). An intentionalist/rationalist approach to the concept of representation for information retrieval. In P. Ingwersen & N. Ole Pors (Eds.), CoLIS 2: Second international conference on conceptions of library and information science: Inte gration in Perspective (October 13-16, 1996) (pp. 135-148). Copenhagen: The Royal School of Librarianship.

Sinclair, J. M., & Coulthard, R. M. (1975). Toward an analysis of discourse: The English used by teachers and pupils. Oxford, UK: Oxford University Press.

Saracevic, T. & Su, L. (1989). Modeling and measuring user-intermediary-computer interaction online searching: Design of a study. Proceedings of the 52nd Annual Meeting of the Association for Information Science, 26, 75-80.

Spink, A., Goodrum, A., Robins, D. & Wu, M. (1995). Elicitations during information retrieval: Implications for IR system design. In H. P. Frei, D. Harman, P. Schauble & R. Wilkinson (Eds.), Proceedings of the 19th Annual International ACM SIGIR Confer ence on Research and Development in Information Retrieval (ACM SIGIR '96) (pp. 120-127). New York: ACM Press.