Interactive and collaborative systems is an area of active research in IR, AI, and business psychology (Cutting et al., 1992; Burke et al., 1996; Davenport, 1994). Much of this research is technology driven and directed at defining applications that would require their users to engage in new types of interaction and collaboration. Our methodology is different. We focus on a behavior which already exists, i.e. collaborative and interactive question-answering in organizations, and develop a technology to support it.
Most IR systems use relevance feedback as the main method to support interaction with the client. Typically, relevance feedback techniques require multiple retrieval interactions, each of which asks the client to state which of the retrieved results are relevant to the submitted question and which are not (Aalbersberg, 1992). In this scenario, the client can become overwhelmed with choices or may not have sufficient knowledge to estimate the relevancy of each retrieved result. To address this problem, Aalbersberg proposes an interaction technique in which documents are retrieved one by one and the client only must specify whether or not the most recently retrieved document is relevant. Our client-oriented relevance feedback technique builds on this idea, but differs from it in explicitness and flexibility. We explicitly limit the number of negative interactions the client can have with the system. Our approach is more flexible in that at each interaction, we offer the client several options: to browse the document collection, redirect the search to another collection, or contact an expert.
Another line of relevance feedback research is aimed at modifying the term weights in the document vectors (Ide & Salton, 1971; Brauen, 1971). The idea is to bring the relevant documents closer to a given question and the nonrelevant ones away from it, so that future clients will benefit from the previous interactions. The difference between this approach and ours is that in our system some vectors in the collection correspond to the experts. If the weights in those vectors are not adjusted fast enough, the expert may receive the same nonrelevant question multiple times. Our approach addresses this problem through negative evidence acquisition.
CIE does not do any explicit word sense disambiguation. The reason for this is twofold. First, as Voorhees (1993) shows, using a constrained spread of activation in WordNet to disambiguate word senses does not result in performance improvement. Second, as Sanderson (1994) demonstrates, IR systems are surprisingly resilient to ambiguity. In fact, Sanderson's experiments show that to be of practical use, disambiguation tools need to operate with at least 90 percent accuracy. To the best of our knowledge, none of the currently available technologies in IR, AI, and computational linguistics is capable of that. Therefore, the question becomes what approach could provide a modest step forward. Our approach is to do a loosely constrained spread of activation on parts of the document at indexing time and let the term weight metrics, relevance feedback, and negative evidence acquisition home in on useful terms. Although no explicit disambiguation is done, the WordNet-based term weight metric in Equation 2 decreases the weight of polysemantic words.
A common criticism of numerical IR approaches by AI researchers is that they are knowledge poor and cannot perform matching on a deep semantic level (Cohen & Kjeldsen, 1987; Burke et al., 1996). But, the proponents of knowledge intensive approaches often omit from their arguments the fact that knowledge representation has costs. First, all of the necessary knowledge must be entered into the system before it is used for deep matches. Second, the knowledge features useful for matching must be reliably extracted from inputs. For example, Cohen and Kjeldsen's system, called GRANT, simulates the performance of a funding advisor and relies on an inference-based spreading activation in a semantic network built specifically for the task. As the authors admit, that semantic network took four person-months to build. The inputs to their system are semantic representations of grant requests. Thus, to use the system, the client must master a query language. Unfortunately, no evaluation is given that compares the performance of GRANT to that of a numerical IR system on the same task. Burke et al. (1996) present an apartment finding system that overcomes the barrier of query language learning by providing the client with a set of browsing interfaces. However, their approach is applicable only in domains with finite sets of features where clients have very specific search goals expressible in terms of those features. While our approach uses some knowledge representation, its complexity does not go beyond a single- or multiple-inheritance hierarchy of topics. Also, our approach allows the clients to interact with the system in natural language.