|
Scheduled Program Sessions
Divided by Tracks PLENARY SESSIONS CONFERENCE OPENING
Get up to speed on the issues with a background and update presentation by Clifford Lynch, Coalition
for Networked Information, followed by the welcome reception. PLENARY
The Importance of Evaluating Networked Information Services and Resources
A panel of speakers, moderated by Charles R. McClure, will debate key issues
and strategies related to improving evaluation efforts related to networked services and resources.
Peter Brophy, Manchester Metropolitan University
Ron Larsen, DARPA
Charles R. McClure
, Syracuse University CONFERENCE SUMMARY AND WRAP UP
What Was Learned; What is Needed?
John C. Bertot, SUNY Albany, Moderator
EVALUATING WEB RESOURCES AND SITES PANELS
Initiatives in Evaluating Federal Websites
Federal agencies have increasingly been providing information and services via the Web. As their Web sites
have grown, agencies have begun to consider the impact of those sites for users and for the organizations. The three panelists all have extensive experience in the evaluation of Federal Web sites and the presentations will
highlight particular components of those evaluations.
Charles R. McClure, Syracuse University. An Information Policy Perspective on Federal Web Site Evaluation: Evaluation at the Department of Education
TBA from the Federal Interagency Task Force on FedStats. Evaluation of Web Sites Spanning Multiple Agencies: The Experience of FedStats
Carol A. Hert, Syracuse University. Understanding the Organizational Impact
of a Federal Web Site: Exploration of Two Cases
Moderator: Carol A. Hert, Syracuse University Evaluating the Web: A Look at Web
Pages, Databases and Evaluation Theory (SIGs/LAN, ED, IAE)
Evaluation of
Web-based information resources is an important part of a librarian's work. But what are the criteria for evaluating the accuracy and authority of Web pages? What are the criteria for evaluating proprietary Web interface databases?
What being taught in ILS/LIS today will equip librarians for this important task? Jeff Rosen and Ann Eagan will discuss how to evaluate "free" Web pages. Keith Stirling's presentation title will be "On Evaluating/Implementing
Vendor Electronic Products." Greg Newby will discuss evaluation and/or related research.
Keith H. Stirling, Brigham Young University
Gregory B. Newby, University of North Carolina at Chapel Hill
Jeff Rosen
, University of Arizona Library
Ann Eagan, University of Arizona, Tucson
Moderator: Laura K. Cousineau, Duke University
CONTRIBUTED PAPERS
How To Make Information on the Internet More Verifiable
. Don Fallis, University of ArizonaDevelopment of an Internet Site Evaluation Tool for Use by Information Management Students. Michael Middleton and S. Edwards, Queensland University of Technology
Ongoing Evaluation of Internet Resources: Policies and Procedures. Susan Calcari and Gerri Wanserski, Internet Scout Project, University of Wisconsin-Madison Methods of Measurement and Evaluation of
Departmental and Individual Web Sites in an Academic Library. Jeanie M. Welch, University of North Carolina at Charlotte North Carolina State University Libraries Evaluate the Use of Network Resources. Keith
Morgan, North Carolina State University Online Journal Use in a Segment of Academe. George S. Porter, Ian Roberts, Caroline Smith, Ed Sponsler and Judith Jo Nollar, all Caltech Library System User's
Searching Behavior and the World Wide Web: Physical, Cognitive and Affective Factors. Peiling Wang and William B. Hawk, University of Tennessee, Knoxville Networked Information Resource Use as Planned
Behavior in Context: A Reflection on the Role of Definition and Measurement in Quantitative User Studies. Jane E. Klobas, The University of Western Australia IMPACT AND USE CONTRIBUTED PAPERS
Use, Benefits and Constraints of Electronic Communications in Africa: How to Assess Them?
Michel J. Menou, CIDEGI; Jane Asaba, Uganda AIDS Commission; Bernard Bazirake, Makerere University (Uganda); Vitalicy Chifwepa, University of Zambia; Nancy J. Hafkin
, African Information Science Initiative; and Abebe Rorissa, University of Namibia"Beam It Up Scotty" - Industrial Designers Using Networks to Distribute Three Dimensional Material on the Internet.
Mark Evans and Paul Wormald, both Loughborough University The Role of User Evaluation in Design and Ongoing Development of a WWW Browser-Based Interface to Library and Networked Information Resources at
the University of Western Australia. Jane E. Klobas, The University of Western Australia What Is Web Usability Anyway? A Conceptual Study on Usability in the Web Environment. Ping Zhang and
Jiangping Chen, Syracuse University Use Analysis of an Academic Library Web Site. Kyunghye Kim, Rutgers University Query Characteristics in the George Bush Digital Library.
Dion H. Goh and Rebecca P. Ang, Texas A&M University PANELS
Economics of Web-Based Information (SIGs/MGT, III)
Only limited financial evaluation of Web-based information is available. For information providers,
relevant questions concern tangible and intangible costs and benefits, direct and indirect costs, how expenditures compare with an equivalent print source, the relationship between capital expenditures and repairs, methods of
estimating benefits from Web-based information. For information users, relevant questions concern the tangible and intangible costs and benefits of Web-based information, getting unreliable information due to noise generated by
cultural differences or because Web pages contain obsolete information. What is the role of newer valuing methodologies, such as total value, value-in-use and aggregate analysis? What benefits can result from internal Web
resources used in knowledge management functions? The panel will shed light on these and other issues, bringing together experts on economics of Web-based information from the corporate, public service, academic library and
academic research domains in the United States and abroad.
Jane K. Starnes, Intel Corporation
Kristi DeShazo, Intel Corporation
Jennifer Krueger, New York Public Library
Kimberly Douglas
, California Institute of Technology
Moderator: Beverly Colby, Arthur D. Little Access to Scholarly Information via Professional Societies' Web Sites: Content, Means and Uses (SIG/STI)
This panel
will address issues in accessing scholarly information via the digital library at professional societies' Web sites. The discussion will focus on the acquisition, selection and delivery of document content, the means and economy of
access and the uses. A speaker from a professional society, a database producer and a corporate user will bring their perspectives to the audience. This panel will start with an introduction to the current situation in Web-based
access to scholarly information via professional societies' Web sites, followed by the three speakers mentioned above. An on-site user survey may be conducted for speaker-audience interaction and discussion.
Elizabeth Snider
, Chemical Abstracts Service
Svetla Baykoucheva, Manager of ACS Library Services, American Chemical Society, Content Management in a Professional Society/Publisher Environment
Nancy Cundiff
, The Dow Chemical Company Business Intelligence Center
Moderator: Jian Qin, University of Southern Mississippi Networked Resources for Environmental Decision Making (SIGs/IFP, STI)
Environmental
decision making and natural resource management require information from sources that cross public and private sectors and multiple political jurisdictions. This session will explore the factors required to provide resource
managers, policy makers and other stakeholders with information for environmental decision making. These include
- variety of information sources;
- selection, integration and quality evaluations needed to select from both current and legacy information sources;
- information system requirements; and
- coordination across sectors.
The panel will also discuss a research agenda needed to improve the development and evaluation of environmental decision support systems.
Deanne DiPietro, California Environmental Resources Evaluation System.
Information for California's Resource Managers:The CERES Experience
Jim Quinn, University of California, Davis. Coordinating Information across Sectors and Jurisdictions: California's Biodiversity Council
Mark Fornwall, Center for Biological Informatics, US Geological Survey/ Biological Resources Division. Supporting Regional and National Policy Decisions: A Research Agenda for Biological Informatics
Gail Hodge
, USGS/Biological Resources Division. The National Biological Information Infrastructure: Tools to Bridge the Gap
Bruce Bargmeyer, US Environmental Protection Agency. Developing an Environmental Metadata Registry
Moderator: Bonnie C. Carroll, Information International Associates, Inc. Information Use in the Professions (Proto-SIG/USE)
Research findings and anecdotal evidence suggest significant changes
in the information behavior of professionals who have ready access to databases and full text resources. Panelists in this session will describe the ways in which different professional groups (such as medicine, law and ministry)
access and use information, with emphasis on how use patterns have changed with the introduction and assimilation of technology. The professions covered will vary significantly in the types of resources used and the amount of time
that resources have been automated, ranging from law, automated in the early 1970s, to the much more recently automated divinity profession. The panel will consist of practitioners or researchers with expertise in working with
particular professions, and their views will be critiqued by a discussant. The panel and the audience will be able to compare and contrast information behaviors across the professions.
Scott Adams,
University of North Carolina at Chapel Hill
Valerie Florance, American Association of Medical Colleges
Moderator: Nancy Roderer, Yale University Accessing Knowledge Through the Window: Database
Providers' Methods for Evaluating Network Interfaces (SIG/CRS)
Commercial database access vendors have recently shifted to graphical user interfaces and Web page formats. However, users of these vendored database products
are not all in agreement about the actual appeal and usefulness of the new interfaces. To gain insight into the evaluation of network interfaces and the decision criteria employed by the vendors, this open forum discussion of
specific questions posed to the vendors prior to the meeting will provide a basis for informal discussion with the audience. Three or four leading commercial database access providers will be invited to discuss how their
companies determine what the network interfaces should look like, what methods of evaluation and feedback they use to support these decisions and how they envision the future of these types of access systems in light of the network
communication evolution. Questions will include
- What evaluation or decision activities were undertaken in designing changes to the interface?
- What studies or methods determine the effects of the interface on search efficiency and on user satisfaction?
- Does your organization have a vision of the future interface of the network access method?
- Has the Web/graphic interface impacted other aspects of your work with the databases and the network systems?
Daniel B. Pliske, Senior Director, Applied Research LEXIS-NEXIS
Joe Pryor, Dialog
Representative from OCLC
Representative from Ovid
Moderator: Vivian Hay, Getty Information Institute
METHODS PANELS
The Next Wave of Z39.50 Implementation: Informed by Evaluation
The panel will inform attendees about the
current status of several important Z39.50 implementations. In addition, these projects provide a lens on the complex challenges of networked discovery and retrieval across distributed resources. More importantly, the panel
presentations will suggest evaluation techniques for understanding complex and interrelated technology deployments such as Z39.50-accessible databases. Finally, the panel will show the importance of ongoing evaluation of
technologies such as Z39.50 to expose and define interoperability problems of networked information discovery and retrieval. Evaluation findings can aid in the creation of new solutions and pathways to successful technology
implementations.
Carrol Lunau, Virtual Canadian Union Catalogue Project, National Library of Canada. Findings from the vCuc Experience [tentative]
Blue Angel Technologies/SILO Project [TBD].
Findings from the SILO Experience [tentative]
William E. Moen, University of North Texas. Putting Z39.50 Evaluation Findings to Use: The Z Texas Project
Moderator: William E. Moen
, University of North TexasDeveloping Performance Measures for Networked Information Resources and Services: Issues and Prospects
This panel will provide an overview of some recent activities in the
development of performance measures for networked information services and resources in a library environment. McClure will give an overview of key issues affecting the development of such measures; Bertot will provide specific
examples of possible performance measures; and Ford will update activities in the United Kingdom and Europe on developing such measures and how they are being used.
Charles R. McClure, Syracuse University
John C. Bertot, SUNY Albany
Geoffrey Ford, University of Bristol METRICS CONTRIBUTED PAPERS
The Effective Electronic Library. Peter Brophy, Manchester Metropolitan UniversityNetworked Information Resources: A Review of Research Methodologies. Kathleen Murray, University of North Texas
Measuring the Impact of the WWW On Non-Work Based Tasks. John D'Ambra, University of New South Wales Structural and Administrative Metadata for Digital Libraries: The Making of America II Project.
Howard Besser, University of California, Berkeley Alexandria Digital Library User and Use Evaluation: Experiments with a Neural Network Method for Log Data Analysis. Philip Sallis
, University of Otago, NZ; Linda Hill, Mary Larsgaard, Kevin Lovette, Catherine Masi and Mary-Anna Rae, all Alexandria Digital Library Project, UCSB, Santa Barbara
Transaction Log Analysis in Web-based Information Systems. John Fieber, Indiana University Methodologies for Using Electronic Surveys to Evaluate Networked Information Resources. Jonathan Lazar
and Jennifer Preece, University of Maryland Baltimore County PANEL
Measurement and Evaluation of Federated Digital Libraries
The complex world of distributed heterogeneous digital libraries brings with it additional
complexity in measurement and evaluation. Issues to be dealt with include the distributed nature of the digital library, the importance of user interfaces to the system and the need for systems approaches to deal with heterogeneity
amongst the various components of the digital library.In this panel session, these issues are explored from three perspectives. The work of the D-Lib Working Group on Digital Library Metrics is discussed (objective: to develop a
consensus on usable and useful metrics to evaluate and compare the effectiveness of digital libraries and component technologies in a distributed environment). The D-Lib Test Suite is a group of digital library testbeds that are
made available over the Internet for research in digital libraries, information management, collaboration, visualization and related disciplines. Finally, the Networked Computer Science Technical Reference Library (NCSTRL) is a
confederation of over 100 institutions providing a federated library of computer science material, i.e., a seamless federation of collections and associated library services accessible to the broad community. Evaluation results and
lessons from the NCSTRL environment are discussed.
Barry Leiner, Corporation for National Research Initiative
William Y. Arms, Corporation for National Research Initiatives
Carl Lagoze
, Cornell University SOCIAL ASPECTS
CONTRIBUTED PAPERS Privacy, Electronic Commerce Resources and the Web: An Actor-Theoretic Examination of the P3P Project. Mark S. Ackerman, University of California, IrvineTexts and Users: The Relevance of Humanities-Based Theories of Text Production to the Study of Networked
Information Use. D. Grant Campbell, University of Western Ontario Private Lives and Public Spaces. Gretchen Whitney, University of Tennessee, Knoxville SPECIAL TOPICS Verification
PANELS
Authentication and Authorization: How Is It Being Implemented? (SIG/LAN)
Using
networked information resources and services today increasingly involves controlling access to licensed databases or student coursework and course materials. This control requires some method of first identifying positively who the
requesters are (authentication) and ensuring that they are allowed to enter that resource (authorization). What are the options for accomplishing this control? How are libraries, instructors and the university community handling
this challenge?
Clifford Lynch, Coalition for Networked Information. An Overview of the Authentication and Authorization Challenge
Derek Brink, Gradient Technologies, Inc. Opening the Doors -
Selectively - To the Big Ten Virtual Library
Sal Gurnani, California Digital Library. Certificate and Directory Based Access Control to Licensed Web Content
Moderator: Merri Beth Lavagnino
, Committee on Institutional Cooperation (CIC) Community Networks
CONTRIBUTED PAPERS
CNI Research and Evaluation Activities: Methodological Issues Related to Evaluating User
Needs and Outcomes Related to Community Information Systems. Ann P. Bishop, University of Illinois at Urbana-Champaign
Winners and Survivors: Evolution of Digital Community Networks. Linda Schamber,
University of North Texas PANELS
Evaluation of Community Networks
Community networks (CNs) provide online information resources and communication tools to residents of particular geographic communities. They typically make
extensive use of volunteers. A natural alliance exists between the goals and services of CNs and public libraries. Of special significance to LIS is the role that CNs play in reducing the "digital divide" that currently separates
haves and have nots in the information age. CNs are a relatively new "genre" of information system, which raises special problems in assessing user needs and outcomes. The panel will describe efforts to evaluate the use and impact
of CNs and discuss conceptual and methodological issues in this realm.
Gregory Newby, University of North Carolina at Chapel Hill
Phil Agre, University of California at Los Angeles
Douglas Schuler
, author of New Community Networks: Wired for Change
Joan Durrance, University of Michigan
Moderator: Ann Bishop, University of Illinois, Urbana-Champaign Distance Education
CONTRIBUTED PAPERS
Web-based Course Delivery: The Effect of the Medium on Student Learning and Use of Library-based Resources. Vicki L. Gregory, James Carey and Derrie R. Perez,
University of South Florida
Evaluating a Gateway to Faculty Syllabi (GFS) on the Internet. Sam G. Oh, Sung Kyun Kwan University Images
PANELS
Network Access to Visual Information: A Study of Costs and Uses (SIGs/AH, VIS)
This session will outline the findings of the Mellon-sponsored study of digital image distribution focusing on the Museum Educational Site Licensing Project (MESL). In the MESL project
seven repositories supplied an identical set of 10,000 images and accompanying descriptive metadata to seven universities, and each university mounted this information within its own customized delivery system. The study evaluated
the costs, infrastructure and efforts involved in implementing the MESL project, as well as user reaction to functionality. The study also examined costs of running analog slide libraries and compared these to costs and
functionality associated with digital image distribution.Panelists will discuss the cost-center models used to examine the distribution of digital and analog images, including creating digital images and metadata, mounting and
distributing digital images, maintaining a distribution house, running a slide library and an analysis of hybrid image libraries. They will present a comparison of user interfaces and search engines from the MESL universities. They
will also report on the results of focus groups discussing faculty adoption of digital images for classroom use.
Howard Besser, University of California, Berkeley
Robert Yamashita
, California State University, San Marcos
Rosalie Lack, University of California, Berkeley
Joanne Miller, University of California, Berkeley
Lena Stebley, San Jose State University
Moderator:
Howard Besser, University of California, Berkeley IR Evaluation
CONTRIBUTED PAPERS
Cognitive Approach to Feedback in Information Retrieval Applied to the Design
of "Enabling" Devices for Undergraduates. Charles Cole, Concordia University Evaluation of a Visualization System for Information Retrieval at the Front End and the Back End. Gregory B. Newby
, University of North Carolina, Chapel Hill Show Me the Pictures! or Evaluation Studies of Art Image Retrieval. Samantha Kelly Hastings and T. J. Russell, University of North Texas |
