A Sociotechnical Framework for Digital Library Evaluation

Michael Khoo

ASIS&T Annual Meeting - 2006 (ASIS&T 2006)
Austin, Texas, November 3-9, 2006

---

Abstract

The National Science Digital Library (NSDL: http://www.nsdl.org) is an NSF-funded program that supports all levels of science, technology, engineering, and mathematics education. NSDL is conducting a program-wide evaluation of all its activities, with the aim of inventorying its achievements to date, and identifying directions for future development. The scale and complexity of the NSDL program pose significant challenges for evaluation work. This poster outlines a sociotechnical theoretical framework, the ‘resource lifecycle,’ that is being used to guide the evaluation of different aspects of the NSDL program, and to represent it to internal community members and to external partners such as NSF.

The specific sociotechnical model developed for the evaluation of NSDL is called the 'resource lifecycle' model. This model assumes that the overall purpose of the NSDL program is to transform scientific data into valuable pedagogical resources. This transformation is achieved by NSDL at the program level by coordinating a series of linked value-adding operations operated by the individual projects. The resource lifecycle model outlines five core NSDL value-adding activities: (a) resource creation and review; (b) resource aggregation and collection development; (c) web site and search engine design; (d) classroom use/reuse and educator tools; and (e) communication and knowledge infrastructure (meetings, committees, e-mail lists, wikis, outreach, etc.).

The resource life-cycle meta-framework has several advantages for NSDL evaluation work. First, the model's stages provide useful conceptual boundaries within which evaluation efforts may be focused and applied. Second, it provides a coherent overview of how apparently disparate evaluation activities – such as webmetrics and ethnographic observation – may be integrated into and contribute towards an overall evaluation plan. Third, it provides a framework for evaluating NSDL's organizational communication and knowledge processes across the program. Finally, the model provides a coherent narrative structure for reporting the evaluation to NSDL and to NSF. Using a small set of basic concepts to address NSDL's organizational complexity, the resource lifecycle model can coordinate multi-faceted evaluation efforts within a coherent meta-narrative that is useful for future NSDL development efforts, and generate a coherent series of specific research questions that addresses major areas of NSDL activity.

---

  

START Conference Manager (V2.52.6)

Maintainer: rrgerber@softconf.com