Museum objects are being presented in a new and informative light through reflectance transformation imaging (RTI). This photographic method provides detailed images of an object’s shape, color attributes and surface texture along with illumination from different angles, showing details that are often unavailable with the naked eye. Beyond revealing brushstroke details and surface impressions, the RTI technique can be used under a microscope and enhanced with infrared or ultraviolet light, broadening its utility for museum professionals, archaeologists and historians. Associated software records the series of images as a digital lab notebook. RTI is part of the emerging field of computational photography, extracting data from a sequence of digital photographs and synthesizing a new representation holding previously unrecognized information, while protecting the original resource. The digital representations of fine art and natural science museum objects afforded by this new technology facilitate cooperation among museum professionals across disciplines and around the world.

museums
museum informatics
digitized images 
image analysis
physical attributes

Bulletin, February/March 2012


Advanced Imaging Tools for Museum and Library Conservation and Research

by Carla Schroer

Reflectance transformation imaging (RTI) is a photographic method that captures a subject’s surface shape as well as the color. This enhancement allows special viewing software to dynamically relight the subject from any direction and enhance and transform both the surface shape and color attributes. The technology is being adopted in the museum and library context primarily for conservation and research.

Figure 1
Figure 1.
Three views from the same RTI of a small bronze sculpture of the goddess Tutela. RTI allows objects to be dynamically relit using special viewer software. The light source is illuminating the object from the left, from above and from the right. Canton of Valais, Switzerland, Archaeological collection.

The implications of this ability are more far-reaching than we might think. Enhancing the surface shape can reveal information about an object that we cannot see even with the most careful direct observation.

Furthermore, RTI imaging allows for preservation of the digital representations of art and artifacts in a much more comprehensive way than has been available until now. The technology is being developed with strict attention to the requirements of digital documentation and accountability, as discussed below. A comprehensive digital record allows museum and library conservation labs to continue studying an object without danger of degradation to the original and with the potential for wide access to one another's work. 

RTI can be, and has been, applied in a great variety of fields and disciplines. It is generating enthusiastic support in both fine art and natural science museum circles and in the scientific communities. The technology is relatively easy to learn, it can be performed with inexpensive off-the-shelf equipment and all software is free, much of it open-source.

Example: Lovis Corinth Painting from the Worcester Art Museum
Lovis Corinth (1859-1925) was a German artist whose style is characterized by vigorous brushwork and whose work is often grouped among German impressionists and expressionists. His work was posthumously declared decadent by the Nazis, and much of it was burned. A painting of his daughter, Wilhelmine with Braids, came to the United States in the custody of his daughter, but was damaged by an apartment fire decades later. The fire-damaged painting was donated to the Worcester Art Museum by Wilhelmine’s children in hope that it could be restored. In addition to smoke residue and degradation of the varnish layer (which was removed as part of the conservation treatment), the heat of the fire caused blistering in the pronounced impasto of the paint surface. As part of their ongoing treatment efforts conservators at the Worcester Art Museum sought a way to study and document the surface damage.

In 2006 Philip Klausmeyer, scientist and associate paintings conservator, approached Cultural Heritage Imaging to see if RTI might help. Cultural Heritage Imaging is a non-profit corporation that develops and implements new imaging technologies for cultural, historic and artistic heritage and scientific research. A team from CHI traveled to the museum and shot some test images. The resulting RTI showed remarkable surface detail, even tiny blisters on top of blisters within the brushstrokes. 

Figure 2
Figure 2.
Details of blistering on paint surface caused by heat damage from a fire. Left callout shows the detail area of the RTI viewed in normal light; right callout shows the shape and reflectance information. Lovis Corinth, Wilhelmine with Braids. Worcester Art Museum, Gift of Wilhelmine Corinth Klopfer, at the request of her children (2005.202).

The conservation team immediately grasped the potential for RTI in conservation documentation, planning, condition monitoring and research and has since funded and carried out a larger project to bring RTI capability to the lab.

How RTI Works
RTI images are created from information derived from multiple digital photographs of a subject shot from a stationary camera position. In each photograph, light is projected from a different known, or knowable, direction. This process produces a series of images of the same subject with varying highlights and shadows. Lighting information from the images is synthesized to generate a mathematical model of the surface, enabling a user to re-light the RTI image interactively and examine its surface using viewing software.

Each RTI resembles a single, two-dimensional (2D) photographic image. Unlike a typical photograph, reflectance information is derived from the three-dimensional (3D) shape of the image subject and encoded in the image per pixel so that the synthesized RTI image “knows” how light will reflect off the subject. When the RTI is opened in RTI viewing software [1] each constituent pixel is able to reflect the software's interactive “virtual” light from any position selected by the user. This changing interplay of light and shadow in the image discloses fine details of the subject's 3D surface form.

RTI was invented by Tom Malzbender at Hewlett Packard Labs. A landmark paper describing the first tools and methods within the RTI technology family, named polynomial texture mapping (PTM), was published in 2001 [2]. 

How RTI Data Is Captured
The original method for capturing RTI data (that is, capturing a sequence of images using different lighting positions) required the use of specialized light arrays (also called domes). In 2006 the team at CHI and Tom Malzbender invented the highlight method for capturing RTI data, sometimes referred to as HRTI. Placing reflective spheres in view of the camera creates a highlight on the sphere, such as a ball bearing, that open-source processing software called RTIBuilder uses to determine the light position for that image. 

Figure 3
Figure 3
. A typical highlight RTI setup with camera facing down to shoot an object on the floor. The camera is connected to a laptop computer. A radio trigger on the camera fires the flash when the camera is triggered. A string is used to keep an equal distance for the light from each position.

Figure 4
Figure 4.
Highlight on a reflective sphere placed in the scene of an image sequence. The person holding the string is measuring the distance for the light during set up.

This innovative use of reflective spheres allowed for the widespread adoption of RTI at a very low cost. It also allowed RTI to be easily applied to a wide range of materials and material sizes ranging from microscopic to a square meter or more. A user guide with step-by-step instructions for capturing RTI data, along with an instructional video, are available for download on the Cultural Heritage Imaging website [3].

RTI light arrays are still widely used because they are more efficient for capturing and processing RTI data for large collections of smaller material. Most light arrays are custom made and can be quite expensive. The size and cost also increase with the ability to image larger objects.

The RTI technique is very flexible; it can be applied under a microscope and can also be performed using wavelengths outside of the visible spectrum. A microscopy example is included in the subsection, Maker’s Marks and Stamps, below. To apply the technique in the infrared (IR) or ultraviolet (UV) bands, we use a camera that can take IR or UV images, along with a light source that produces IR or UV light in the desired bandwidth. We then follow the same process of RTI capture as with visible light. Infrared bandwidths have been used to disclose hidden surface features below the outer surface of paintings.

Scientific Accountability
For centuries scientists have recorded their experiences and subsequent analysis that described the evidence and results of their inquiries, which then became an integral element of their published results. Scientific information cannot be understood in the absence of the meaning of the data and the history of its generation.

In order to ensure that digital representations of objects are reliable sources for study and research, the CHI RTI software keeps a record of all the process steps applied to the set of images when producing the RTI. We refer to this process history as a “digital lab notebook” (http://culturalheritageimaging.org/Technologies/Digital_Lab_Notebook/), and it serves the same function as a scientist’s written lab notebook. In its simplest form the digital lab notebook contains the record of the means used to digitally capture information about our world and the history of events describing this information’s subsequent processing until it reaches its final state as a completed digital representation.

The digital lab notebook is key to successfully implementing many of the guiding principles of technologies we develop. It provides the means to evaluate a digital surrogate’s scientific reliability and suitability for reuse for both users and conservators or archivists. Among the benefits of a digital lab notebook:

  • A user can determine how images were captured and processed to produce the representation and whether that same image data can be reused with newly developed or updated techniques. For example, newly developed algorithms have been shown to produce more detailed results for certain kinds of materials without reshooting the photographic image sequence, protecting the object from additional handling. 
  • The digital lab notebook enhances the long-term preservation chances of digital representations by making capture and processing histories transparent, thus helping the digital archivist maintain the data over time. Our approach of creating detailed records of image processing and using open file formats and open-source software help RTI adopters meet the Library of Congress’s sustainability factors for digital preservation (www.digitalpreservation.gov/formats/sustain/sustain.shtml).

Fostering Adoption
To foster adoption of RTI technology, CHI has developed do-it-yourself materials and software user guides, as well as a hands-on training program. The training program ensures that people learn the RTI methods in detail for a variety of objects and situations, as well as how to deploy RTI in their institutions or in the field.

CHI currently has an Institute of Museum and Library Services (IMLS) 21st Century Museum Professionals grant to deliver four-day RTI training workshops at each of the six graduate programs in art conservation in North America and four U.S. regional museum sites. Over 150 museum professionals and pre-professionals will be trained during this grant period. We have completed six of the 10 workshops, including sessions at the Institute of Fine Arts at New York University, San Francisco Museum of Modern Art and the Straus Center for Conservation and Technical Studies at the Harvard Art Museums. The remaining workshops will take place in 2012. 

Apart from the above grant, an additional 70 museum professionals, archaeologists and historians have completed the four-day training, and more than 150 people have attended CHI-led workshops ranging from half-day to two-day sessions that introduce RTI, how it works and how it is done. Many training participants spread their RTI knowledge within their institutions as part of adopting the technology.

With the introduction of RTI, I believe conservators have acquired an important new tool with which to document artistic and historic heritage. As we integrate it into customary conservation practice, I believe its value will become even more apparent as we find new uses and applications for this tool. 

-Rita Albertson, Chief Conservator
Worcester Art Museum
 
Letter to IMLS in support of RTI training program

Applications
RTI is being applied to many materials to reveal fine surface details often not visible with the naked eye. This has proved to be a valuable tool in a variety of fields and disciplines. Among the materials and types of details revealed are the following:

• brushstroke details in paintings on wood and on canvas

• condition of paper and impressions made from marks and other surface details in drawings

• scratches on plates and chemical depositions of images on plates in daguerreotypes

• shape of vellum and state of ink flaking from characters in illuminated manuscripts (http://vimeo.com/30213656

• tool marks on furniture

• tool marks on stone

• subtle surface details including fine scratches from wear on coins, which effectively “fingerprint” objects [4]

• details that help scholars decipher previously unread inscriptions (www.hpl.hp.com/research/ptm/antikythera_mechanism)

• details that help determine line order for petroglyphs and decipher worn areas of rock art (http://culturalheritageimaging.org/What_We_Do/Fields/rock_art/)

Microscopy Example: Maker’s Marks and Stamps
A four-day RTI training took place at the Straus Center for Conservation and Technical Studies of the Harvard Art Museums in May of 2011. One area of interest was researching maker’s marks on gold and silver objects, so examples were imaged under a microscope. A tiny ball bearing was placed on the object (Figure 5) and a flash was used to illuminate the object. The results showed fine details of the monogrammatic stamp on the back of a gilt bronze plaquette (Figure 6).

Figure 5
Figure 5.
Microscope setup at the Straus Center for Conservation and Technical Studies of the Harvard Art Museums. Tiny spheres are placed near the small stamp, which will be imaged under the microscope.

Figure 6
Figure 6
. RTI digital representation of the small monogrammatic stamp of the Florentine connoisseur and art dealer Stefano Bardini (1836-1922) located on the back of a gilt bronze plaquette representing Valerio Belli’s (Vicenza, 1468-1546) Adoration of the Shepherds (Harvard Art Museums, Fogg Museum). The bottom portion of the figure shows the plaquette under normal light, and the upper portion shows the enhanced reflectance revealing the surface shape.

New Research and New Directions 
RTI is part of a family of technologies in the new and emerging field of computational photography. Computational photography is based on the computational extraction of relevant information from a sequence of digital photographs. This information is then synthesized into a new digital representation, which conveys information about the subject material that is not found in the individual source photographs (http://culturalheritageimaging.org/Technologies/Overview/).

A new technology, algorithmic rendering (AR), which uses the same datasets as RTI, is being developed by Princeton University and Cultural Heritage Imaging supported by a grant from the National Science Foundation (http://culturalheritageimaging.org/Technologies/Algorithmic_Rendering/). The result of this grant project will be an open-source tool that will extract and merge visual information available only under certain lighting conditions, certain wavelengths or certain imaging modalities. Users will be able to generate high quality, comprehensible illustrations for documentation, scientific study and sharing with colleagues, collection visitors and the public.

Summary
RTI imaging is a promising new technology that opens up exciting avenues of research in arts and sciences. The techniques for creating RTI images are easy to learn and inexpensive to implement. The digital lab notebook that is built into the workflow provides reliable documentation and allows the data to be easily shared and used in an ever-expanding range of applications. 
In addition to protecting the original resources, digital representations facilitate international and interdisciplinary cooperation among fine art and natural science museum professionals and within the scientific communities.

Resources Mentioned in the Article
1. RTIViewer (open-source software): http://culturalheritageimaging.org/What_We_Offer/Downloads/rtiviewer/ Note: Additional software for viewing RTIs is available from other sources.

2. Malzbender, T., Gelb, D., & Wolters, H. (2001). Polynomial texture maps. In Computer Graphics: SIGGRAPH ’01: The 28th International Conference on Computer Graphics and Interactive Techniques [pp. 519-428]. New York: ACM. Retrieved December 31, 2011, from www.Hpl.hp.com/personal/Tom_Malzbender/papers/PTM.pdf 

3. Schroer, C., Bogart, J., Mudge, B., & Lum, M. (2009). Guide to highlight image capture. Cultural Heritage Imaging. Retrieved December 31, 2011, from http://culturalheritageimaging.org/What_We_Offer/Downloads/#capture 

4. Mudge, M., Voutaz, J.-P., Schroer, C., & Lum, M. (2005). Reflection transformation imaging and virtual representations of coins from the Hospice of the Grand St. Bernard. Proceedings of 6th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST2005) [pp. 29-39]. Eurographics Association. Retrieved December 31, 2011, from http://culturalheritageimaging.org/What_We_Do/Publications/vast2005/index.html


Carla Schroer, co-founder and director of Cultural Heritage Imaging (CHI), leads the training programs at CHI, and works on field capture projects with reflectance transformation imaging and related computational photography techniques. Carla also directs the software development and testing activities at CHI. She spent 20 years in the commercial software industry and has extensive experience in software licensing and open-source projects in both the commercial and non-profit sectors. She can be reached at Carla<at>c-h-i.org.