Go to National Library of Australia Home PageNational Library of Australia Online

SEARCH:

Presented at the 2nd National Preservation Office Conference: Multimedia Preservation - Capturing the Rainbow, in Brisbane, 28-30 November 1995 by Nancy E. Elkington, Research Libraries Group.
  • Introduction
  • The RLG Context for Collaboration
    • Fundamentals
    • Complicating Factors
    • Levelling Influences
    • Exemplars
  • Why Should You Care?
  • SGML Finding Aids Initiative(s)
  • Joint RLG/CPA Digital Archiving Task Force
  • Collaborative Scanning Projects
  • Inevitable Tensions as we Move Forward
  • Refining the Collaborative Model
  • What's Ahead?

Introduction

Good afternoon!

I'm pleased to have been asked to join you for this conference, because of course it's given me the chance to come here and see something of your lovely country. But more importantly, because I'm increasingly intrigued by the possibility that national strategies for preserving digital information are both imaginable and implementable. I'm also convinced that international collaboration will be the vehicle by which we solve this hardest problem of all. When Jan Lyall and I exchanged email back in the summer, I volunteered to talk a bit about the role that I believe collaboration will play in our global success or failure on this front. What I didn't know was that I would be assigned my now traditional spot of after-lunch speaker. I can't decide if this is because I've achieved an international reputation for peacefully lulling conferees to sleep or if it is because it's the kindest place to put a non-native speaker: this way, if you don't understand much of what I say we can all take comfort in the belief that my accent was simply too thick to be comprehended! In any case, I promise to throw out just enough questions and assertions to keep most of you awake and on your virtual toes.

The RLG Context for Collaboration

First a moment or two of background. The Research Libraries Group is a not-for-profit membership consortium of universities, archives, historical societies, museums, and other institutions devoted to improving access to information that supports research and learning. Founded in 1974, RLG was a pioneer in North America in developing cooperative solutions to the problems that these organizations faced in the acquisition, delivery, and preservation of research information. And, we continue to be a catalyst for collaborative efforts in pursuit of our members' missions. We now have 150 members in 7 countries, including the British Library, the Library of Congress, the US National Archives, the US and Canadian National Galleries of Art and most of the major universities in the US, UK and Ireland. For twenty years, staff from our member institutions have looked to us to help them solve the hardest problems they face -- most often, the problems that no single institution can hope to solve on its own. And, while the US has no national library, no national preservation office, and no national preservation strategy, we do have a number of key stakeholders and leaders forging common solutions on the combined preservation and digital landscape. RLG works cooperatively with the U.S. Commission on Preservation and Access, the Association for Research Libraries, and other North American and European organizations to implement pieces of our interlocking agendas.

We're committed to creating and coordinating a variety of mechanisms for effective international collaboration among institutions that care for the world's shared cultural heritage. And because we're one of the few venues for action rather than talk, we expect to be able to build on the unique gathering of institutions under our banner and move the digital agenda forward together across traditional institutional and political boundaries.

Fundamentals

I'll be talking to you today from the perspective of a preservation specialist now looking more broadly at the digital landscape and what it holds in the way of hope for integrating preservation work into the mainstream of libraries, archives, and museums.

The November 18 issue of The Economist carried an interesting sidebar about the distribution of multimedia equipment around the globe. The chart showed the relative number of telephones, televisions, and personal computers in 15 countries. I'm somehow not surprised to see that, in the US, there are significantly more televisions per 100 people than there are telephones and that the number of personal computers is on the rise. I'm sure you'll be happy to hear that Australia is second on the list in multi-media access, coming in ahead of France, Germany, Japan, Britain and the rest of the pack. As I read about, help design, and participate in digitizing efforts these days, it strikes me that there are two fundamental issues that we need to be aware of and plan for: conversion of materials from analog to digital formats and retention of information in a variety of digital forms including originally-created machine-readable electronic files. Ultimately, the distinction to be made between conversion and retention for access versus conversion and retention for preservation has to do with our ability to sustain access over an unknowable time span and across an indefinite number of technologies. In the case of access-driven projects, we are generally concerned with ensuring the widest possible access for as long as there exists a public need for the materials in question. The time element here is often driven more by commercial interests or usage volume than with any intrinsic desire to keep things around indefinitely. In the case of preservation-driven initiatives, we are generally more concerned with ensuring high quality access to digital objects for as long as we believe they will be wanted. And, since we assume that a preservation copy of something will likely be desired well into the foreseeable future, we are left with a very large problem, indeed: how to sustain access to that information into a future whose shape we cannot now imagine? A word or two about nomenclature. Replacement, surrogate, and supplement are all terms that folks have been using today. It's helpful as we think about managing digital reformatting projects to be clear about our primary goals: replace implies the withdrawal/discard of the original source document while surrogate or supplement usually refers to the copy provided that will serve as an alternate use copy.

Complicating Factors

If we concede that we will want to preserve indefinitely some body of digital objects that has yet to be identified, and that we will want to manage this process in a distributed environment across a global information landscape, what are some of the factors that complicate our search for solutions?

Well, to begin with, a rapidly changing technical environment. Personally, I think it's a mistake to focus on longevity issues for CD-ROMs. They are a delivery mechanism and not a particularly useful storage medium; further, because of technology advances, they won't be around long enough to warrant much significant investment in their long-term survival.

Other complications:

  • An unevenly networked infrastructure.
  • A tendency toward competition rather than cooperation among contributing organizations.
  • An uneven capacity to support the high and ongoing costs of maintaining the preserved data.
  • An impulse toward trying to subsidize the cost of long-term preservation effort on the back of current access.
  • An uneven understanding of how or when or even why to develop selection methodologies in concert with colleague institutions and nations.
  • Sometimes disparate approaches being designed amongst the world's cultural repositories, including libraries, archives and museums.

Levelling Influences

What are the influences that we can already feel that will level the playing field and make our job as creators, curators and caretakers of the cultural heritage simpler, cheaper, and more efficient?

Standards and Best Practices

  • Z39.50
  • Document Type Descriptions (DTDs)
  • File headers
  • Quality guidelines

Training

This issue was raised yesterday and again this morning in the context of the new skill sets that will be necessary as we create a new workforce of digital conversion specialists. Actually, I'm not especially worried about the folks standing at the scanners. They are important to the effort, but not nearly as important as are the skills of the managers and policy-makers who are going to be driving the digital ship. We're all on a very steep learning curve right now, trying to decide how much we need to know in order to sensibly manage this new array of responsibilities. The good news is that we probably don't need to fully get it about bits and bytes any more than most of us get it about how pcs work and what magic holds a jet 38,000 feet in the sky. We mostly need to know just enough to take best advantage of the technology. The bad news is that we need to know enough to ask the right questions and to seek the right answers. And it's not entirely clear just how much that is Economics. Always the great leveller. This will be the single most important element in our ability to manage an integrated information environment that cedes no territory to political instability or economic variability or collective will. Sounds impossible, doesn't it? It is. So our job is to figure out exactly what IS possible at a price we can afford to pay and then set about putting those mechanisms in place.

Collaboration

A growing recognition that no one can "go it" alone. Doesn't matter if you're the national library or a local historical society or an organization as broadly conceived as UNESCO.

  • Collaboration as a means to an end - a view that will be shared by many rather than the few
  • Increasing recognition that unusual and non-replicatable prototypes should be avoided because they only add to the problem rather than contributing to a common set of solutions.
  • Investment in identification of mutually-beneficial factors as baseline to making the business case for new initiatives - making sure that the garden we're planting contains something that everyone can eat.

Exemplars

So, what are some of the projects that can serve as exemplars of the kind of forward-thinking collaboration that I think we must strive for? I've selected a handful of efforts, not because they represent the best or the most interesting or the most unusual, but because they each attempt to achieve something noteworthy and substantive either through their design or in their outcomes. You all know that there are dozens of projects being worked on in institutions around the world. Some you'll hear about in the next few days, others you can read about easily in the standard library literature. The ones I've chosen to highlight are, I hope, new to most of you, and therefore of greater value as examples of where we need to move. A warning: I've included only a few "pure" scanning projects because few exist that were conceived and implemented as collaborative efforts.

Caveats: I'm ignoring text archives projects, journal conversion efforts, and single institution scanning initiatives because they are out of scope for what I'm trying to argue. That is NOT to say that they are not critical in the evolution of new data resources in digital form and in pushing at the walls of encoding, indexing, and network limitations.

National (US) Digital Library Federation

How many of you are familiar with the US National Digital Library Federation? I'm starting with this because it has a great potential for success but the odds of achieving its own goals in the very near term are, I think, relatively low. Some of you may have been following the development of this group in the newsletter of the Commission on Preservation and Access; if so, I apologize for repeating what may already be pretty commonly understood. The Policy Board of the NDLF just a few weeks ago articulated its principal objective and here I quote:

To establish the governance structure and technical infrastructure for a collaboratively managed, physically distributed, not-for-profit repository of digital information in support of instruction and research. The Federation seeks to integrate the unique characteristics and capabilities of digital technologies with the existing strengths of the nation's research libraries and institutions of higher education to provide convenient and affordable access to our intellectual and cultural heritage.

In order to achieve that objective, the Policy Board also just adopted a series of principles regarding the nature and governance of the Federation, among them:

  • that it is not a membership organization;
  • that a policy board will continue to define the criteria and standards necessary for furthering its objectives;
  • that all allied institutions are welcome to join in the effort;
  • that the Commission on Preservation and Access will serve as managing partner to catalyze, coordinate, and raise funding for the NDLF;
  • that the organizational structure will remain flexible;
  • that off-the-shelf products will be preferred over specialized one-off applications;
  • that priorities for development of new solutions not now addressed by off-the-shelf products will be identified;
  • that areas for non-attention will be identified in the context of seeking and defining the specific technical conditions necessary to achieve the Federation's objectives.

So why do I think of this as an exemplar for our consideration? Because, as conceived, the Federation attempts to do collaboratively in the US environment what each nation will ultimately need to think about and do in order to ensure that its cultural heritage is preserved and made available. Is there anything missing from the picture that's been painted so far? Yes, but its not entirely clear if the missing bits are coming in later documents or if they are simply viewed as being less vital to the stakeholders involved thus far.

  • The objective as articulated argues that we will depend on a physically distributed environment. I think they and we need to take the next step and admit that the environment must be networked in order to survive and be of use. Think of the thousands of CD-ROMS that have been created by publishers and institutions wanting to create commercially viable products from the gems in their collections. Think of all the money that has already been spent on digitizing these materials. Think about the Archives of the Indes, the drawings of Frank Lloyd Wright, think of any of the fabulous resources being converted to digital form only to be placed on only a locally-accessible server or a CD that may or may not ever come to the attention of the people who can benefit from its contents. I think - and I'm certainly not alone in this - that we need to stop putting our energy and funds into creating yet more bits and pieces. Instead, we need to find ways to link these resources across a series of global networks and get the stuff into the hands of those who can use it.
  • Distinctions between resources created for temporary or time-bound usage and those created as masters to be held indefinitely.
  • A better understanding of the audiences for whom materials are being created. We're not working with a single, homogeneous community of users, so we need to be thinking about who will use the materials and to what purpose at the point that decisions are made about how we digitize them. More and more often we're hearing of decisions having been made about capture and display and storage options selected at early points in projects that turn out to be inappropriate or, worse, just plain wrong-headed.
  • Selectivity. As we think about converting analog materials into digital forms we simply have to consider questions of how and why and when and at what cost. Part of this is working with potential audiences, part of it is taking realistic approaches to the wealth of materials available. And know that no matter how much gets done, there will always be some body of stuff that you can't afford. And the day after you decide you can't afford it, someone will come asking for it.

RLG's Studies in Scarlet Project

This is a new project, still being shaped, whose primary goals are fourfold:

  1. To create a virtual research resource from contributions made by multiple partner institutions without regard to their physical location;
  2. To make the resource available to the networked scholarly community;
  3. To test the perceived and actual value of such a resource amongst a target audience;
  4. To develop mechanisms necessary to ensure continued and sustainable access to the resource over time.

The subject focus for the project was developed by a task force working with RLG staff and scholarly advisors: its central theme is the legal and social context for marriage and sexuality in the US and Britain between 1815 and 1915. The core collection will be constructed of materials contributed by three institutions (Harvard, Leeds, and New York Public Library) and will comprise:

  • Published accounts of trials dealing with marriage and sexuality in the US and Britain, court reports, broadsides, testimony transcripts, illustrations, pictures, and other ephemera, none of which has been reissued in any format.
  • British reported court decisions, parliamentary papers, books, serials, manuscripts and articles on a variety of related issues, combining primary and secondary sources on core and corollary topics.
  • Commentaries, treatises, and other works on the law from the US and UK; New Yorks state case law, and works documenting social conventions and women's status.

Additional materials will be contributed by four more institutions and will include:

  • Exhaustive coverage of the topic from the perspective of a single state. State statutes, judicial decisions, petitions, and county records will be amongst the materials added to the core.
  • Focus on particular sets of laws that affected many states and the federal justice system. Materials will include appellate court decisions, sample trial transcripts, finding aids to related archival holdings, and selected secondary sources in paper and microfilm.

I think this project pushes outward, away from prototype projects that involve only one or two institutions, and into the arena of meaningful collaboration. By combining the energies of many institutions, we can address critical issues at both the policy and operational levels. And at the same time create the kernel of a new resource in a scalable, international environment. What issues do I consider to be critical? You've heard them already:

  • selection
  • access
  • security and authentication
  • archivability
  • use and evaluation
  • costs

Family of Projects at Cornell (and Yale)

Cornell has been at the forefront of experimentation in using digital technology to solve preservation problems. They continue to launch new initiatives that poke and prod at our assumptions about this new beast but, unlike a host of other institutions who merely experiment and report and move on, Anne Kenney and her colleagues take what they learn and turn it into valuable fodder for the rest of us. The most recent project, and one that both Michael Lesk and I are involved in, centers on the attempt to take digital files and output them onto microfilm. This is neither a new trick nor a particularly difficult one; COM (computer output microfilm) has been around for ages. What is new and different is the work to establish the output as a stable, preservation media that will meet the kinds of quality standards that we've learned to demand and expect of traditional preservation microfilm products.

CIMI CHIO

This is a project that will influence greatly our knowledge and handling of image-based materials. I'll just touch on a few issues briefly.

CIMI is the Consortium for the Interchange of Museum Information. It is sponsored by RLG, the Getty Art History Information Program, and the Canadian Heritage Information Network.

Key elements:

  • SGML encoding for image retrieval, new DTD
  • Z39.50 for exchange of information

Why Should You Care?

Images:

  • broad public appeal
  • willingness to pay for access
  • solve some of the complicated rights and use fee questions faster, cheaper access to images and image content

SGML Finding Aids Initiative(s)

The University of California at Berkeley has lead the US effort to apply SGML encoding to the finding aids produced as access tools to archival collections. This is actually a series of initiatives, taking place at Berkeley and amongst colleague institutions around North America. It has also been taken up by the Society of American Archivists as a professional concern, so lots of people are working on the problem together.

What is the basic goal? To enable effective access to information about archival collections over the Internet. Until now, researchers have merely been able to access this information via collection-level records in national bibliographic utilities. While the bibliographic record is pretty effective in the world of books (you can identify the needed work, find a nearby copy, and go use it), it is insufficient for research using primary source materials. These materials exist in only one location, so it is critical that researchers have access to more information about the contents of the collection before they make travel plans to go use the collection. Contents of archival collections vary widely; which makes it difficult indeed to describe within the confines of a traditional descriptive catalog record. Since standard archival practice already includes the creation of a finding aid, it seems a reasonable next step to link this more detailed information to the national level collection record and local holdings records and ultimately make it accessible via the Internet. SGML encoding of the finding aid will provide terrific new opportunities to create meaningful data about unique documentary resources that can be accessed and evaluated from anywhere in the world.

Berkeley's contribution to this effort, in addition to leading the wave of interest in SGML encoding, is to develop consensus around a new document type definition for finding aids. In other words, they are leading the charge to standardize the way finding aids are structured so we can all more effectively share the contents across the networked universe.

What does this have to do with preserving digital documents? Why should you care? Because without appropriate access mechanisms, preservation of digital objects is a thorough waste of everyone's time, money and expertise. Because scanning millions of documents -- even if those documents are the fundamental documents of a nation -- is a useless enterprise unless we're also figuring out how to create interpretive structures that a researcher can navigate unassisted.

Joint RLG/CPA Digital Archiving Task Force

Last December, the Research Libraries Group and the Commission on Preservation and Access jointly formed the Task Force on Archiving of Digital Information. A 21-member task force worked for about 8 months to investigate the means of ensuring "continued access indefinitely into the future of records stored in digital electronic form." The group included representatives from the technology industry, museums, archives, libraries, publishing, scholarship, and government. Our specific charge incorporated the need to:

  • frame the key problems (organizational, technological, legal, economic, etc.) that need to be resolved for technology refreshing to be considered an acceptable approach to ensuring continued access to electronic digital records indefinitely into the future;
  • define the critical issues that inhibit resolution of each identified problem;
  • recommend actions to remove each inhibiting factor ;
  • consider alternatives to technology refreshing; and
  • make other generic recommendations as appropriate.

In late August, the co-chairs Don Waters and John Garrett issued a draft report that was made available in paper as well as via ftp and through our Web site. Included in that report are a series of recommendations for action by a number of stakeholders. Comments have been received from individuals and groups in Canada, the US and UK, Australia, and Western continental Europe. A final report will be issued this winter and distributed as widely as possible.

Why is this task force on my list of exemplars? Because it represents one of those few exciting moments in history when a problem articulated is a problem shared. Not only was the group itself engaged in a highly collaborative process (we met twice, conducted dozens of conference calls, and exchanged voluminous email) but the response to its work has been something of a surprise. Yesterday and today, a group is convening in the UK to talk about how to develop a national strategy for digital archiving. Next week, Jan Lyall is hosting a meeting in Canberra to address the same question for Australia. Clearly, the problems associated with maintaining digital information are not new. But it seems that the time is right to get the folks together in a variety of spots around the globe and begin working out the very difficult issues involved in designing solutions.

Without going into exhaustive detail on the contents of the report, I think you might all be interested in the eight recommendations that come at the end. These are directed to the two sponsoring organizations, RLG and CPA:

  1. Solicit proposals for participants in a cooperative project that would place digital objects from the early digital age into trust for use by future generations.
  2. Secure funding and sponsor an open competition for proposals to advance digital archives, particularly with respect to removing legal and economic barriers.
  3. Foster practical experiments in the archival application of technologies and services such as those that support property rights and authentication mechanisms.
  4. Coordinate appropriate organizations in the development of standards, criteria, and mechanisms for identifying and certifying digital archives.
  5. Engage actively in national policy efforts to design and develop the national information infrastructure to ensure that longevity of information is an explicit goal.
  6. Sponsor the development of a white paper on the foundations needed in intellectual property law to support the aggressive rescue of endangered digital information through an effective fail-safe mechanism.
  7. Engage representatives of professional societies in a variety of disciplines in a series of fora designed to elicit creative thinking about the means of creating and financing digital archives.
  8. Commission follow-on case studies to identify current best practices and to benchmark costs in one or more of the following areas: a) storage of massive quantities of information; b) use of metadata for digital preservation; and c) migration paths.

Collaborative Scanning Projects

While there are lots of projects being mounted by single institutions around the world, there are fewer that were conceived and implemented as collaborative efforts and which therefore fit my profile for this paper. A few of the more interestings ones are:

  • Michigan Papyrus Project
  • Project Diana
  • Cornell CORE Agriculture
  • Andre Sakharov Papers (Brandeis and Russian Archive)
  • Australian Cooperative Digitization Project (1840-1845)

Inevitable Tensions as We Move Forward

It will not have escaped your notice that the projects I described have as much to do with access as they do with preservation. Alan Howell, in yesterday's workshop, mentioned that there is a certain amount of "polarization" now occurring within the preservation community worldwide. He was referring to the divergent views being expressed by those who want to see microfilm technology continued as a preferred preservation option and by those who want to see digitizing overtake the more traditional art. He in fact argued the case for a hybrid approach that can successfully merge the two technologies in order to achieve gains for us all. But I think he touched on a much larger issue, a great tension that now exists in the library and archival worlds, and one that is not going to go away. In fact, it will be squarely in all our faces as we try to advance our joint agenda. The tension I'm talking about is the very real one between preservation and access.

Just a few years ago, conservators and preservation administrators alike were concerned that the advent of large, centralized funding for preservation microfilming would eliminate or at the very least overshadow the need for a continued focus on item treatments and collections care. But this was an internal debate, and one that finally settled down with little intervention from outside. But the current debate is taking place at the very heart of cultural organizations and has as much to do with finance as it does with decisions about emphases and territory. Digital reformatting is firmly with us. And electronic records in a range of digital forms are also here to stay. In fact they will drown us if we don't learn how to cope with them quickly and effectively. So we'll not only have to learn to use the technology as a tool in our preservation kit, we'll have to acquire expertise in managing the long-term care of these new collections. The bottom line? Preservation needs to move into the mainstream and its managers need to participate as full partners in this huge organizational investment we're all making. We need to merge preservation requirements with access needs instead of continuing to force an artificial either/or question in every instance.

Refining the Collaborative Model

What has to happen to the collaborative model as we move away from preservation microfilming to other technologies? Well, we're going to have to rely on experts from an even wider community, encompassing folks in the information industry, in public policy, and in a host of traditional arenas. More than ever before we're going to have to be able to count on institutional will not just to make projects happen today, but to sustain their commitment to the preservation of digital information over indefinite periods into the future. And that, of course, leads directly to funding. Our current financial models assume that reformatting costs are incurred once and thereafter film storage costs will remain steady over time. But as we begin to consider digital masters as preservation copies, we'll need new financial models that assume similar initial document preparation costs, potentially lower reformatting costs, almost certainly declining storage costs, but as-yet-indeterminable costs to migrate information from today's technical platforms to tomorrow's. The Digital Archiving report I mentioned a few moments ago suggests a number of issues that will impact on organizations everywhere and also sets out an interesting cost model for the long- term impact of these costs on organizations. The last component of the collaborative model -- oversight -- is, once again, of critical importance. None of our collective questions about using the technology, whether for preservation or access or both, can be answered in isolation. The National Library of Australia and all of you, my organization and others in North America and Europe, and other convenors of like-minded institutions and individuals around the world will continue to function at the heart of the collaborative process in the digital age.

What's Ahead?

In the next few years we will need to work together to expand and adapt our collaborative models to embrace globally scattered partners and learn how to incorporate international initiatives that succeed despite -- and in many instances because of -- geography. We'll be building collections of preserved and accessible materials without regard for the physical location of either the original or the master; in other words, we'll be creating, preserving, and maintaining networked access to truly international collections. Finally, we'll need to identify ongoing funding sources that allow us to harness and manage the power of digital information.

Danny Hillis, writing in WIRED Magazine, said something recently that resounds in my head: "I sense that I'm alive at a time of important change and I feel a responsibility to make sure that the change comes out well." I suspect if we can all agree on such a basic tenet, the future will, indeed, come out just fine.


Return to 1995 NPO Conference
Last updated 27 February 1998