Staff Papers

PANDORA - Towards a national collection of Australian electronic publications

A paper presented by Fiona Ash Relf, Australian Electronic Unit, at the 10th national Library Technicians' conference, Fremantle, 8-10 September 1999.


CONTENTS:


Abstract

The National Library of Australia strongly believes that the preservation of Australian online publications is an essential component of the Library’s broader responsibility to preserve and make available Australia’s documentary heritage. In June 1996 the National Library established the PANDORA (Preserving and Accessing Networked Documentary Resources of Australia) Project in order to develop policies and procedures for the selection, capture, archiving and provision of long-term access to Australian electronic publications. By the end of 1997 a functional proof-of-concept archive existed and policy documents outlining the conceptual framework for a permanent electronic archive had been developed. The Project’s next challenge is the development of a working model for a national collection of Australian electronic publications. Another priority is to foster working partnerships with other national libraries and overseas agencies who are also undertaking research and development in this area.

This paper looks briefly at the PANDORA Project and its key business principles. Collaborative work with the State libraries to develop a viable distributed model for preserving Australian online publications is also discussed. Some issues that need addressing in order to ensure the ultimate success of the PANDORA Project, for example permanent naming, authentication and rights management, are also explored.


Introduction

The purpose of this paper is to outline the National Library of Australia’s response to the challenge of preserving and providing long term access to Australian electronic publications. Preservation and enabling future access to publications have always been the core functions of libraries and will remain so in our electronic age. This paper seeks to emphasise the importance of preserving Australia’s electronic publications as an integral part of the national imprint

The National Library’s role includes preserving the documentary resources of Australia and it once attempted to do so exhaustively. In more recent years, the Distributed National Collection concept has led to cooperative collecting and preservation arrangements amongst Australian libraries. This approach was developed long before electronic publishing emerged and still works well for traditional library materials. However, the recent explosion in electronic publishing has now seriously undermined the ability of Australian libraries to maintain this level of control over the national imprint. Extending the Distributed National Collection concept to include electronic publications is a practical solution for dealing with this new publishing phenomenon.

The PANDORA Project is the first step along the road towards this aim.

With collaboration between libraries, academic institutions, and the creators and publishers of electronic publications, Australian libraries can still manage their preservation and access functions in the electronic age. The Project seeks to create a workable national model to meet this objective. A national approach has facilitated the successful management of Australia’s national imprint up until now, and is even more essential if libraries are to cope with the vast amount of electronic material being published.

This paper will cover some of the background and policy decisions surrounding the Project. It will also briefly outline the processes and problems involved in the archiving of electronic publications and discuss some of the issues affecting preservation and access. The current move towards a national approach to this issue is described and its importance underlined.


1. The PANDORA Project and its key business principles

The PANDORA Project’s overall objective is to develop policy and procedures for preserving and making available Australian electronic material while simultaneously creating an archive of selected Australian Internet publications. PANDORA is a catchy acronym and stands for Preserving and Accessing Networked Documentary Resources of Australia. But the name also brings to mind the story of Pandora’s box from Greek mythology. One dictionary definition of the term Pandora’s box describes it as ‘any source of extensive troubles, especially one expected at first to yield blessings’ (Penguin Macquarie Dictionary 1986). However PANDORA’s staff prefer to take heart in the Roman version of the myth which states that hope remained in the box after all the evils had departed! Definitions and mythology aside, the National Library’s PANDORA Project has achieved a great deal both nationally and internationally and is a source of much pride for the National Library.

1.1 The Project and its purpose

The Project was set up three years ago to develop and implement procedures for the selection, capture, archiving and provision of long-term access to online Australian publications. Right from the outset, the Project’s ultimate goal has been to utilise the knowledge gained to develop a proposal for a national strategy for preserving electronic publications. The approach taken has involved the creation of a proof-of-concept archive, which in turn has provided a practical framework for the development and testing of policies and procedures. Strategies adopted to achieve these outcomes have included:

To achieve these aims the Project has involved staff from various areas of the Library contributing their expertise as necessary. The ongoing work of developing and managing the PANDORA Archive is currently undertaken by the Library’s Electronic Unit, (comprising six staff at present), in conjunction with significant IT assistance.

1.2 The proof-of-concept archive

The PANDORA Archive is the Project’s showpiece (visit the Archive at: http://pandora.nla.gov.au/ )1. The Archive is impressive not only because it represents the information future, but more poignantly, it already represents the past as well. We are admittedly talking recent past now, but for researchers in the decades and centuries to come its value will be immeasurable. Some of the publications archived have already disappeared from the Internet and now only exist in the Archive. The archived version has thus become irreplaceable. Others have changed location or presentation and those changes have been tracked and recorded. But as the Internet continues to grow and electronic publications proliferate, PANDORA is less capable of keeping up, largely because of resource constraints. A broader national approach must be taken sooner rather than later to address this and extending the concept of the Distributed National Collection is one way to do this.

It must be stressed that the Archive is NOT an archive of digitised traditional library materials like manuscripts or pictorial works, but rather an archive of ‘digitally-born’ publications. It is an ongoing initiative that also serves as an experimental platform for trying out new procedures, policies and technical solutions. PANDORA is a fully functional live archive that is growing steadily and is accessible to the public via Kinetica and the Library’s OPAC. At the end of June this year there were 229 titles in the Archive and many of these titles contain multiple versions or issues. At least that many titles again have been selected for inclusion but have not yet been archived largely for technical or resource reasons.

1.3 Key business principles

In order to commence the business of archiving, the PANDORA Project came up with a set of principles to guide their activities. These principles still govern the way the Archive is managed on a daily basis. They include:

These principles were largely developed for practical reasons governed by resource and technical constraints. They are subject to revision as the Project develops and more Australian libraries become involved. The selection guidelines in particular may be relaxed and broadened as an improved technical infrastructure reduces the requirement for the labour-intensive procedures currently undertaken.


2. Creating the Archive and problems encountered

The process of creating the Archive has thrown up innumerable challenges. As no precedent existed, each procedure was developed from scratch and then strenuously tested. In effect a parallel workflow was designed to operate alongside traditional library materials. Given the current complexity of procedures and the technological problems encountered, it seems unlikely that the work will be integrated with traditional technical services work in the short to medium term. The work as it stands today requires staff possess highly-developed technical services skills, competence with serials, strong problem-solving abilities and IT proficiency. These can be onerous staff demands to make and lend weight to the argument for increased national cooperation.

The actual work processes undertaken are briefly outlined below:

2.1 Selection

In building the Archive, the National Library has chosen to take a selective rather than inclusive approach for a number of important reasons. Primarily, it remains a truism that many Internet publications have little current or long term research value. Selecting high quality publications, providing them with full descriptions in the National Bibliographic Database and then ensuring they operate and are accessible seems the best strategy at present (Cameron & Phillips, in press). To fulfil this goal the Library’s current approach is to archive selected publications in a dedicated repository, keep it updated and then as need arises, migrate the data in response to technological change (Smith 1997).

The selection of electronic publications for the archive is governed by PANDORA’s selection guidelines (see http://www.nla.gov.au/scoap/guidelines.html ) They act as a guide and are not strictly prescriptive. Generally, works should be about Australia, or be on a topic of significance and relevance to Australia and by an Australian author, or be written by an Australian of recognised authority and be of international significance. Scholarly publications, publications of national interest and those of current and long term research value are selected. Other publications are included as cultural snapshots or if they represent innovative Internet usage. Some categories of titles excluded are: those with a print or microform equivalent, online databases, discussion lists and bulletin boards, and titles of local rather than national significance.

The Electronic Unit carries out regular Internet searching to locate suitable works but many titles are also referred to the Unit through various channels. All titles considered for selection are brought to regular group selection sessions involving all Electronic Unit staff. These sessions are essential for maintaining the Archive’s integrity and ensuring only the most appropriate titles are included. Due to the complex nature of many electronic publications it is often very difficult to single-handedly decide if a title should be included or not. Titles are never rejected solely on the basis that they will be technically difficult to archive.

2.2 Work processes

The actual process of getting a title into the archive involves:

2.3 Problems encountered

The problems encountered with the above processes are numerous and can dramatically extend the time and effort it takes to archive a publication. Problems frequently encountered include:

The length of time taken to complete all the work processes for even relatively straightforward titles requires high levels of commitment and patience from the staff involved.


3. Issues in archiving and provision of access

The creation of the PANDORA Archive is a tremendous achievement but its existence automatically means that a whole range of other issues must be considered and worked through. Libraries around the world are exploring these issues and the National Library is cooperating internationally in various ways. These issues can be roughly divided into three categories:

3.1 Technology issues

This is the area where the biggest challenges for the future of preserving and managing archives of electronic publications lie.

3.2 Access issues

3.3 Library issues


4. Moving towards a National Collection

The development of a National Collection of Australian Electronic Publications is considered essential by the Library, if Australia is to successfully preserve and make available electronic publications. The huge volume of material and high costs involved in archiving and preservation means networking will benefit all partners. To this end the Library is seeking cooperation from other major libraries in Australia to share the responsibility. In a national model, it is envisaged that State libraries will ultimately store and manage their own distributed archives on their own servers, with all titles accessible nationally through the National Bibliographic Database. In the shorter term the National Library is prepared to host State libraries’ collections if requested. Discussions have also taken place with university libraries although they have not yet been considered as formal partners. At this stage of development, what matters ‘is not how each library manages the material they are responsible for but rather that the end result is a co-operatively managed distributed national archive which is based firmly on an agreed set of principles’ (Cameron 1999). These principles are outlined in 4.2 below.

4.1.Current situation

The State Library of Victoria is currently the only formal partner in PANDORA. It is selecting, archiving and managing Victorian electronic publications and at present is using PANDORA’s server to house its archived material. The National Library of Australia provides help with technical matters and training. The State Library of New South Wales and the State Library of Tasmania are carrying out their own archiving projects. Ongoing discussions are currently taking place with other State libraries and there has been much interest expressed in PANDORA’s aims and objectives.

4.2 Principles for a national model

The National Library considers it vital that for a national model to succeed the following principles are adopted:

4.3 Adoption of international standards

The National Library is participating in discussions with other national libraries to share knowledge gained, and work towards the development of international protocols governing access between digital archives. There is much work being done at the international level in the area of electronic archiving. Significantly, Australia joins only a handful of other countries that have actually created archives of electronic publications and our experience is consequently highly valued internationally.


Conclusion

Hopefully the points made above have helped demystify the concept and practice of electronic archiving. And hopefully this paper has stimulated interest in Australian electronic publishing and its place in the national imprint. It deserves reiterating once again that the only way Australian libraries can effectively manage this emerging information medium is to collaborate.

To conclude this paper, the following quote from Chilvers and Feather (1998) highlights the most definitive aspect of electronic publishing:

When asked if electronic records were short-term in nature an archivist expressed the view that ‘I suppose yes unless you ensure otherwise. That they are short-term in terms of life expectancy of the media and access compared to if they were in print, it is perhaps their most distinctive feature’ (p. 368)

This statement emphasises the inherently transient and fragile nature of Internet publishing. The Internet has entrenched the idea of instant access to information but that does not make this information less valuable. Librarians must treat electronic publications as equal partners to traditional library materials or risk losing this information altogether.

Australians have wholeheartedly embraced the Internet and our libraries must recognise that the Internet is rapidly becoming the first choice for information exchange. With Australia’s vast geographic distances and sparse population, the nation’s libraries must cooperate to manage this phenomenon. In the end it does not matter how we do it, but it does matter a great deal if we do not do it.


Bibliography

Cameron, J. (1999) National Collection of Australian Electronic Publications
(URL: http://www.nla.gov.au/nla/staffpaper/cameron1.html )

Cameron, J. & Phillips, M. (in press) Building national collections of Internet publications. In World libraries on the Information Superhighway

Chilvers, A. & Feather, J. (1998) The management of digital data: A metadata approach. The Electronic Library 16 (6), pp. 365-372

The Penguin Macquarie dictionary: The international dictionary for all Australians (1986) Ringwood, Vic.: Penguin Books.

Smith, W. (1997) Future access: Long term preservation of Australian electronic legal publications
(URL: http://www.nla.gov.au/nla/staffpaper/wsmith2.html)


1 30 May 2001 - Link to Pandora Archive changed from http://www.nla.gov.au/pandora/archive.html to correct link of http://pandora.nla.gov.au/

Top

About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us