Staff Papers

Executive Summary on Senior Executive Fellowship to Research Digital Archiving in National Libraries

This provides a summary of the study trip undertaken by Pam Gatenby (Assistant Director General, Collections Management) on the Senior Executive Fellowship awarded to research developments in digital archiving among national libraries. The Fellowship enabled attendance at a major international library conference and visits to libraries active in the area of digital archiving. The study trip was undertaken between August 16 and September 10 2002.

A large part of the documentary heritage of countries is now published in online form only. Important information is being disseminated via the Internet from a wide range of sectors. Unless steps are taken early in the life of online information to prevent it from disappearing, its availability for use will be short-lived and future generations will be denied access to an important component of their documentary heritage.

There are many complex issues involved in ensuring important information in online form remains available for use in the long-term. They relate to the ephemeral nature and rate of growth of the Internet, deciding what is worth saving, the rapid rate of change of the technology on which digital publications depend, and a raft of legal and economic concerns.

Most national libraries now accept that their traditional role of collecting and safeguarding access to information published in their countries must extend beyond print based publications to those in online form. However, national libraries are at very different stages in developing their responses to this new responsibility.

The National Library of Australia has been at the leading edge of world developments in collecting and archiving cultural heritage information resources in online form since 1996, when it established the PANDORA (Preserving Access to National Documentary Online Resources of Australia) project. Collecting online resources is now a routine activity. The Library accepts a special responsibility for collecting and preserving the published output of Commonwealth government agencies and it is a priority for the Library to develop collaborative approaches with agencies to do this. Collaboration with other creators and publishers of online resources is also an essential part of the Library’s strategy.

Some other national libraries have also been collecting online resources routinely for a few years now adopting either a selective approach like Australia, or the Whole Domain Harvesting approach. Others are still experimenting with various approaches and researching solutions to particular problems.

At the broad level, the chief purpose of the Senior Executive Fellowship was to research developments among national libraries in collecting and managing publications in online digital form and in providing on-going public access to them. At a more specific level, I was interested in investigating developments relevant to particular challenges the Library is currently facing. These relate to the need to extend the scope of our collecting, managing more complex web resources, and to developing more cost-efficient ways of operating.

To maximize the benefit to be gained from the Fellowship study trip, I visited key national libraries, which are active in digital archiving, and I also attended and participated in the annual International Federation of Library Associations (IFLA) Conference. The libraries visited were the national libraries of Sweden, Denmark and Canada, and the Library of Congress. I also visited OCLC, a large US library consortium providing online services to research libraries in the United Sates and around the world.

The study trip was extremely useful in providing current information on the digital archiving activities of national libraries and insight into their concerns and intentions. The observations made during the trip and the conclusions I reached have helped me determine the Library’s digital archiving action plan for the coming year. (This is outlined in the report.) The main observations and conclusions were:

  • National libraries are committed to carrying out their responsibilities for digital resources but are cautious because of the costs and issues involved. Many expressed the need for more professional guidance and discussion on the issues.
  • Those libraries with experience are at a crossroads, reviewing their current approaches with the view to addressing acknowledged limitations.
  • Australia is well positioned compared to the countries visited with regard to the extension of legal deposit and copyright to digital resources.
  • National libraries on the whole do not develop special partnerships with the government sector except that the National Library of Canada is doing some interesting work in this area.
  • Sustainable solutions to managing complex online resources (eg. those that are database driven or dependent on specific software for access) are still some way off. Major projects such as the National Digital Information Infrastructure and Preservation Program at the Library of Congress could address these issues in time.
  • Some of the shortcomings of the Whole Domain Harvesting approach are being addressed through projects like the Nordic Web Archive project, and the work of the Internet Archive. However, there are still major barriers to be overcome before this approach is a viable option for national libraries.
  • Collaboration is considered essential by all institutions with a responsibility for online resources. A forum for collaboration that has emerged is the CDNL Digital Issues Group.

The overall conclusion I reached as a result of the Fellowship study trip was that the National Library of Australia is still at the forefront of digital archiving and preservation activity among national libraries. The Fellowship enabled me to confirm that the Library's approach to building its collection of significant Australian online resources and to meeting its objectives in the area of digital archiving, is still sound. A more cost-effective approach is not yet in place at any other national library or library related organization. However, activities are taking place that we must keep up with and take into account in our future planning.

The main implication of the Fellowship findings for the wider APS is that agencies must be active partners in ensuring their online publishing remains available to the public into the future. The National Library cannot do this alone and at this stage there would appear to be no easy technology based solutions emerging that agencies can rely on. To carry out their roles effectively, agencies that have not already done so would need to commit resources and expertise to the digital archiving aspects of their online publishing activities.

Back to the top

  • Staff Papers
  • Senior Executive Fellowship to Research Digital Archiving in National Libraries