National Library of Australia - Gateways>

ISSN 1039-3498 | no. 52| August 2001


Archiving the Web

The challenges of managing and preserving digital information resources were the key themes of a visit to Sweden, Denmark and the Netherlands by Warwick Cathro, Assistant Director-General, Information Technology in June.

The National Library had been invited to present a paper on PANDORA at a conference in Copenhagen entitled ‘Preserving the Present for the Future’. The conference examined strategies for preserving information on the Internet for future generations.

Speakers from the Centre for Internet Research at the University of Århus emphasised the indispensable role of the Internet for contemporary information dissemination. They argued that everything on the web should be archived because contemporaries do not have a good view of what will be significant in the future, but also suggested that ‘selective, context sensitive archiving’ should supplement the non-selective approach. They noted that highly dynamic web sites present difficulties because they can change during the archiving process, creating an authenticity problem. For such sites, deposit of time-stamped snapshots may be a superior approach.

Allan Arvidson from the Royal Library of Sweden presented a paper on the Kulturarw3 Project. This Project is archiving the entire Swedish web domain. At present, no access to the Kulturarw3 archive has been provided due to copyright barriers. The Royal Library is developing ‘surfing’ software to support browsing of the files in both network space and time. However, for the medium term future, access to the archive is likely to be on-site in the Royal Library only.

Warwick Cathro spent a day at the Royal Library in Stockholm prior to the Copenhagen conference, in order to understand in more detail the technical strategy underpinning the Kulturarw3 Project. This understanding will assist the National Library of Australia in its study (which commenced in May) of the feasibility of archiving the entire Australian web domain.

Sverre Bang, from the National Library of Norway, brought the conference up to date on the Nordic Web Archiving Project supporting access, through a search engine, to those national web archives existing in the Nordic countries. (So far they exist for Sweden, Finland and Iceland). The Nordic countries are cooperating for indexing and searching, but will maintain separate web archives. The Project has been funded by both NORDUnet (provider of Internet backbone services for research and education in the Nordic countries) and the Nordic national libraries. The Project has now selected the search engine FAST Search and Transfer ASA, see www.fastsearch.com/.

The copyright and legal deposit issues associated with web archiving were discussed by Martin von Haller Grønbæk (from Von Haller, Solicitors) and Birgit Henriksen (from the Royal Library of Denmark).

Norway is the only Nordic country with full legal deposit for the Internet. The Danish legal deposit law was changed in 1997 to provide for deposit of ‘static’ electronic works (those not continuously updated). The Royal Library is now receiving this material via download from the publishers. The Royal Library is interested in extending the scope of legal deposit to include dynamic material.

The recent INFOSOC Directive upheld the difference between physical and digital forms of information, from the copyright viewpoint. It gives authors exclusive rights ‘to authorise or prohibit any communication to the public of their works’. Furthermore, Web access may be regarded as a ‘public performance’ where the author’s rights are never exhausted. For this reason, most commentators believe that open access to non-selective web archives is not legally possible at present.

Warwick Cathro presented a paper (co-authored by Colin Webb and Julie Whiting) on the PANDORA archive. The full text of this paper can be found at www.nla.gov.au/nla/staffpaper/2001/cathro3.html. The paper described:

the reasons why the Library has undertaken a selective approach to archiving the Web;

the process of gathering, checking and presenting the archived copies of the web sites, and how this is supported by software developed recently by the Library;

the use of persistent identifiers for the items in PANDORA;

the approach which the Library has taken to access control;

the Library’s plans to preserve access to the content of the archive; and

possible opportunities for global cooperation, such as the sharing of information on the best ways to migrate obsolete files, and better information exchange between archives, libraries and researchers working in digital preservation.

After the Copenhagen conference, Warwick Cathro visited the Royal Library of the Netherlands, to gain a better understanding of its project with IBM and the British Library for the development of a digital object management system. The specifications for this project are based on the latest draft (February 2001) of the Reference Model for an Open Archival Information System (OAIS). The National Library and the Royal Library were able to exchange information about their requirements and specifications for digital object management.

The current system used by the Royal Library (which is based on a version of IBM Digital Library) supports the management of digital journal articles deposited by publishers (Elsevier and Kluwer), of digitised collections from the collaborative Memory of the Netherlands project, and of digital theses contributed by some universities.

This was a useful and timely visit that will assist the Library to plan its future web archiving strategies. The work of the National Library of Australia in this area (through services such as PADI and PANDORA) was widely recognised and appreciated.
—Warwick Cathro

| Contents list for this issue of Gateways |

Please address any comments and enquiries to The Editor, Gateways: dthomas@nla.gov.au

Gateways logo