National Library of Australia Home > News & Events > What's On > Conferences & Seminars  
Archiving Web Resources - Issues for cultural heritage organisations International Conference
 National Library of Australia, Canberra 9 - 11 November 2004

Abstracts

The National Library has reproduced this material on the conference web site with permission of the authors. Anyone wishing to reproduce or publish the material should contact the relevant author directly.

Stephen Abrams
Martha Anderson
Robin Batterham
Adrian Brown
Axel Bruns
Penny Carnaby
Warwick Cathro
Paolo Cherchi Usai
Adrian Cunningham
Robin Dale
Tom Desley
Beth Dulabahn
Malcolm Gillies
Timothy Hart
Jane Hunter
Hans Jansen
Michele Kimpton

Brian Lavoie
Hanno E Lecher
Hans Liegmann
Nancy McGovern
Timothy Mackey
Johan Mannerheim
Julien Masanes
James Michalko
Margaret Phillips
Melinda Rackham
Abby Smith
Andreas Stanescu
John Tuck
Caroline Wiegandt
Ian Wilson

The role of file format registries in digital preservation
by Stephen Abrams

Detailed knowledge of the internal properties of digital formats is necessary to interpret properly the full information content of otherwise opaque digital objects. Digital repositories charged with responsibility for the long-term preservation of the digital objects under their care need to be able to identify, validate, characterize and process those objects on a format-specific basis. This is particularly true for harvested Web resources that may be created by processes, and conform to standards and practices, that are outside the direct control or influence of the preservation repository and which may not be accompanied by appropriate, and valid, metadata at the time of submission to the repository. Since format representation information is of potential use to all institutions and individuals engaged in digital preservation, the maintenance of that information can be preformed most effectively and efficiently on a centrally-organised, collaborative basis. The Digital Library Federation (DLF) has sponsored preliminary work towards establishing a Global Digital Format Repository (GDFR) that will function as a sustainable public service for the collection, maintenance, and dissemination of authoritative information about the significant syntactic and semantic properties of digital formats and of systems that support and manipulate those formats. This format representation information can be used in the development of preservation tools and services integrated into repository preservation workflows. GDFR will be implemented as a distributed network of registry caches that communicate via a common protocol. This architecture provides archival integrity through redundant storage, promotes global accessibility, and permits interoperability at the protocol level with existing registry systems, such as PRONOM. In addition to deploying a registry cache reference implementation, the GDFR project will define an editorial process for the technical review of format representation information submitted for registration and will formulate recommendations for a governance structure and business model necessary to ensure the long-term sustainability of the registry. GDFR provides a mechanism to leverage the shared experience and expertise of the wider digital preservation community with regard to the digital formats that are fundamental to many, if not most, preservation concerns and workflows.

Presentation: The role of file format registries in digital preservation


Thematic Web Archiving
by Martha Anderson

The characteristics of high risk of loss and the vastness of the web challenge organizations seeking to collect cultural heritage content from the web. Thematic web archiving was developed as a way to focus on a body of time sensitive at-risk web content. The primary advantages of this approach are economies of scale as well as fidelity to some of the essential characteristics of the content as it is presented on the web.

The Library of Congress began event-based collections with the US national elections in 2000. Recently, the Library began exploring approaches to building topic-based collections focused on larger public policy themes such as health care and terrorism. There are lessons to be learned from traditional collection policies when considering the best approaches to large-scale web archiving. New tools are becoming available that promise to aid the difficult task of identifying and selecting sites within a theme.

Presentation: Thematic Web Archiving


Use of the Web by Australian Scientists
by Dr Robin Batterham

Australia is a relatively small player in global research. Australian research publications account for about 3% of the world's output, and compared to other counties like the United States, Germany, and the United Kingdom, has a modest capacity to invest in research.

Independent peer review of a research paper prior to acceptance for publication is standard practice in reputable professional journals. However, peer review is much more than that. It is central to the organisation of modern science and sits at the base of established work. Peer review is also the process that controls access to funding.

The advent of the World Wide Web, and the competitive and financial pressures that are put on scientists and peer reviewed/learned journals, have made for drastic change. There seems to be nothing that will slow down the amount of information being produced, much of which is being posted on the Web. This offers challenges to the learned journals, which have increased their prices due to demands by publishers. This increase in turn affects scientists and academic libraries, which are unable to purchase subscriptions to all the journal titles needed by their users. When mediocre research is supported, neither science nor innovation is likely to gain much direct benefit.

A word on excellence. Excellent science papers contribute strongly to innovation. A recent review that included the biotechnology sector noted that the most highly cited scientific papers have nine times the prospect of leading to patents, compared to other papers.

So what does the future hold? How do we secure a reliable source of the salient information needed for scientific investigation? What about Public/institutional repositories, mechanisms for review - pay reviewers, author-pays publishing, or a Web based 'chat room' with membership by initial peer review.

Presentation: Use of the Web by Australian Scientists


Building the UK Government Web Archive
by Adrian Brown

In the new world of electronic government, the web is rapidly becoming the prime means of communication between the state and its citizens. In addition to providing a conduit for the rapid dissemination of information, an increasing number of transactions are now taking place in this environment, from submitting tax returns to registering for allowances. We can be sure that the historians of the future will wish to see how issues such as foot and mouth, the Iraq war and SARS were presented by the government over the web. The preservation of government websites is therefore of great importance to the UK National Archives.

We face four key decisions: which websites to preserve, how frequently to archive them, how technically to achieve this, and how to present archived sites to users. The websites in which we are interested fall into two categories: firstly, there are websites relating to specific events, such as public inquiries, which are selected in accordance with existing policies - these have a finite life and only need to be archived at their conclusion, when we can be sure that no further changes will be made. Secondly, we wish to preserve regular 'snapshots' of websites across the full domain of government activities and functions. To this end, we have developed a selection policy, which identifies 60 core sites covering all the major activities of government and responsibilities of cabinet ministers. The frequency with which these sites are archived is flexible - during an event such as the Iraq War, we might wish to take certain defence and foreign policy-related sites on a weekly basis, whereas at other times we might be content to take a 6-monthly snapshot.

Technically, TNA is taking three approaches to collecting websites: capturing directly from source, contracted-out remote harvesting using the services of the Internet Archive, and in-house remote harvesting via a common infrastructure developed by the UK Web Archiving Consortium. Each approach has particular advantages and disadvantages. We are also investigating various methods for delivering the collected sites to users, and the issues which this raises.

To date, TNA has archived over 15 million web pages, and is adding to the collection at a rate of approximately 0.25 million pages per week. This paper will discuss the issues involved in setting up a web archiving programme, including selection, collection, quality assurance, delivery, and preservation.

Presentation: Building the UK Government Web Archive


Contemporary culture and the Web
by Dr Axel Bruns

Contemporary uses of the Web are multiplying and diversifying rapidly, and are taking on increasingly complex and interconnected formats. What once were relatively static and simple homepages, for example, are now highly responsive database-driven Weblogs which are themselves further interconnected as social, intellectual, and technological communities and networks; what used to be Websites intended merely for the display of information and knowledge are now interactive and editable knowledge bases using Wiki technology or other content management systems. Information available in these sites is often changed on a daily or hourly basis, and the ideas moving through them spread more like information viruses (carried by the Web as well as by other means of communication from SMS to television) than along the pathways prescribed by traditional channels of publication. Editorial practices follow heterarchical rather than hierarchical models, and users are significantly involved as content contributors. Such features present significant new challenges for the evaluation and archiving of Web content.

Presentation: Contemporary Culture and the Web


National Library of New Zealand Web archiving strategy
by Penny Carnaby

What do Web archiving and the World Summit on the Information Society and a national digital strategy have in common?

This paper explores the strategic journey of the National Library of New Zealand in securing legislation that brings legal deposit into an electronic domain and the recent budget announcement of $NZ24 million to build a trusted digital repository to preserve New Zealand's sound, vision and textual memory, including options for the archiving of the .nz domain.

This paper outlines the challanges, options and learning of the journey so far and explores the potential for international collaboration in the future.

Presentation: National Library of New Zealand Web archiving strategy


Preserving the Outputs of Research
by Dr Warwick Cathro

Research generates primary outputs (such as data sets) and secondary outputs (such as scholarly publications). These outputs usually exist as Web accessible digital files.

Cliff Lynch's seminal paper on institutional repositories has been one stimulus for a major effort internationally in the higher education sector to improve the management and preservation of these research outputs. The success of this effort will depend more than anything else on the policies and management processes adopted in this sector.

This paper will review the current status of the institutional repository movement as it relates to the preservation of research outputs. The review will cover programs such as the Digital Curation Centre in the United Kingdom, and the newly formed Australian Partnership for Sustainable Repositories.

This paper will discuss the role of national scientific and cultural institutions as a safety net in the long-term preservation of this information.

Presentation: Preserving the outputs of research


Digital is Politics Pursued with Other Means
by Paolo Cherchi Usai

Like nuclear power, digital technology can improve our lives or be used as a weapon of mass destruction. In the audiovisual archive and museum world, the boundary between these two options is somewhat confused. We have perceived an opportunity to make our cultural heritage available to a much broader audience, but we have no clue on how to monitor - let alone control - its side effects: the industry is influencing the principles, methods and legal parameters of management of the collective patrimony; public and private support to archives and museums has become contingent upon an explicit endorsement of the Internet ideology. For the first time in history, economy is actively reshaping the past. Thanks to its impermanence, the Web might become the audiovisual archivist's best chance to demonstrate it, and it should be exploited as such.


Managing government electronic records through the whole life cycle
by Adrian Cunningham

Four years ago, the National Archives of Australia (NAA) published policies and guidelines for Australian government agencies on the archiving of Web-based resources. This guidance was issued as part of the Archives' e-permanence suite of modern recordkeeping standards and guidelines. The aim of these policies and guidelines was to improve the quality of recordkeeping by government agencies in an age when technological change and changing work practices were making good records an endangered species. This initiative was consistent with the NAA's objective of ensuring that the right records are properly kept and maintained by the records' creators. While there are strong administrative, legal and accountability reasons why it is important to influence the behaviour of records creators, it is also critical if archivists are to have any hope of preserving a reliable record of governance in the early 21st century for the benefit of future generations. Since 2000 / 2001, the NAA has expanded its emphasis from issuing standards and guidelines to records creators to include working with agencies to implement better practices and to audit or self-assess progress towards that goal. Having put in place a strategy for influencing digital records creation, the Archives has more recently given a high priority to developing a digital preservation strategy for records of archival value. In September this year, the NAA launched its XENA suite of open-source XML-based software tools for normalising records created in proprietary application environments. This presentation describes the various digital recordkeeping initiatives pursued by the NAA specifically in relation to how these initiatives relate to the making and keeping of Web-based records.

Presentation: Managing government electronic records through the whole life cycle


Establishing trustworthy repositories: organizational models, roles, and imperatives
by Robin Dale

Since 1994 when early seminal thinking about the preservation of our digital cultural heritage was codified in Preserving Digital Information, there has been a call for the establishment of trusted - or more appropriately, trustworthy digital repositories. Acknowledging that a network of repositories would be required to preserve the vast and diverse content, the experts behind the report asserted that such a networked system of national repositories could only be held together through two essential mechanisms: a. repositories needed to be able to prove their capabilities through a process of independently administered certification, and b. repositories required "critical fail safe" mechanisms allowing them to exercise an aggressive rescue function to save culturally significant digital information.

Ten years later, many institutions - including national libraries and archives - have embarked upon the process of building digital repositories for the long-term preservation of our digital cultural memory. In doing so, these institutions have joined - intentionally or unintentionally - the greater international, networked effort envisioned back in 1994. Though these activities are positive steps toward the preservation of culturally significant digital information, we still lack the critical mechanisms identified by Preserving Digital Information. With this in mind, do we have an international system of digital repositories in place which is trustworthy? Can we rely upon the emerging digital repositories to preserve the information correctly? How can we trust that other repositories are capable of filling the roles and responsibilities relegated to them as a part of the larger effort? Shouldn't trustworthiness require not only the ability to trust partner and peer repositories, but also verification of that trust? The big questions related to digital repository trustworthiness could be addressed by an independently administered certification program. But since such a program is not yet available, can we as a community afford to wait for such arrangements? Should we instead consider different organisational models for digital preservation and instead build centralized repositories or can we rely upon the vision of the distributed networks of repositories?

This presentation will address these difficult questions and will discuss the different roles some institutions have taken to address the issue of trustworthiness as the grapple with establishing workable, sustainable archiving and preservation arrangements. It will also draw on the ongoing work to establish a certification program for digital repositories charged with preserving our digital cultural memory.

Presentation: Establishing trustworthy repositories: organizational models, roles and imperatives


Resource Discovery and Archived Resources: Approaches and Issues
by Tom Delsey

In planning for the archiving of Web resources, the host repository will have to address a number of key issues relating to the provision of metadata to support resource discovery. Assessments will have to be made as to the need for supplementing data that can be extracted automatically from the resources themselves and the tradeoffs that might be made between enriching the metadata and enhancing the retrieval software. The distinctive characteristics of Web resources and the relationship between the "live" resource and the archived resource will add another level of complexity when evaluating metadata schema and "content rules" for description. Cost will be a major factor with respect both to metadata creation and to software development, and the cost factor in turn will require assessing the feasibility of pooling resources with other repositories in a cooperative endeavour.

This presentation examines metadata for archived resources in the context of the network technologies that support resource discovery, and provides a framework to assist repositories in addressing their resource discovery requirements both individually and collectively. It focuses on the nature of the archived resource at one end and the expectations of the user at the other end, as well as on the interfaces between the resource, the metadata, and the user.

Presentation: Resource Discovery and Archived Resources: Approaches and Issues


The National Digital Information Infrastructure and Preservation Program (NDIIPP): Future Directions and Relevance to Other Countries
by Beth Dulabahn

The U.S. Library of Congress is responsible for a national program to preserve significant digital content that is at risk of loss. Through NDIIPP, the Library is building a network of partners consisting of federal agencies, state governments, universities, private companies and cultural heritage organisations. Parallel efforts are also underway to develop tools, models, methods and technology to support digital preservation.

NDIIPP has three focus areas for its implementation: 1. network of preservation partners; 2. preservation and architecture; and 3. digital preservation research agenda. The network of parteners is the program's cornerstone. The volume, diversity and complexity of digital content compels a distributed, collaborative approach that enables individual organisations to manage content while also sharing best practices for preservation. Partnership network operations are guided by the preservation architecture, which is a conceptual framework for deploying infrastructure components. The research agenda supports exploration of the advanced technology and methodology needed to sustain digital content over time.

The Library has made important recent progress in all three areas. A highlight is the launch of Partnership Arrangement 1, where the Library has committed $US 15 million to eight consortia consisting of 25 partners to select and preserve a diverse range of content. Among the types of content the partners will work with are web sites, geospatial data, social science data sets, business records and TV broadcasts.

NDIIPP is relevant to other countries for two basic reasons. First, it provides a model for a national approach to digital preservation with one major cultural heritage institution serving as a facilitator to engage other stakeholders and help them work together on the basis of common interests. Second, NDIIPP is itself actively engaged on the international front. One example is the Library's membership in the International Internet Preservation Consortium and its commitment to help develop the methods that all nations need for effective web archiving.

Presentation: The National Digital Information Infrastructure and Preservation Program (NDIIPP): Future Directions and Relevance to Other Countries


Born Digital Born Free? The Cultural Impact of the Web
by Malcolm Gillies

The Web impacts on contemporary society as the printing press did upon the Renaissance. Today's "data deluge" is producing massive amounts of information, yet our strategies for accessing, analysing and preserving "virtual spaces" over time are still in an embryonic stage. The curatorship of born-digital materials requires greater global coordination and collaboration than was needed in past ages for the acquisition and storage of passive print forms of knowledge. The digital environment provides a hugely expanded range of creative challenges and opportunities: in film, music, sound, the visual arts and multimedia. Human and social outputs require preservation for posterity through new, systematic approaches to digital archiving. Will "born digital" also mean that collecting institutions will themselves be transformed? Professor Gillies' paper examines the cultural frameworks for the collecting of digital artefacts and the social, economic and political actions necessary for todays digital memories to be available to tomorrows citizens.

Paper: Born Digital Born Free? The Cultural Impact of the Web


Museums and the Web
by Timothy Hart

Museums in Australia have been using the web since 1994. The first museum website in Australia was developed at the Western Australian Maritime Museum; it’s still accessible today at http://web.archive.org. From these first tentative steps there are now over 510 Australian Museum websites listed on Australian Museums Online (AMOL). Between 1995 – 2000 there was an explosion of creativity and exploration as museums experimented with how to utilise the web based medium of information dissemination and collaboration – following the dotcom collapse things have steadied – technologies have matured and websites have become part of the core business of all major museums across the world.

This paper will examine the evolution of museum websites concentrating on Australia with some important international examples. Museum websites provide in-depth information, collection based stories, research, reference information and educational resources, I will illustrate the rich diversity of content and information architecture approaches taken by museums in Australia in 2004. The web has made many things possible that we could simply not do before, the number and complexity of national and international collaborations using collection data for example, Australian Museums Online (AMOL), Picture Australia, Ozcam, Australia’s Virtual Herbarium (AVH) to name a few, projects which have all played a part in developing and making accessible the distributed national collection.

In future museum websites will continue to evolve into richer and more compelling visitor experiences – aligned and complementary with the actual experience visitors have within the physical museum – a recent article in Museums Journal reported that a recent survey has placed museums second to libraries as the two most trustworthy sources of information in our society. This article reminded me of Lynn Elliott Sherwood’s words, who as the head of the Canadian Heritage Information Network (CHIN) in 1997 said that “museums are islands of trust on the internet” – I believe that we have to date remained faithful to that observation and have managed to maintain our reputations as scholarly institutions in the face of the edutainment onslaught of which the web is a major part.

The preservation of our websites has not been addressed in any meaningful way to date! - Certainly an issue for organisations based on collections and collecting!


The PANIC Project - A Scalable, Collaborative Approach to Preservation Based on Semantic Web Services
by Jane Hunter

It is widely recognized that there is no single best solution to digital preservation. The most appropriate strategy depends on the particular requirements of the custodial organization, the producers and users of its collection and the nature of the objects in the collection. In this paper we describe a scalable, flexible, semi-automated approach to preservation which leverages existing work on preservation metadata, registries, tools and services by integrating them and making them available through a Semantic Web Services architecture.

The system which we have developed provides metadata extraction and capture tools which enable the recording of preservation metadata for digital objects in a collection. The collections' preservation metadata is periodically compared with software and format registries (e.g, GDFR). When an incompatibility is detected, a notification is sent to the relevant agent (human or software). If preservation action is requested, then the system dynamically discovers or composes the most appropriate preservation service(s). This is implemented by making preservation software modules available as Web services and describing them semantically using a machine-processable ontology (OWL-S). Collection managers then have the option to choose and invoke a particular service from the list of preservation services retrieved and recommended by the software agents. Consequent preservation events are recorded.

We believe that the semi-automated, distributed, Semantic Web Services approach which we have developed within PANIC represents a significant advance towards a viable, cost-effective solution to the long term preservation of large scale collections of digital objects.

Presentation: The PANIC Project - A Scalable, Collaborative Approach to Preservation Based on Semantic Web Services


Permanent access: the e-Depot at the Koninklijke Bibliotheek
by Hans Jansen

Together with IBM the Koninklijke Bibliotheek, National Library of the Netherlands, has developed the e-Depot. The e-Depot is a system that enables us to safely store and give access to the electronic output from two types of publishers. The first group comprises Dutch publishers that for the greatest part are represented by the Dutch publishers association. For this group a general agreement is in place that is an extension to the one for printed publications. The second group consists of the publishers 'without boundaries', the international STM-publishers. The KB has signed individual archiving agreements with these publishers to deposit their electronic journals at the KB. The presentation will focus on three topics that play an important role within our e-Depot environment: what are these agreements with publishers, what are the requirements for permanent archives and what kind of strategies do we see for permanent access?

Paper: Permanent access: the e-Depot at the Koninklijke Bibliotheek


Saving the Web for future generations
by Michele Kimpton

The Internet Archive (IA) is a 5013C non-profit corporation, whose mission is to build a public Internet digital library. Over the last 6 years, IA has built the largest public web archive to date, hosting over 400 TB of data. The web collection is comprised of billions of web pages from over 35 million sites around the world.

Attempting to archive the publicly available web appears to be such an enormous task that many believe it is not possible and some question the value. However, an equally challenging task is deciding what to selectively archive for future generations today. Due to the everchanging content on the web, and its seamless borders, one can argue the cost of selection is greater and more risky than the cost of trying to save it all.

Internet Archive will present its approach in archiving the global web. The techniques used to collect, store and provide access to the data, the challenges faced in collecting and preserving billions of web pages, and the benefits and compromises made in large scale web archiving will be discussed.

Presentation: Saving the Web for future generations


Preservation metadata initiatives: status and direction
by Dr Brian Lavoie

Preservation metadata is information needed to carry out, document, and otherwise support repository functions and policies directed at securing the long-term retention of digital materials. Regardless of the specific technique chosen to preserve digital materials, metadata will nearly always play a critical role. Most current digital preservation initiatives have implemented preservation metadata schemas in one form or another, and there has also been collaborative activity within the international digital archiving community to develop standardized preservation metadata applicable across a broad range of digital preservation activities. The outcome of this work suggests that it is indeed possible to define a "core" set of preservation metadata elements, but cross-institutional differences in repository architectures, technologies and preservation policies necessarily limit the scope of this core set. Nevertheless, there are potential opportunities to cooperatively create and share preservation metadata and the international digital archiving community would benefit from further exploration of this issue. One concern about preservation metadata is the effort and expense required to create and maintain it. Preservation metadata that is shareable and reusable would help mitigate this problem, but beyond this, automatic tools for managing preservation metadata are needed as well. Some progress has been made on this front, but much work remains to be done. Developing such tools would be greatly assisted by a shared view in the digital archiving community on what metadata is needed to support digital preservation.

Presentation: Preservation metadata initiatives: status and direction


Informant networks, alarm systems and research contributors: selection and ingest process for the Digital Archive for Chinese Studiesby Hanno E. Lecher

In China, the Internet plays a peculiar role: the government of the Peoples' Republic of China views it as absolutely necessary for the development of the country, but at the same time it is also afraid of its incalculable threat to their grip on power. This results in a fascinating quality of the Chinese Internet: its infrastructure is very well developed and used as a major channel for public discourse by subcultures, dissidents and party conformists alike, while at the same time it is also closely monitored and controlled by the government. Content perceived as being too critical or "dangerous" for the public is quickly removed and its publishers not seldomly punished.

Obviously we cannot expect the Chinese government to archive and publicise these data, but unquestionably they are an essential part of the Chinese cultural heritage and form an important resource for research. In order to make at least parts of these resources available for the future the Sinological Institutes of Heidelberg University (Germany) and Leiden University (Netherlands) have begun to archive parts of the Chinese Internet that reflect social and political discourse in or about China.

Based on the fact that the project is run by two rather small institutions this presentation focuses on an issue that is central to is success: how to develop strategies for selection of relevant resources and how to provide a meaningful environment to make their relevance comprehensible even in the future.

Presentation: Informant networks, alarm systems and research contributors: selection and ingest process for the Digital Archivie for Chinese Studies


Die Deutsche Bibliothek: cooperative efforts on the European and national level
by Hans Liegmann

Die Deutsche Bibliothek is the national library of Germany, a country with federal structure: 16 individual German states are responsible for their own cultural affairs. Germany is also part of the European Union, which has enlarged to 25 member states since May 2004. Member states of the EU are urged to pay particular attention to the need to sustain and promote cultural diversity. Cooperation, division of labour and virtual integration of autonomous efforts are key elements to keep cultural diversity possible and alive in a world of reduced financial budgets and limited funding. The EU framework program and project TEL - The European Library as an example for a pan-European virtual information service will be outlined. German activities and projects concerning collection and preservation of digital resources will be highlighted.

Presentation: Die Deutsche Bibliothek: cooperative efforts on the European and national level


Virtual Remote Control: A risk management approach
by Nancy Y. McGovern

Virtual Remote Control (VRC) is Cornell's risk management approach for Web resources. Virtual because the approach develops abstract models to represent essential features of selected Web sites that enable ongoing monitoring of these sites over time to identify, respond to, and mitigate potential risks to their integrity and longevity. Remote because the approach is intended for use by cultural heritage organizations interested in the longevity of Web resources residing on remote servers, specifically sites administered by another organization that pertain to or extend the monitoring organization's collecting areas. Control because at the most proactive end of the VRC approach, a monitoring organization exerts control over another organization's resources by agreement or implicit consent through notification and action. The VRC approach includes but does not presume the capture of Web sites because a monitoring organization may not have the means, authority or desire to capture all or some iterations of a Web site. Ongoing monitoring and evaluation allows monitoring organizations to intelligently manage Web resources over time. In conducting our research, we have learned a lot about good Web site management and about promulgating good practice to encourage Web longevity. We have developed a six-stage VRC model, a Web tool evaluation method and inventory, and a control site for evaluating Web tools.

Presentation: Virtual Remote Control: A rish management approach


Maps: the impact of the Internet
by Timothy Mackey and Geoff Lawford

Topographic and other thematic maps have been archived by libraries as a permanent historical record in the same way as they archive books and other media. Despite the move to digital cartography in the 1980's, until recently the primary method of map distribution remained as mass produced hard copy maps.

The advent of database technology and the Internet have led to a shift away from mass production of maps. Instead of the data for a geographic area being updated once every twenty years, the data are now continuously updated. Changes to the data (including previous versions) are recorded in a relational database. Clients commonly use the Internet to access the data as unique digital data files or "print on demand" maps that are derived from the latest version of the data, as recorded in the database.

These new products represent a paradigm shift for libraries and archivists, as a centralised archive of products is no longer practical. Geoscience Australia is addressing this issue through the Seamless Topographic Database Project. The project offers a potential model for the ongoing archival of map data. The system architecture makes it possible for clients to request data as it was at virtually any time since the project was initiated.

This model offers benefits over traditional archival practices, as the custodian is able to manage the entire "information life cycle", from data creation, through analysis, distribution and archival.

Presentation: Maps: the impact of the Internet


Collect all, catalogue some
by Johan Mannerheim

The Royal Library has collected the Swedish web comprehensively since early 1997. Without support by deposit law the library has sent robots to search for and download all Swedish pages about twice a year. It has thus created an ephemera collection of online publications parallel to the traditional ephemera collection of Swedish print, which is searchable but not catalogued. The printed collection totals over 10 million items, while the digital in only seven years has reached 260 million files fetched from 280 thousand web sites. The digital collection has a tremendous research value reflecting the early growth of the web.

But the comprehensive, automatic approach has its limitations. The collection is not complete. A recent government report suggests, supported by the Royal Library, that deposit plight should comprise all online publications, also protected ones. The robot collection of today should be supplemented with complete collection of material spread over the Internet by Swedish professional publishers and producers. Many of these documents should also be catalogued.

Presentation: Collect all, catalogue some


International Internet Preservation Consortium (IIPC) work program to develop Web collecting tools
by Julien Masanès

The development of tools for web archiving is one of the major goals of the International Internet Preservation Consortium (IIPC). Acknowledging the lack of appropriate and scalable tools in this domain (even if much can be borrowed from traditional IR and search engine technology), IIPC members have addressed this problem by launching several software development projects.

Using common tools, when possible, is also key for facilitating future exchange, cross access and sharing of collection. In so doing, IIPC members intend to help laying the ground for collaboration, a necessity when considering the size and border less nature of the Web.

IIPC's achievements in this domain (Heritrix, DeepArc, eXplore) as well as its strategy for the future will be presented.

Presentation: International Internet Preservation Consortium (IIPC) work program to develop Web collecting tools


Key issues for cultural agencies
by James Michalko

The organizations that will collect the materials of the web are only just beginning to take shape. There will be a multiplicity of these agencies, from across all the cultural sectors, that ultimately emerge as a system of collection, access and preservation. For this emergent system to satisfy all the audiences as well as the diverse demands for this archived material a common set of service expectations must be established. Those expectations can only be fulfilled if the guiding policies and the supporting infrastructures of these agencies acknowledge their role in a broader system. This talk will capture some of the issues that must be addressed, reflect on the current state of progress and consider the fiscal dimensions of the challenge.

Presentation: Key issues for cultural agencies


The selective approach to web archiving at the National Library of Australia
by Margaret Phillips

In building PANDORA, Australia's Web Archive, the National Library of Australia and its partners have taken a selective approach to the archiving of Australian online publications and web sites, because it enables us to meet our primary collecting objectives. This paper analyses the advantages and disadvantages of the selective approach, outlines the selection priorities, describes some of the characteristics of the PANDORA Archive, and refers to two research projects underway that are expected to increase the volume of titles and to extend the range of dynamic formats that can be archived.

Presentation: What to collect and how to do it: the National Library of Australia's selective approach


The art of the network
by Dr Melinda Rackham

Today, on-line, we are surrounded by many forms of Internet art. From streaming video, elegant interactive Java works, electronic poetry, net.radio and tactical media, to the phenomenally successful genre of intimate video or panoramic blogs. We may interact within artist-created multi-user Virtual Reality and Game spaces, and watch new theatrical and cinematic forms like avatar performance and machinima.

Flash animation has penetrated almost every corner of the Web; and the network itself becomes an artwork as software art and browser interventions reformat and recontextualise HTML code. Net.art creates a mixed reality when data captured from one space is transposed and revisualised in another location; or when the monitored interactions of both artists and users becomes the bio-input for an artwork.

While these works provide entertaining cultural experiences, they also raise many challenges for preservation as they often utilize specific software and hardware, and/or the traffic, rhythm and compressions of the network itself as content. Can these works be successfully archived or migrated to retain their aesthetic and artistic intent? Or can only a slice be documented? These are critical questions as pioneering net.art works, from less than a decade ago, already have compromised functionality, and may soon be lost.

Paper: The Art of the Network


The future of Web resources: who decides what gets saved and how do they do it?
by Dr Abby Smith

The World Wide Web- massive in scale, virtually barrier-free for publishers, and highly dynamic- has considerably complicated the preservation function of libraries and archives. Beyond the technical challenges of capturing and stabilizing data, the Web has made traditional and deliberative processes for selecting and preserving information resources obsolete. Abby Smith will talk about the challanges of selection in this new environment: who should be responsible for institutional selection policies; what criteria they should consider in making those decisions; and how will libraries and archives redefine themselves in light of this data deluge. She will review a number of approaches currently underway.

Paper: The future of Web resources: who decides what gets saved and how do they do it?


Assessing the Durability of Formats in a Digital Preservation Environment: the INFORM Methodology
by Andreas Stanescu

INFORM (INvestigation of FOrmats based on Risk Management) is a methodology for investigating and measuring the risk factors of digital formats and providing guidelines for preservation action plans. In other words, this methodology attempts to discover specific threats to preservation and measure their possible impact on preservation decisions. Moreover, by repeating the process, involved parties can detect changes in the threat model over time, to which digital archives can act accordingly.

Digital archives, institutional repositories and digital libraries can take advantage of the measurements offered by the INFORM method to select digital formats most apt for long term viability. While individuals are biased and subjective in their aversion to risk, collating the assessments of many individuals should generate group-consensus or group-averaged objective results. Hence, preservation plans can be based on objective analysis of risk trends instead of individuals' opinion developed in the relative isolation of their institutions.

Presentation: Assessing the Durability of Formats in a Digital Preservation Environment: the INFORM Methodology


The British Library: Web Archiving Challenges and Initiatives
byJohn Tuck

The Legal Deposit Libraries Act 2003 extended the principle of legal deposit in the United Kingdom to the non-print world. Within the context of the new legislation and its progress, the presentation seeks to outline the challenges faced and progress made by the British Library in developing a strategy and operations for archiving the UK web presence. Reference will be made to the pilot Domain.uk project of 2001 upon which is built the current Web Archiving Programme. This includes membership of the UK Web Archiving Consortium and the International Internet Preservation Consortium. Challenges of the collection development policy, staff resources and preservation through the Library's Digital Object Management programme will be covered.

Paper: The British Library: Web Archiving Challenges and Initiatives


International Internet Preservation Consortium (IIPC): objectives and workplan
by Caroline Wiegandt

Archiving the Web is both a necessary and challenging task that no institution can achieve alone. The Web is not only immense, it is also rapidly and unforseeably changing. It represents a highly distributed and borderless mass of documents which real historical signification can only be reflected at this scale. The necessity of cooperation among heritage institutions is thus manifest when considering the nature of this new media.

The International Internet Preservation Consortium was formed to help in making effectual steps in this direction, bearing in mind the importance of gaining practical experience and building common expertise to be effective. This new domain of Web archiving indeed raises new issues not only at the technical level but also on selection, organisation and processing of collections. Through the development of common methods, tools and protocols, the IIPC is trying to help build future interoperability of heritage collections as well as best possible coverage for this media of historical preservation.

The IIPC has divided its activity into Working Groups to share experience and prepare reports and protocols and projects to share the development of tools. The six Working Groups are working to define a common technical framework, assessing the researchers of internet needs, defining applicable metrics and test beds, access tool requirements, methods for archiving the 'deep' or 'hidden' Web, and collection management. A rapid overview of the first results and workplan of these Working Groups will be presented.

The main tools developed at this stage are the Heritrix archiving crawler developed by Internet Archive and the Nordic Libraries, Deep Arc and eXplore for archiving databases of deep Web sites developed by bnF and the NLA. The are or will be soon available freely and in open source.

Presentation: International Internet Preservation Consortium (IIPC): objectives and workplan


Library and Archives Canada: Digital and Metadata Initiatives
Dr Ian Wilson

Library and Archives Canada is an innovative knowledge institution that combines the collections, services and staff expertise of the former National Library of Canada and National Archives of Canada. Our objective is to provide all Canadians with easy, one-stop access to the texts, records, photographs and other multi-media documents that reflect their cultural, social and political development. Working closely with other archives and libraries, we will continue to acquire and preserve Canada's documentary heritage in all its forms. Our mandate is: to preserve the documentary heritage of Canada for the benefit of present and future generations; to be a source of enduring knowledge accessible to all, contributing to the cultural, social and economic advancement of Canada; to facilitate in Canada cooperation among communities involved in the acquisition, preservation and diffusion of knowledge; and to serve as the continuing memory of the government of Canada and its institutions.

Catalytic initiatives were established earlier this year to act as agents of change within LAC. They are "catalytic" because they are focused on challenging areas in which LAC must explore, invest and innovate if it is to realize its vision of the future. The proposed scope and activities of the catalytic initiatives will involve many areas within the new organization and will have a broad impact. They will explore issues and bring about change in key work processes through research and demonstration projects, studies, policy work, and implementation projects. Two key catalytic initiatives that Ian Wilson will describe today include the digital collection initiative and the metadata initiative. The digital collection initiative will focus on the challenges of acquiring, describing, managing and preserving access to digital content, including web resources. The metadata initiative will develop a metadata framework to facilitate delivery of content and services to users.

Presentation: Library and Archives Canada: Digital and Metadata Initiatives


The National Library has reproduced this material on the conference web site with permission of the authors. Anyone wishing to reproduce or publish the material should contact the relevant author directly.