Abstracts
The National Library has reproduced this
material on the conference web site with permission of the authors. Anyone
wishing to reproduce or publish the material should contact the relevant author
directly.
Detailed knowledge of the internal properties of digital formats
is necessary to interpret properly the full information content of otherwise
opaque digital objects. Digital repositories charged with responsibility for
the long-term preservation of the digital objects under their care need to be
able to identify, validate, characterize and process those objects on a
format-specific basis. This is particularly true for harvested Web resources
that may be created by processes, and conform to standards and practices, that
are outside the direct control or influence of the preservation repository and
which may not be accompanied by appropriate, and valid, metadata at the time of
submission to the repository. Since format representation information is of
potential use to all institutions and individuals engaged in digital
preservation, the maintenance of that information can be preformed most
effectively and efficiently on a centrally-organised, collaborative basis. The
Digital Library Federation (DLF) has sponsored preliminary work towards
establishing a Global Digital Format Repository (GDFR) that will function as a
sustainable public service for the collection, maintenance, and dissemination
of authoritative information about the significant syntactic and semantic
properties of digital formats and of systems that support and manipulate those
formats. This format representation information can be used in the development
of preservation tools and services integrated into repository preservation
workflows. GDFR will be implemented as a distributed network of registry caches
that communicate via a common protocol. This architecture provides archival
integrity through redundant storage, promotes global accessibility, and permits
interoperability at the protocol level with existing registry systems, such as
PRONOM. In addition to deploying a registry cache reference implementation, the
GDFR project will define an editorial process for the technical review of
format representation information submitted for registration and will formulate
recommendations for a governance structure and business model necessary to
ensure the long-term sustainability of the registry. GDFR provides a mechanism
to leverage the shared experience and expertise of the wider digital
preservation community with regard to the digital formats that are fundamental
to many, if not most, preservation concerns and workflows.
Presentation: The role of file
format registries in digital preservation
The characteristics of high risk of loss and the vastness of the
web challenge organizations seeking to collect cultural heritage content from
the web. Thematic web archiving was developed as a way to focus on a body of
time sensitive at-risk web content. The primary advantages of this approach are
economies of scale as well as fidelity to some of the essential characteristics
of the content as it is presented on the web.
The Library of Congress began event-based collections with the US
national elections in 2000. Recently, the Library began exploring approaches to
building topic-based collections focused on larger public policy themes such as
health care and terrorism. There are lessons to be learned from traditional
collection policies when considering the best approaches to large-scale web
archiving. New tools are becoming available that promise to aid the difficult
task of identifying and selecting sites within a theme.
Presentation: Thematic Web
Archiving
Australia is a relatively small player in global research.
Australian research publications account for about 3% of the world's output,
and compared to other counties like the United States, Germany, and the United
Kingdom, has a modest capacity to invest in research.
Independent peer review of a research paper prior to acceptance
for publication is standard practice in reputable professional journals.
However, peer review is much more than that. It is central to the organisation
of modern science and sits at the base of established work. Peer review is also
the process that controls access to funding.
The advent of the World Wide Web, and the competitive and
financial pressures that are put on scientists and peer reviewed/learned
journals, have made for drastic change. There seems to be nothing that will
slow down the amount of information being produced, much of which is being
posted on the Web. This offers challenges to the learned journals, which have
increased their prices due to demands by publishers. This increase in turn
affects scientists and academic libraries, which are unable to purchase
subscriptions to all the journal titles needed by their users. When mediocre
research is supported, neither science nor innovation is likely to gain much
direct benefit.
A word on excellence. Excellent science papers contribute strongly
to innovation. A recent review that included the biotechnology sector noted
that the most highly cited scientific papers have nine times the prospect of
leading to patents, compared to other papers.
So what does the future hold? How do we secure a reliable source
of the salient information needed for scientific investigation? What about
Public/institutional repositories, mechanisms for review - pay reviewers,
author-pays publishing, or a Web based 'chat room' with membership by initial
peer review.
Presentation: Use of the Web
by Australian Scientists
In the new world of electronic government, the web is rapidly
becoming the prime means of communication between the state and its citizens.
In addition to providing a conduit for the rapid dissemination of information,
an increasing number of transactions are now taking place in this environment,
from submitting tax returns to registering for allowances. We can be sure that
the historians of the future will wish to see how issues such as foot and
mouth, the Iraq war and SARS were presented by the government over the web. The
preservation of government websites is therefore of great importance to the UK
National Archives.
We face four key decisions: which websites to preserve, how
frequently to archive them, how technically to achieve this, and how to present
archived sites to users. The websites in which we are interested fall into two
categories: firstly, there are websites relating to specific events, such as
public inquiries, which are selected in accordance with existing policies -
these have a finite life and only need to be archived at their conclusion, when
we can be sure that no further changes will be made. Secondly, we wish to
preserve regular 'snapshots' of websites across the full domain of government
activities and functions. To this end, we have developed a selection policy,
which identifies 60 core sites covering all the major activities of government
and responsibilities of cabinet ministers. The frequency with which these sites
are archived is flexible - during an event such as the Iraq War, we might wish
to take certain defence and foreign policy-related sites on a weekly basis,
whereas at other times we might be content to take a 6-monthly snapshot.
Technically, TNA is taking three approaches to collecting
websites: capturing directly from source, contracted-out remote harvesting
using the services of the Internet Archive, and in-house remote harvesting via
a common infrastructure developed by the UK Web Archiving Consortium. Each
approach has particular advantages and disadvantages. We are also investigating
various methods for delivering the collected sites to users, and the issues
which this raises.
To date, TNA has archived over 15 million web pages, and is adding
to the collection at a rate of approximately 0.25 million pages per week. This
paper will discuss the issues involved in setting up a web archiving programme,
including selection, collection, quality assurance, delivery, and
preservation.
Presentation: Building the UK
Government Web Archive
Contemporary uses of the Web are multiplying and diversifying
rapidly, and are taking on increasingly complex and interconnected formats.
What once were relatively static and simple homepages, for example, are now
highly responsive database-driven Weblogs which are themselves further
interconnected as social, intellectual, and technological communities and
networks; what used to be Websites intended merely for the display of
information and knowledge are now interactive and editable knowledge bases
using Wiki technology or other content management systems. Information
available in these sites is often changed on a daily or hourly basis, and the
ideas moving through them spread more like information viruses (carried by the
Web as well as by other means of communication from SMS to television) than
along the pathways prescribed by traditional channels of publication. Editorial
practices follow heterarchical rather than hierarchical models, and users are
significantly involved as content contributors. Such features present
significant new challenges for the evaluation and archiving of Web content.
Presentation: Contemporary Culture
and the Web
What do Web archiving and the World Summit on the Information
Society and a national digital strategy have in common?
This paper explores the strategic journey of the National Library
of New Zealand in securing legislation that brings legal deposit into an
electronic domain and the recent budget announcement of $NZ24 million to build
a trusted digital repository to preserve New Zealand's sound, vision and
textual memory, including options for the archiving of the .nz domain.
This paper outlines the challanges, options and learning of the
journey so far and explores the potential for international collaboration in
the future.
Presentation: National Library
of New Zealand Web archiving strategy
Research generates primary outputs (such as data sets) and
secondary outputs (such as scholarly publications). These outputs usually exist
as Web accessible digital files.
Cliff Lynch's seminal paper on institutional repositories has been
one stimulus for a major effort internationally in the higher education sector
to improve the management and preservation of these research outputs. The
success of this effort will depend more than anything else on the policies and
management processes adopted in this sector.
This paper will review the current status of the institutional
repository movement as it relates to the preservation of research outputs. The
review will cover programs such as the Digital Curation Centre in the United
Kingdom, and the newly formed Australian Partnership for Sustainable
Repositories.
This paper will discuss the role of national scientific and
cultural institutions as a safety net in the long-term preservation of this
information.
Presentation: Preserving the
outputs of research
Like nuclear power, digital technology can improve our lives or be
used as a weapon of mass destruction. In the audiovisual archive and museum
world, the boundary between these two options is somewhat confused. We have
perceived an opportunity to make our cultural heritage available to a much
broader audience, but we have no clue on how to monitor - let alone control -
its side effects: the industry is influencing the principles, methods and legal
parameters of management of the collective patrimony; public and private
support to archives and museums has become contingent upon an explicit
endorsement of the Internet ideology. For the first time in history, economy is
actively reshaping the past. Thanks to its impermanence, the Web might become
the audiovisual archivist's best chance to demonstrate it, and it should be
exploited as such.
Four years ago, the National Archives of Australia (NAA) published
policies and guidelines for Australian government agencies on the archiving of
Web-based resources. This guidance was issued as part of the Archives'
e-permanence suite of modern recordkeeping standards and guidelines. The aim of
these policies and guidelines was to improve the quality of recordkeeping by
government agencies in an age when technological change and changing work
practices were making good records an endangered species. This initiative was
consistent with the NAA's objective of ensuring that the right records are
properly kept and maintained by the records' creators. While there are strong
administrative, legal and accountability reasons why it is important to
influence the behaviour of records creators, it is also critical if archivists
are to have any hope of preserving a reliable record of governance in the early
21st century for the benefit of future generations. Since 2000 / 2001, the NAA
has expanded its emphasis from issuing standards and guidelines to records
creators to include working with agencies to implement better practices and to
audit or self-assess progress towards that goal. Having put in place a strategy
for influencing digital records creation, the Archives has more recently given
a high priority to developing a digital preservation strategy for records of
archival value. In September this year, the NAA launched its XENA suite of
open-source XML-based software tools for normalising records created in
proprietary application environments. This presentation describes the various
digital recordkeeping initiatives pursued by the NAA specifically in relation
to how these initiatives relate to the making and keeping of Web-based
records.
Presentation: Managing
government electronic records through the whole life cycle
Since 1994 when early seminal thinking about the preservation of
our digital cultural heritage was codified in Preserving Digital
Information, there has been a call for the establishment of trusted - or
more appropriately, trustworthy digital repositories. Acknowledging that a
network of repositories would be required to preserve the vast and diverse
content, the experts behind the report asserted that such a networked system of
national repositories could only be held together through two essential
mechanisms: a. repositories needed to be able to prove their capabilities
through a process of independently administered certification, and b.
repositories required "critical fail safe" mechanisms allowing them to exercise
an aggressive rescue function to save culturally significant digital
information.
Ten years later, many institutions - including national libraries
and archives - have embarked upon the process of building digital repositories
for the long-term preservation of our digital cultural memory. In doing so,
these institutions have joined - intentionally or unintentionally - the greater
international, networked effort envisioned back in 1994. Though these
activities are positive steps toward the preservation of culturally significant
digital information, we still lack the critical mechanisms identified by
Preserving Digital Information. With this in mind, do we have an
international system of digital repositories in place which is trustworthy? Can
we rely upon the emerging digital repositories to preserve the information
correctly? How can we trust that other repositories are capable of filling the
roles and responsibilities relegated to them as a part of the larger effort?
Shouldn't trustworthiness require not only the ability to trust partner and
peer repositories, but also verification of that trust? The big questions
related to digital repository trustworthiness could be addressed by an
independently administered certification program. But since such a program is
not yet available, can we as a community afford to wait for such arrangements?
Should we instead consider different organisational models for digital
preservation and instead build centralized repositories or can we rely upon the
vision of the distributed networks of repositories?
This presentation will address these difficult questions and will
discuss the different roles some institutions have taken to address the issue
of trustworthiness as the grapple with establishing workable, sustainable
archiving and preservation arrangements. It will also draw on the ongoing work
to establish a certification program for digital repositories charged with
preserving our digital cultural memory.
Presentation: Establishing
trustworthy repositories: organizational models, roles and
imperatives
In planning for the archiving of Web resources, the host
repository will have to address a number of key issues relating to the
provision of metadata to support resource discovery. Assessments will have to
be made as to the need for supplementing data that can be extracted
automatically from the resources themselves and the tradeoffs that might be
made between enriching the metadata and enhancing the retrieval software. The
distinctive characteristics of Web resources and the relationship between the
"live" resource and the archived resource will add another level of complexity
when evaluating metadata schema and "content rules" for description. Cost will
be a major factor with respect both to metadata creation and to software
development, and the cost factor in turn will require assessing the feasibility
of pooling resources with other repositories in a cooperative endeavour.
This presentation examines metadata for archived resources in the
context of the network technologies that support resource discovery, and
provides a framework to assist repositories in addressing their resource
discovery requirements both individually and collectively. It focuses on the
nature of the archived resource at one end and the expectations of the user at
the other end, as well as on the interfaces between the resource, the metadata,
and the user.
Presentation: Resource Discovery
and Archived Resources: Approaches and Issues
The U.S. Library of Congress is responsible for a national program
to preserve significant digital content that is at risk of loss. Through
NDIIPP, the Library is building a network of partners consisting of federal
agencies, state governments, universities, private companies and cultural
heritage organisations. Parallel efforts are also underway to develop tools,
models, methods and technology to support digital preservation.
NDIIPP has three focus areas for its implementation: 1. network of
preservation partners; 2. preservation and architecture; and 3. digital
preservation research agenda. The network of parteners is the program's
cornerstone. The volume, diversity and complexity of digital content compels a
distributed, collaborative approach that enables individual organisations to
manage content while also sharing best practices for preservation. Partnership
network operations are guided by the preservation architecture, which is a
conceptual framework for deploying infrastructure components. The research
agenda supports exploration of the advanced technology and methodology needed
to sustain digital content over time.
The Library has made important recent progress in all three areas.
A highlight is the launch of Partnership Arrangement 1, where the Library has
committed $US 15 million to eight consortia consisting of 25 partners to select
and preserve a diverse range of content. Among the types of content the
partners will work with are web sites, geospatial data, social science data
sets, business records and TV broadcasts.
NDIIPP is relevant to other countries for two basic reasons.
First, it provides a model for a national approach to digital preservation with
one major cultural heritage institution serving as a facilitator to engage
other stakeholders and help them work together on the basis of common
interests. Second, NDIIPP is itself actively engaged on the international
front. One example is the Library's membership in the International Internet
Preservation Consortium and its commitment to help develop the methods that all
nations need for effective web archiving.
Presentation: The National
Digital Information Infrastructure and Preservation Program (NDIIPP): Future
Directions and Relevance to Other Countries
The Web impacts on contemporary society as the printing press did
upon the Renaissance. Today's "data deluge" is producing massive amounts of
information, yet our strategies for accessing, analysing and preserving
"virtual spaces" over time are still in an embryonic stage. The curatorship of
born-digital materials requires greater global coordination and collaboration
than was needed in past ages for the acquisition and storage of passive print
forms of knowledge. The digital environment provides a hugely expanded range of
creative challenges and opportunities: in film, music, sound, the visual arts
and multimedia. Human and social outputs require preservation for posterity
through new, systematic approaches to digital archiving. Will "born digital"
also mean that collecting institutions will themselves be transformed?
Professor Gillies' paper examines the cultural frameworks for the collecting of
digital artefacts and the social, economic and political actions necessary for
todays digital memories to be available to tomorrows citizens.
Paper: Born Digital Born
Free? The Cultural Impact of the Web
Museums in Australia have been using the web since 1994. The first
museum website in Australia was developed at the Western Australian Maritime
Museum; its still accessible today at http://web.archive.org. From these
first tentative steps there are now over 510 Australian Museum websites listed
on Australian Museums Online (AMOL). Between 1995 2000 there was an
explosion of creativity and exploration as museums experimented with how to
utilise the web based medium of information dissemination and collaboration
following the dotcom collapse things have steadied technologies
have matured and websites have become part of the core business of all major
museums across the world.
This paper will examine the evolution of museum websites
concentrating on Australia with some important international examples. Museum
websites provide in-depth information, collection based stories, research,
reference information and educational resources, I will illustrate the rich
diversity of content and information architecture approaches taken by museums
in Australia in 2004. The web has made many things possible that we could
simply not do before, the number and complexity of national and international
collaborations using collection data for example, Australian Museums Online
(AMOL), Picture Australia, Ozcam, Australias Virtual Herbarium (AVH) to
name a few, projects which have all played a part in developing and making
accessible the distributed national collection.
In future museum websites will continue to evolve into richer and
more compelling visitor experiences aligned and complementary with the
actual experience visitors have within the physical museum a recent
article in Museums Journal reported that a recent survey has placed museums
second to libraries as the two most trustworthy sources of information in our
society. This article reminded me of Lynn Elliott Sherwoods words, who as
the head of the Canadian Heritage Information Network (CHIN) in 1997 said that
museums are islands of trust on the internet I believe that
we have to date remained faithful to that observation and have managed to
maintain our reputations as scholarly institutions in the face of the
edutainment onslaught of which the web is a major part.
The preservation of our websites has not been addressed in any
meaningful way to date! - Certainly an issue for organisations based on
collections and collecting!
It is widely recognized that there is no single best solution to
digital preservation. The most appropriate strategy depends on the particular
requirements of the custodial organization, the producers and users of its
collection and the nature of the objects in the collection. In this paper we
describe a scalable, flexible, semi-automated approach to preservation which
leverages existing work on preservation metadata, registries, tools and
services by integrating them and making them available through a Semantic Web
Services architecture.
The system which we have developed provides metadata extraction
and capture tools which enable the recording of preservation metadata for
digital objects in a collection. The collections' preservation metadata is
periodically compared with software and format registries (e.g, GDFR). When an
incompatibility is detected, a notification is sent to the relevant agent
(human or software). If preservation action is requested, then the system
dynamically discovers or composes the most appropriate preservation service(s).
This is implemented by making preservation software modules available as Web
services and describing them semantically using a machine-processable ontology
(OWL-S). Collection managers then have the option to choose and invoke a
particular service from the list of preservation services retrieved and
recommended by the software agents. Consequent preservation events are
recorded.
We believe that the semi-automated, distributed, Semantic Web
Services approach which we have developed within PANIC represents a significant
advance towards a viable, cost-effective solution to the long term preservation
of large scale collections of digital objects.
Presentation: The PANIC Project -
A Scalable, Collaborative Approach to Preservation Based on Semantic Web
Services
Together with IBM the Koninklijke Bibliotheek, National Library of
the Netherlands, has developed the e-Depot. The e-Depot is a system that
enables us to safely store and give access to the electronic output from two
types of publishers. The first group comprises Dutch publishers that for the
greatest part are represented by the Dutch publishers association. For this
group a general agreement is in place that is an extension to the one for
printed publications. The second group consists of the publishers 'without
boundaries', the international STM-publishers. The KB has signed individual
archiving agreements with these publishers to deposit their electronic journals
at the KB. The presentation will focus on three topics that play an important
role within our e-Depot environment: what are these agreements with publishers,
what are the requirements for permanent archives and what kind of strategies do
we see for permanent access?
Paper: Permanent access: the
e-Depot at the Koninklijke Bibliotheek
The Internet Archive (IA) is a 5013C non-profit corporation, whose
mission is to build a public Internet digital library. Over the last 6 years,
IA has built the largest public web archive to date, hosting over 400 TB of
data. The web collection is comprised of billions of web pages from over 35
million sites around the world.
Attempting to archive the publicly available web appears to be
such an enormous task that many believe it is not possible and some question
the value. However, an equally challenging task is deciding what to selectively
archive for future generations today. Due to the everchanging content on the
web, and its seamless borders, one can argue the cost of selection is greater
and more risky than the cost of trying to save it all.
Internet Archive will present its approach in archiving the global
web. The techniques used to collect, store and provide access to the data, the
challenges faced in collecting and preserving billions of web pages, and the
benefits and compromises made in large scale web archiving will be
discussed.
Presentation: Saving the Web
for future generations
Preservation metadata is information needed to carry out,
document, and otherwise support repository functions and policies directed at
securing the long-term retention of digital materials. Regardless of the
specific technique chosen to preserve digital materials, metadata will nearly
always play a critical role. Most current digital preservation initiatives have
implemented preservation metadata schemas in one form or another, and there has
also been collaborative activity within the international digital archiving
community to develop standardized preservation metadata applicable across a
broad range of digital preservation activities. The outcome of this work
suggests that it is indeed possible to define a "core" set of preservation
metadata elements, but cross-institutional differences in repository
architectures, technologies and preservation policies necessarily limit the
scope of this core set. Nevertheless, there are potential opportunities to
cooperatively create and share preservation metadata and the international
digital archiving community would benefit from further exploration of this
issue. One concern about preservation metadata is the effort and expense
required to create and maintain it. Preservation metadata that is shareable and
reusable would help mitigate this problem, but beyond this, automatic tools for
managing preservation metadata are needed as well. Some progress has been made
on this front, but much work remains to be done. Developing such tools would be
greatly assisted by a shared view in the digital archiving community on what
metadata is needed to support digital preservation.
Presentation: Preservation
metadata initiatives: status and direction
In China, the Internet plays a peculiar role: the government of
the Peoples' Republic of China views it as absolutely necessary for the
development of the country, but at the same time it is also afraid of its
incalculable threat to their grip on power. This results in a fascinating
quality of the Chinese Internet: its infrastructure is very well developed and
used as a major channel for public discourse by subcultures, dissidents and
party conformists alike, while at the same time it is also closely monitored
and controlled by the government. Content perceived as being too critical or
"dangerous" for the public is quickly removed and its publishers not seldomly
punished.
Obviously we cannot expect the Chinese government to archive and
publicise these data, but unquestionably they are an essential part of the
Chinese cultural heritage and form an important resource for research. In order
to make at least parts of these resources available for the future the
Sinological Institutes of Heidelberg University (Germany) and Leiden University
(Netherlands) have begun to archive parts of the Chinese Internet that reflect
social and political discourse in or about China.
Based on the fact that the project is run by two rather small
institutions this presentation focuses on an issue that is central to is
success: how to develop strategies for selection of relevant resources and how
to provide a meaningful environment to make their relevance comprehensible even
in the future.
Presentation: Informant
networks, alarm systems and research contributors: selection and ingest process
for the Digital Archivie for Chinese Studies
Die Deutsche Bibliothek is the national library of Germany, a
country with federal structure: 16 individual German states are responsible for
their own cultural affairs. Germany is also part of the European Union, which
has enlarged to 25 member states since May 2004. Member states of the EU are
urged to pay particular attention to the need to sustain and promote cultural
diversity. Cooperation, division of labour and virtual integration of
autonomous efforts are key elements to keep cultural diversity possible and
alive in a world of reduced financial budgets and limited funding. The EU
framework program and project TEL - The European Library as an example for a
pan-European virtual information service will be outlined. German activities
and projects concerning collection and preservation of digital resources will
be highlighted.
Presentation: Die Deutsche
Bibliothek: cooperative efforts on the European and national
level
Virtual Remote Control (VRC) is Cornell's risk management approach
for Web resources. Virtual because the approach develops abstract models to
represent essential features of selected Web sites that enable ongoing
monitoring of these sites over time to identify, respond to, and mitigate
potential risks to their integrity and longevity. Remote because the approach
is intended for use by cultural heritage organizations interested in the
longevity of Web resources residing on remote servers, specifically sites
administered by another organization that pertain to or extend the monitoring
organization's collecting areas. Control because at the most proactive end of
the VRC approach, a monitoring organization exerts control over another
organization's resources by agreement or implicit consent through notification
and action. The VRC approach includes but does not presume the capture of Web
sites because a monitoring organization may not have the means, authority or
desire to capture all or some iterations of a Web site. Ongoing monitoring and
evaluation allows monitoring organizations to intelligently manage Web
resources over time. In conducting our research, we have learned a lot about
good Web site management and about promulgating good practice to encourage Web
longevity. We have developed a six-stage VRC model, a Web tool evaluation
method and inventory, and a control site for evaluating Web tools.
Presentation: Virtual Remote
Control: A rish management approach
Topographic and other thematic maps have been archived by
libraries as a permanent historical record in the same way as they archive
books and other media. Despite the move to digital cartography in the 1980's,
until recently the primary method of map distribution remained as mass produced
hard copy maps.
The advent of database technology and the Internet have led to a
shift away from mass production of maps. Instead of the data for a geographic
area being updated once every twenty years, the data are now continuously
updated. Changes to the data (including previous versions) are recorded in a
relational database. Clients commonly use the Internet to access the data as
unique digital data files or "print on demand" maps that are derived from the
latest version of the data, as recorded in the database.
These new products represent a paradigm shift for libraries and
archivists, as a centralised archive of products is no longer practical.
Geoscience Australia is addressing this issue through the Seamless Topographic
Database Project. The project offers a potential model for the ongoing archival
of map data. The system architecture makes it possible for clients to request
data as it was at virtually any time since the project was initiated.
This model offers benefits over traditional archival practices, as
the custodian is able to manage the entire "information life cycle", from data
creation, through analysis, distribution and archival.
Presentation: Maps: the impact of
the Internet
The Royal Library has collected the Swedish web comprehensively
since early 1997. Without support by deposit law the library has sent robots to
search for and download all Swedish pages about twice a year. It has thus
created an ephemera collection of online publications parallel to the
traditional ephemera collection of Swedish print, which is searchable but not
catalogued. The printed collection totals over 10 million items, while the
digital in only seven years has reached 260 million files fetched from 280
thousand web sites. The digital collection has a tremendous research value
reflecting the early growth of the web.
But the comprehensive, automatic approach has its limitations. The
collection is not complete. A recent government report suggests, supported by
the Royal Library, that deposit plight should comprise all online publications,
also protected ones. The robot collection of today should be supplemented with
complete collection of material spread over the Internet by Swedish
professional publishers and producers. Many of these documents should also be
catalogued.
Presentation: Collect all,
catalogue some
The development of tools for web archiving is one of the major
goals of the International Internet Preservation Consortium (IIPC).
Acknowledging the lack of appropriate and scalable tools in this domain (even
if much can be borrowed from traditional IR and search engine technology), IIPC
members have addressed this problem by launching several software development
projects.
Using common tools, when possible, is also key for facilitating
future exchange, cross access and sharing of collection. In so doing, IIPC
members intend to help laying the ground for collaboration, a necessity when
considering the size and border less nature of the Web.
IIPC's achievements in this domain (Heritrix, DeepArc, eXplore) as
well as its strategy for the future will be presented.
Presentation: International
Internet Preservation Consortium (IIPC) work program to develop Web collecting
tools
The organizations that will collect the materials of the web are
only just beginning to take shape. There will be a multiplicity of these
agencies, from across all the cultural sectors, that ultimately emerge as a
system of collection, access and preservation. For this emergent system to
satisfy all the audiences as well as the diverse demands for this archived
material a common set of service expectations must be established. Those
expectations can only be fulfilled if the guiding policies and the supporting
infrastructures of these agencies acknowledge their role in a broader system.
This talk will capture some of the issues that must be addressed, reflect on
the current state of progress and consider the fiscal dimensions of the
challenge.
Presentation: Key issues for
cultural agencies
In building PANDORA, Australia's Web Archive, the National Library
of Australia and its partners have taken a selective approach to the archiving
of Australian online publications and web sites, because it enables us to meet
our primary collecting objectives. This paper analyses the advantages and
disadvantages of the selective approach, outlines the selection priorities,
describes some of the characteristics of the PANDORA Archive, and refers to two
research projects underway that are expected to increase the volume of titles
and to extend the range of dynamic formats that can be archived.
Presentation: What to
collect and how to do it: the National Library of Australia's selective
approach
Today, on-line, we are surrounded by many forms of Internet art.
From streaming video, elegant interactive Java works, electronic poetry,
net.radio and tactical media, to the phenomenally successful genre of intimate
video or panoramic blogs. We may interact within artist-created multi-user
Virtual Reality and Game spaces, and watch new theatrical and cinematic forms
like avatar performance and machinima.
Flash animation has penetrated almost every corner of the Web; and
the network itself becomes an artwork as software art and browser interventions
reformat and recontextualise HTML code. Net.art creates a mixed reality when
data captured from one space is transposed and revisualised in another
location; or when the monitored interactions of both artists and users becomes
the bio-input for an artwork.
While these works provide entertaining cultural experiences, they
also raise many challenges for preservation as they often utilize specific
software and hardware, and/or the traffic, rhythm and compressions of the
network itself as content. Can these works be successfully archived or migrated
to retain their aesthetic and artistic intent? Or can only a slice be
documented? These are critical questions as pioneering net.art works, from less
than a decade ago, already have compromised functionality, and may soon be
lost.
Paper: The Art of
the Network
The World Wide Web- massive in scale, virtually barrier-free for
publishers, and highly dynamic- has considerably complicated the preservation
function of libraries and archives. Beyond the technical challenges of
capturing and stabilizing data, the Web has made traditional and deliberative
processes for selecting and preserving information resources obsolete. Abby
Smith will talk about the challanges of selection in this new environment: who
should be responsible for institutional selection policies; what criteria they
should consider in making those decisions; and how will libraries and archives
redefine themselves in light of this data deluge. She will review a number of
approaches currently underway.
Paper: The future of Web
resources: who decides what gets saved and how do they do
it?
INFORM (INvestigation of FOrmats based on Risk Management) is a
methodology for investigating and measuring the risk factors of digital formats
and providing guidelines for preservation action plans. In other words, this
methodology attempts to discover specific threats to preservation and measure
their possible impact on preservation decisions. Moreover, by repeating the
process, involved parties can detect changes in the threat model over time, to
which digital archives can act accordingly.
Digital archives, institutional repositories and digital libraries
can take advantage of the measurements offered by the INFORM method to select
digital formats most apt for long term viability. While individuals are biased
and subjective in their aversion to risk, collating the assessments of many
individuals should generate group-consensus or group-averaged objective
results. Hence, preservation plans can be based on objective analysis of risk
trends instead of individuals' opinion developed in the relative isolation of
their institutions.
Presentation: Assessing the
Durability of Formats in a Digital Preservation Environment: the INFORM
Methodology
The Legal Deposit Libraries Act 2003 extended the principle of
legal deposit in the United Kingdom to the non-print world. Within the context
of the new legislation and its progress, the presentation seeks to outline the
challenges faced and progress made by the British Library in developing a
strategy and operations for archiving the UK web presence. Reference will be
made to the pilot Domain.uk project of 2001 upon which is built the current Web
Archiving Programme. This includes membership of the UK Web Archiving
Consortium and the International Internet Preservation Consortium. Challenges
of the collection development policy, staff resources and preservation through
the Library's Digital Object Management programme will be covered.
Paper: The British Library: Web
Archiving Challenges and Initiatives
Archiving the Web is both a necessary and challenging task that no
institution can achieve alone. The Web is not only immense, it is also rapidly
and unforseeably changing. It represents a highly distributed and borderless
mass of documents which real historical signification can only be reflected at
this scale. The necessity of cooperation among heritage institutions is thus
manifest when considering the nature of this new media.
The International Internet Preservation Consortium was formed to
help in making effectual steps in this direction, bearing in mind the
importance of gaining practical experience and building common expertise to be
effective. This new domain of Web archiving indeed raises new issues not only
at the technical level but also on selection, organisation and processing of
collections. Through the development of common methods, tools and protocols,
the IIPC is trying to help build future interoperability of heritage
collections as well as best possible coverage for this media of historical
preservation.
The IIPC has divided its activity into Working Groups to share
experience and prepare reports and protocols and projects to share the
development of tools. The six Working Groups are working to define a common
technical framework, assessing the researchers of internet needs, defining
applicable metrics and test beds, access tool requirements, methods for
archiving the 'deep' or 'hidden' Web, and collection management. A rapid
overview of the first results and workplan of these Working Groups will be
presented.
The main tools developed at this stage are the Heritrix archiving
crawler developed by Internet Archive and the Nordic Libraries, Deep Arc and
eXplore for archiving databases of deep Web sites developed by bnF and the NLA.
The are or will be soon available freely and in open source.
Presentation: International
Internet Preservation Consortium (IIPC): objectives and
workplan
Library and Archives Canada is an innovative knowledge institution
that combines the collections, services and staff expertise of the former
National Library of Canada and National Archives of Canada. Our objective is to
provide all Canadians with easy, one-stop access to the texts, records,
photographs and other multi-media documents that reflect their cultural, social
and political development. Working closely with other archives and libraries,
we will continue to acquire and preserve Canada's documentary heritage in all
its forms. Our mandate is: to preserve the documentary heritage of Canada for
the benefit of present and future generations; to be a source of enduring
knowledge accessible to all, contributing to the cultural, social and economic
advancement of Canada; to facilitate in Canada cooperation among communities
involved in the acquisition, preservation and diffusion of knowledge; and to
serve as the continuing memory of the government of Canada and its
institutions.
Catalytic initiatives were established earlier this year to act as
agents of change within LAC. They are "catalytic" because they are focused on
challenging areas in which LAC must explore, invest and innovate if it is to
realize its vision of the future. The proposed scope and activities of the
catalytic initiatives will involve many areas within the new organization and
will have a broad impact. They will explore issues and bring about change in
key work processes through research and demonstration projects, studies, policy
work, and implementation projects. Two key catalytic initiatives that Ian
Wilson will describe today include the digital collection initiative and the
metadata initiative. The digital collection initiative will focus on the
challenges of acquiring, describing, managing and preserving access to digital
content, including web resources. The metadata initiative will develop a
metadata framework to facilitate delivery of content and services to users.
Presentation: Library and Archives
Canada: Digital and Metadata Initiatives
The National Library has reproduced this
material on the conference web site with permission of the authors. Anyone
wishing to reproduce or publish the material should contact the relevant author
directly.
|