Web archiving
1. Purpose of page
The emergence of the World Wide Web as a global information resource
has created special challenges for the archiving of digital materials. The
Web's ever-expanding size, the dynamic and ephemeral nature of its content, and
how this is to be captured, stored and made accessible for the long-term are
some of the key questions being addressed by electronic archiving programs.
This page serves as an introduction to some of the major archiving
initiatives that have been established by national libraries and other national and international organisations around the world
to preserve their country's Web heritage. The main models that have emerged for
archiving Web content are listed, together with descriptive 'snapshots' of the
key projects and collaborative initiatives. Links are also provided to
PADI resources which provide further
discussion of Web archiving issues. Information about related topics such as
legal deposit,
selection and
preservation can be
found on the relevant PADI Topic page, or by
searching the PADI
database.
The following countries have well-established national Web archiving
programs in place or are actively pursuing relevant issues:
2. Web archiving models
In many countries, national libraries have taken the leading role in
researching and experimenting with Web archiving processes, in some cases
supported by a legal deposit mandate for collecting electronic resources. In
addition, a number of international partnerships have also been successful in
exploring and testing digital archiving theory and practice. Several models
have developed for content collection which can be broadly categorised as
follows:
Whole domain The whole domain (or comprehensive)
approach involves the collection of Web sites and online resources using
harvesters to automatically retrieve material in broad sweeps of national Web
space, such as that undertaken by the Swedish
Kulturarw3 (Cultural Heritage
Cubed) and the Finnish
EVA project. The
US-based Internet Archive is the most
ambitious example of this model, an initiative which is seeking to preserve the
entire content of the global Web.
Selective Selective approaches are aimed at
archiving defined portions of Web space or particular kinds of resources
according to specified criteria. Selection may be based on the significance or
quality of resources, their theme or topic, or by targeting a related set of
Web sites. Rather than attempting to archive all content, the selective
harvesting model, such as that initially adopted by Australia's
PANDORA archive, entails the
collection of specified site or document 'snapshots' at scheduled intervals.
Thematic Another form of selective archiving
involves the collection and preservation of Web content relating to a
particular theme or event. The Library of Congress's
Minerva project has used a thematic
approach in selecting electronic publications to archive as part of its
Election 2002 and Winter Olympics projects.
Deposit In some countries publishers deposit
online material based on legal or voluntary deposit codes. At this stage the
deposit of Web resources by content owners is not well-established in any
country, though several experimental systems are underway. A successful
voluntary deposit scheme for journals in electronic form operates in
The Netherlands through agreements concluded with
publishers, whilst in Sweden the deposit of static and
dynamic Internet resources is a legislative requirement.
Combined approaches A growing number of Web
archiving programs are concluding that no one archiving model is entirely
satisfactory for preserving national online heritage. Instead, countries such
as France and Denmark have found
that by employing a combination of comprehensive, selective and thematic
collection techniques, the optimum coverage of material is achieved.
Each of the foregoing archiving models have benefits and difficulties
according to the particular Web context to which they are applied. Other
strategies are also emerging for archiving Web content, including the 'by
discipline' approach of some universities and research institutes, such as the
DACHS (Digital Archives
for Chinese Studies) project based at the University of Heidelberg. In an
innovative approach, the Virtual Remote Control
(VRC) program at Cornell University is monitoring changes to Web sites over
time, and capturing those at risk of the loss of information as a last option.
A useful discussion on the relevant advantages and disadvantages of the
various Web archiving strategies can be found in the study
Collecting
and Preserving the World Wide Web : a Feasibility Study Undertaken for the JISC
and Wellcome Trust. Further introductions to Web archiving theory and
practice are listed under Web archiving
resources - Overviews.
Some Web archiving case studies have been collected on this site:
Australia | Austria |
Canada | Czech Republic |
Denmark | Finland |
France | Germany |
Japan | Lithuania |
The Netherlands | New Zealand
| Norway | | Portugal | Sweden |
United Kingdom | U.S.A |
Collaborative initiatives
3. Case studies of archiving approaches
Australia
The National Library of Australia has been involved in digital archiving
since 1996, being one of the first national libraries to undertake
investigations in this area. The major focus of its program has been the
creation of the PANDORA
archive, which is an archive of selected Australian online publications deemed
to be of national significance and long term research value. The system
operates on a model of selecting, collecting and storing Web resources from the
Australian domain, in cooperation with one territory and five state libraries,
the Australian War Memorial, AIATSIS (the Australian Institute of Aboriginal
and Torres Strait Islander Studies) and the Nationa Film & Sound Archive. To support the
system the NLA has also developed the PANDAS (PANDORA Digital Archiving System)
software to assist in the automatic collection, description and provision of
access to the archive.
In Australia, legal deposit legislation does not yet cover electronic
publications, though a voluntary deposit scheme is in place with a number of
publishers. Working within the Library's resource constraints, emphasis has
been placed on creating a representative, rather than comprehensive archive,
and guidelines have been created for the selection of online materials for
preservation. Within the NLA, members of the Library's Web Archiving Section
regularly scan the Internet for new publications that comply with selection
guidelines, and harvesting software is employed for capture of resources.
Selection is also undertaken by participating institutions.
Fundamental to PANDORA's philosophy has been the provision of access to
archived content which is controlled in accordance with rights permission
negotiated with each publisher. A proportion of titles are regularly
re-harvested to preserve changes to content and some minor migration
experiments have been made. The underlying strategy has been to store material
in original formats, whilst maintaining access to another copy through
available technologies.
Mindful that the selective approach inevitably misses important resources and takes them out of context of the web pages to which they are linked, since 2005 the Library has also undertaken annual large scale .au domain harvests. The Library is investigating the legal and technical obstacles to providing access to the content of whole domain harvests.
Key documents describing aspects of the Library's archiving program in
more detail are available from its
About PANDORA
page. Links to more specific information about Australian Web archiving
initiatives are also listed under Web archiving resources - Australia.
Updated September 2009
Austria
A July 2000 amendment to Austrian media law extended legal deposit
requirements to digital offline material. Networked digital resources were not
included in the provisions and in response to the legislative gap, the
AOLA (the Austrian Online
Archive) project was established in 1999 to archive the Austrian Web space. A joint initiative of the Austrian National Library and the Department of
Software Technology at the Vienna University of Technology, harvester software
is used to collect the .at domain and other sites of Austrian interest through
periodic snapshots of publicly available Web sites.
An initial AOLA pilot study was followed by the official first phase of
the project in 2001. A partially successful attempt at gathering a snapshot of
the Austrian Web space was made with the NEDLIB harvester in May of that year,
but this was later aborted due to crawler instability. In the following month,
a second effort using an adapted version of the Kulturarw3 Project's Combine
harvester was successful, as was a further crawl in 2002. A Data Warehouse was
established to analyse the characteristics of the harvested data.
At present the project is discontinued due to a lack of funding. A
resumption of captures is envisaged for 2005. A combined selective and a free
harvesting approach to collection is seen as the most effective strategy for
the future. The archive is not yet publicly accessible, and important areas for
further work include the selection of appropriate storage systems, the
development of a suitable legal framework, and the choice of optimum methods
for ensuring future access and preservation. The migration techniques employed
for preserving resources at present are seen as a short-range solution, to
which emulation methods will be added in the long-term.
Links to more specific information about Austrian Web archiving
initiatives are listed under Web archiving resources - Austria.
Updated August 2004
Canada
In 1994-95, the National Library of Canada conducted an
Electronic
Publications Pilot Project (EPPP) to examine the issue of the deposit of
online electronic publications. The final project report made recommendations
on important issues such as selection, copyright, and long-term preservation of
e-publications. As an outcome of the EPPP, Libraries and Archives Canada (as
the NLC is now known following amalgamation with the National Archives in 2004) has continued to collect online books and periodicals on
a voluntary basis through its
Electronic
Collection Web site. The E-Collection, as the archive is popularly known,
contains publications from commercial publishers, non-trade publishers,
individuals, and federal and provincial governments. At present, unrestricted
access is provided to approximately 80 percent of the archive through the LAC
Web site, the residual content remaining restricted due to publishers' concerns
regarding unfettered public access to their publications in the E-Collection.
In the absence of specific legal deposit legislation at the time, a selective and
distributed model for collecting networked online Canadiana was adopted by
the LAC, as part of the broader objective to collect and archive all online
resources originating in Canada or Canadiana published in other domains.
Detailed work was undertaken on defining collecting scope, and formal
collection guidelines were issued in 1998. As part of the selection process
resources are manually assessed and a collecting status is assigned to each one
according to a three-tier scale. 'Archived' resources are stored on the LAC's
Web site and carry a permanent commitment on the part of the LAC to preserve
them. 'Served' resources are hosted by the LAC but are not endorsed with the
promise of preservation, responsibility for which may be taken up by other host
organisations. Those non-hosted resources for which the LAC has made an ongoing
commitment to preserve a link to are assigned a 'Linked' status.
The Library and Archives Canada Act received Royal assent on 22 April 2004. This allowed LAC to collect a representative sample of Canadian websites for preservation purposes. In 2005 LAC began experimenting with harvesting the Government of Canada web domain. On 1 January 2007 the Legal Deposit of Publications Regulations extended Legal Deposit to online publications. The Government of Canada Web Archive was opened to the public in November 2007.
Links to more specific information about Canadian Web archiving
initiatives are listed under Web archiving resources - Canada.
Updated May 2004
Czech Republic
The archive of the Czech online-born documents (known as WebArchiv) was originally launched within the R & D project funded by the Ministry of Culture in 2000 and since then it has been implemented in the National Library of the Czech Republic and funded almost exclusively through various grants. There are two other institutions involved in the project: Moravian Library, which is responsible for IT issues, and Institute of Computer Science of Masaryk University as external co-operator.
The aim of the WebArchiv project is to protect the Czech web cultural heritage. There is implemented a comprehensive solution in the field of archiving of the national web, i.e. bohemical online-born documents, including the Czech National Bibliography. Tools and methods for collecting, archiving and preserving web resources as well as providing long-term access to them are applied. Both large-scale automated harvesting of the entire national web and selective archiving are being carried out, including thematic collections on a given subject. At present these methods are tested and are a subject of further research. To run all operations in a routine way, two conditions must be met: sufficient long-term funding and legal background, primarily the legal deposit legislation.
Concerning legal issues, the current state of legislation in this field is not convenient. Legal Deposit Act doesn‘t cover online-born documents, new applicable legislation is being prepared. Nevertheless, since June 2006 the Copyright Law updated in harmony with the Directive 2001/29/EC will come into force; this means that it will be possible to open access to the whole archive from the library premises for the purpose of research or private study. Besides, agreements with publishers are made for some important resources applying selection criteria, which authorize the library to make these archived resources online accessible.
Collecting online documents is an automated process carried out by a set of software tools that harvest, index and save data in the archive according to preassigned parameters. Nowadays, most of the software tools used are open source products developed by the International Internet Preservation Consortium (IIPC) with the rest developed locally. So Heritrix tool is being used for web crawling. Harvested files including relevant metadata are saved in standardized archival formats supported by the IIPC consortium. Next to the server, the data is stored on the redundant disk array (RAID) with expected transfer to National Library’s new large data storage facility in the near future. At present (May 2006) data of approximate size of 2 TB (i.e. 26 million unique files) are stored in the archive. Another one server is used to give online access to a limited dataset whose content is covered by agreements with its original publishers. Fulltext indexing is performed by open-source system Nutch and is accessed through Nutchwax and WERA tools. The recent changes in Czech copyright law will demand for a fulltext index of the whole archive and require a new server dedicated to fulltext indexing.
International standards are being used for description and identification (MARC21, Dublin Core, ISSN and URN) and archiving (ARC). Selected online-born documents are catalogued in an ALEPH library system which supports Z39.50 (both client and server levels) and OAI-PMH protocols (both repository and harvesting levels with the profile for MARC21 and qualified Dublin Core); records are thus also registered in the Czech National Bibliography.
Links to more specific information about Czech Web archiving initiatives
are listed under Web archiving resources - Czech Republic.
Updated May 2006
Denmark
Danish legal deposit legislation enacted in 1997 extended deposit
requirements to physical format digital publications and static online
documents and was openly-worded to apply to other future electronic formats.
Based on this mandate the Royal Library (Kongelige Bibliotek) has been
selectively collecting Web resources since 1998, with physical format
electronic products, static online publications and e-journals being downloaded
to the online Pligtaflevering
registration system upon notification from publishers. Under Danish law
publishers must supply details of electronic products being submitted to the
system, from which the Royal Library extracts Internet addresses for the
materials for downloading. Royal Library staff than select, check, catalogue
and transfer the collected materials to an archival server.
In a parallel initiative in 2001, the Danish
netarchive.dk was
established by the Royal Library in collaboration with the State Library (Aarhus) and Center for Internet Research, University of Aarhus, as a testbed for various archiving strategies, and to
assess the usefulness of the collected material for researchers. The collection
of Internet coverage of the 2001 Danish municipal elections was used as a case
study for the first phase of the project, which ran to July 2002. Legal
impediments prevented direct capture of content from the Danish Web, and access
and harvesting frequency agreements were instead negotiated with owners, whose
sites were collected using appropriate software.
Work on the second phase of the project has continued with the aim of
refining the processes identified in the pilot and making recommendations for
requisite amendments to Danish legal deposit regulations. Once specific legal
deposit legislation has been enacted for networked material, a combined
selective and harvesting approach to collecting has been recommended as a
future path. This would involve the collection of material via three main
channels: an annual harvest of Internet coverage of one major event or theme
(an 'event-based' harvest), the ongoing selective harvesting of sites of key
interest, and the automatic collection of quarterly snapshots of the Danish
Web. It is envisaged that the State and University Library would take
responsibility for the first two methods, with the Royal Library administering
the latter. Present interest is focused on the release of an official response
to the recommendations of a netarchive.dk report, which was presented to the
Danish Ministry of Culture in April 2004.
The Royal Library has also been active in the
Nordic Web Archive project.
Links to more specific information about Danish Web archiving
initiatives are listed under Web archiving resources - Denmark.
Updated May 2004
Finland
In Finland, the EVA archival system to
collect, register and store Internet publications and ensure their long-term
preservation, ran as a joint project of libraries, publishers and expert
organisations within the country, initiated in 1997. The program was led by the
Helsinki University Library (National Library of Finland) in association with
three Finnish universities, the Edita publishing house and CSC Scientific
Computing, with the objective of collecting publicly available static Web pages
and other inline material in the Finnish domain. Specific aims of the project
have included the formulation of criteria for electronic document selection,
the creation of a direct deposit system for publishers, and the development of
a reliable long-term archive.
Building on the work of two earlier stages, the third phase of the EVA
project was completed in 2001, and the resultant archival system is based on a
whole domain approach to collection. Following a successful first sweep of the
Finnish Web in June 2002, snapshots of the .fi Web space are now gathered
periodically. In addition, partial harvests are carried out in connection with
special events such as national elections. Cooperation has been obtained in
capturing the national domain from the Finnish portal Info Center Finland,
which forwards Finnish server locations in other domains than .fi for loading
into the project harvester.
Finland has also been an active partner in the
Nordic Web Archive and
NEDLIB projects, with programming
of the NEDLIB harvester prepared by the CSC to project specifications. Further
development of the NEDLIB system was jointly undertaken by the CSC and the
Helsinki University Library (HUL). Up to Spring 2004 the NEDLIB harvester was
used, but it will be replaced by the Heritrix harvester, developed jointly by a
consortium consisting of the Internet Archive and a number of national
libraries.
The national library's right to harvest Web content will be granted in
the new Copyright Act, to be approved by the Parliament in late 2004 - early
2004. The new Copyright Act and Legal Deposit Act have been developed in
parallel, so that these laws refer to one another. The new Legal Deposit Act,
which covers electronic resources (on-line and off-line) plus radio and TV
programs, is yet to be passed from the Ministry of Education to the
Parliament.
Links to more specific information about Finnish Web archiving
initiatives are listed under Web archiving resources - Finland.
Updated July 2004
France
An extension of the French legal deposit law to networked digital
material should be voted by the French Parliament in 2004. It will allow BnF
and INA (Institut National de l'Audiovisuel, in charge of TV and Radio
preservation) to harvest sites on-line as well as request a deposit from
publishers, when online harvesting is impossible and make the collection
publicly available on site. At present the experimental archive is not publicly
accessible.
Ahead of any legislative amendments, the
Bibliothèque nationale de
France (BnF) has been active since 2000 in developing a combined
methodology including:
- Automatic large scale domain crawls several times a year
- Continuous crawl of automatically selected sites (10% of the
total)
- Deposit of deep web sites that can't be harvested on-line
- Thematic event-based collection for very ephemeral sites (that
crawlers would take too long to find)
In collaboration with INRIA (the French National Institute for Research
in Computer Science and Automatic Control), the BnF has been experimenting in
Web collection techniques using automatic harvesting and evaluation methods. A
test of automatic selection parameters based in-linking ranking (like Google)
has been made.
The initial phase of the program, which ran until June 2001, focused on
content gathering, involving the collection of 16 audiovisual sites with small
robots to assess the limits of these crawlers. In 2002 two harvests were made:
the first a thematic collection of sites relating to the French elections (1900
sites), and the second a comprehensive crawl of the .fr Web.
In a second pilot on methods for archiving hidden Web sites, which ran
from 2002 to June 2003, 100 selected Web owners were approached to deposit
their Web sites in the BnF for permanent archiving. A portable extraction tool
(DeepArc) was developed to enable simple extraction of database to XML by the
producers. This tool, currently being tested by IIPC members will be released
in open-source soon. It is envisaged that harvester technology may be employed
as a discovery tool to identify sites in the Deep Web of interest for
collection, which would then be transferred to the BnF after a technical
negotiation with site owners.
Aware of the necessity of international collaboration in the domain of
Web Archiving, BnF has launched the
International Web Archiving
Workshop (IWAW) series, and the
Web archive discussion
list and actively participated in the creation of the International
Internet Preservation Consortium (IIPC) that BnF coordinates.
Links to more specific information about French Web archiving
initiatives are listed under Web archiving resources - France.
Updated July 2004
Germany
In 1997, whilst awaiting updating of its legal deposit legislation to
include online publications, the German
National Library (Die Deutsche Bibliothek - DDB, which was renamed Deutsche Nationalbibliothek in 2006) commenced a voluntary deposit trial for electronic
materials. The project began by collecting electronic dissertations and theses
and was extended in 2000 when the SpringerLink publishing house began
depositing electronic journals. The scope of the program was broadened in the
same year with the issue of guidelines for the deposit of networked digital
publications, and the development of an automatic collection process for online
materials in 2001.
A cooperative agreement between the DDB and the German publishers' union
was reached in March 2002 to formalise the voluntary deposit scheme. Under the
terms of the arrangement, publishers deposit their electronic resources in the
Deposit.DBB.DE archive server,
to the contents of which the DDB provides in-house access. Dynamic publications
and databases are able to be transferred to the system subject to certain
transferability criteria, including their ability to operate with standard
software but independently of other servers.
An experiment in collecting dynamic Web sites was also conducted using
the German government Web site (Die Bundesregierung) and HTTrack harvester
technology. Crawls were performed in December 2000 and February 2001, from
which it was concluded that the most efficient and effective strategy for
Germany would be a legally-enforced deposit model rather than a resource
intensive 'pull' approach. The studies have also highlighted the importance of
ensuring the usability of archived material at an early stage in the archival
process so that access problems are not obscured until it is too late for them
to be resolved.
Links to more specific information about German Web archiving
initiatives are listed under Web archiving resources - Germany.
Updated May 2004
Japan
In 2000 amendments were made to Japanese legal deposit law to encompass
CD-ROMs and other physical format materials. The National Diet Library
submitted a request to the Legal Deposit System Research Council in 2002
requesting that either legal deposit legislation be extended to online digital
information, or that a legal framework be developed for their collection by the
NDL.
Whilst awaiting the outcome of the Council's deliberations, the NDL has
initiated three-year trial projects and studies, one of which is
WARP (Web Archiving Project), an
experimental investigation into the gathering and archiving of Web resources,
and another is the research of the country's long-term digital preservation
needs.
Through the WARP project, the NDL is collecting snapshots of Japanese
Web sites and testing storage and preservation solutions, with the agreement of
selected participating organisations. WARP is also being used to collect and
archive electronic periodicals as the basis for an online periodicals
collection. In June 2004, 600 Japanese Web sites and more than 1100 electronic
journals had been collected.
Future research interest is focused on the compilation of comprehensive
selection guidelines for the preservation of Japanese digital heritage, the
exploration of comprehensive harvesting techniques, and the development of
rendering and searching techniques.
Links to more specific information about Japanese Web archiving
initiatives are listed under Web archiving resources - Japan.
Updated July 2004
Lithuania
The Martynas Mažvydas National Library of Lithuania is the legal
deposit institution for Lithuanian publications. Electronic resources were
included in a 1996 general deposit decree, though the exact extent of the
provision was not defined. To guide the NLL in administering its national
collecting responsibilities for online materials, selection criteria were
compiled to include the collection of resources from the .lt domain, other
documents of special quality and of acknowledged Lithuanian interest, and
commercial and government e-publications. In accordance with the guidelines,
physical format materials are to be deposited in a traditional manner, whilst
static electronic documents will be gathered using harvesting methods. Dynamic
resources and other public database content are to obtained by way of
depository agreements concluded with publishers.
To support the collection of online Lithuanian material the
Archive
of Electronic Resources was established in November 2002 as a subsystem of
the LIBIS (Lithuanian Integrated Library Information System) program. Once
fully operational, it is envisaged that the AER will operate as a deposit
system for online publications, based upon the NEDLIB model.
In addition, the NLL has been a participant in the
NEDLIB project and the NEDLIB
harvester was employed in the first sweep of the Lithuanian Web in October
2002. A second crawl in November 2002 involved the selective capture of
approximately 60 registered e-periodicals. Many of these comprised dynamically
generated Web pages, which would be overlooked by existing harvesting
techniques. To address this issue, agreements for the deposit of e-publications
were forwarded in March 2003 to the principal Lithuanian publishers. Regular
harvests of the Lithuanian Web are now conducted every six months and rights
clearances for each resource are being simultaneously obtained, together with
metadata elements in Dublin Core standard.
Links to more specific information about Lithuanian Web archiving
initiatives are listed under Web archiving resources - Lithuania.
Updated May 2004
The Netherlands
In 1995 the Depot van Nederlandse Electronische Publicaties (DNEP)
project was established in The Netherlands to construct a system for the
voluntary deposit of digital publications. The country is one of the few in the
world that does not have legal deposit legislation, and lodgement of Dutch
resources in the National Library of the Netherlands (Koninklijike Bibliotheek)
is effected through voluntary agreements with publishers.
After an extended period of experimentation, the KB entered into a
project with IBM in 2000 to develop a deposit archive for Dutch electronic
publications based on the OAIS framework. The resultant
e-Depot
was introduced at the KB in 2002 where it has become the official archive of
the major Dutch publishing houses Kluwer and Elsevier Science, and other
members of the Dutch Publishers Association. Deposit of electronic publications
in the e-Depot is not restricted to publications with Dutch imprint, but is
based on agreement with publishers. While Elsevier and Kluwer were Dutch
publishers in origin, new arrangements have been drawn up with international
publishers that do not have a specific connection with the Netherlands, like
the open access publisher BioMed Central and Blackwell Publishing. Now fully
operational, work is continuing on the development of a Long-Term Preservation
Module to provide access to stored data, manage metadata, and enable migration
and emulation strategies to be effected. The Dutch research initiatives have
been extensively documented, and a number of reports and a project overview
have been produced by the working group.
In another collaboration, the KB was the lead partner in the
NEDLIB project, which ran between
1998 and 2000. Through this program the KB trialled harvesting of the Dutch
domain using the NEDLIB harvester and participated in the development of
guidelines for online deposit systems.
Links to more specific information about Dutch Web archiving initiatives
are listed under Web archiving resources - The Netherlands.
Updated July 2004
New Zealand
The National Library of New Zealand has been formally researching Web
archiving since 2002 (although earlier tests were undertaken during the 1999
election).
Harvesting experiments have been undertaken using HTTrack technology and
have targeted the 2002 national election, the America's Cup regatta held in
Auckland and government web sites. Over 400 discrete objects and full web sites
have been captured and stored since December 2002. In important supporting
research, the Library has developed a preservation metadata model, the full
version of which was issued in July 2003.
The National Library of New Zealand Act 2003 expanded the scope of the
Library's legal deposit mandate to include online and offline electronic
resources including both publicly available and 'deep-web' Web resources. In
anticipation of the legislative amendments, the NLNZ began actively developing
its processes for the selection, acquisition and archiving of physical format
and online electronic documents into its existing workflows.
In 2003 a selection database was created for recording selection
decisions and in early 2004 the Library began researching available web
archiving software including the National Library of Australia's PANDAS
technology.
In May 2004, following a major review of the NLNZ's digital library
directions, substantial funding was announced for the establishment of a New
Zealand digital archive to safeguard the national output of online and offline
electronic resources which will include web archiving activities.
The National Library of New Zealand's web archiving approach will
comprise a combination of snapshot harvesting of the whole .nz domain at
defined intervals, selective harvesting of specific web sites or parts of web
sites, and event harvesting of resources associated with topics such as the
national elections and the America's Cup regatta.
Links to more specific information about New Zealand Web archiving
initiatives are listed under Web archiving resources - New Zealand.
Updated July 2004
Norway
The National Library of Norway started the Paradigma Project in
August 2001 and it is scheduled to run until 31 December 2004. The main goal of
the initiative is to ensure the satisfactory legal deposit of Norwegian digital
documents, and this includes the development of the technology, methodology and
routines for the selection, collection, description and identification of all
types of digital documents - including dynamic Internet documents.
The Library is also working to give users access to its Internet archive
in compliance with the Legal Deposit Act, for purposes such as research and
documentation. Considered extremely modern when it came in to effect in 1989,
the current Legal Deposit Act, covers all generally available Norwegian
documents stored on any medium: including paper, microforms, photographs,
combined documents, sound fixations, films, video, digital documents and
broadcasting programs. Documents published abroad for Norwegian publishers and
those specially adapted for a Norwegian public are also covered.
Based on the Legal Deposit Act and recommendations from the Paradigma
Project, the National Library has started the general harvesting of all
generally available digital documents from the Norwegian Web space ('.no'). The
first harvesting round was carried out in December 2002-January 2003, and it
resulted in approximately 3.1 million URLs. The second harvesting round in
August 2003 resulted in approximately 4.1 million URLs. The third harvesting
round is currently underway, and in time, documents found on domains such as
'.com', '.org' and '.net', will also be harvested.
An extremely challenging issue is the deposit of resources from the Deep
Web, such as Internet newspapers, streaming media, documents from web cameras,
interactive media and e-materials of all types stored in databases. The
Paradigma Project has started the daily collection of approximately 65 Internet
newspapers, and it will be downloading several entire newspaper databases in
the near future, thus complementing the daily 'snapshots'. The National Library
of Norway is also discussing Deep Web problems within the framework of the
International Internet
Preservation Consortium.
The Paradigma Project has recommended selection criteria for digital
documents based on the Legal Deposit Act and the Library's collection strategy.
These criteria will influence the Library's manual routines as well as the
design of its Internet archive's system architecture. The Project plans to
implement a three-phase selection process, so that librarians receive technical
help to find the few documents that qualify to be catalogued at some level. The
first selection phase finds and collects the Norwegian and Sami documents from
the Internet. The second phase gives librarians the opportunity to
automatically produce ranked lists based on specific queries. These lists are
based on the use of vectors containing metadata that has been automatically
extracted from the collected documents. In the third phase, librarians choose
specific documents from the ranked lists for manual registration at some level,
using the selection criteria mentioned above.
The Paradigma Project is currently investigating metadata formats for
the description of digital documents. Although many digital documents may
deserve a bibliographic description, the Paradigma Project estimates that far
less than 1 per cent of the material collected from the Norwegian Internet
domain may ever be subject to cataloguing at some level. In contrast, 100 per
cent of the Internet documents will be fully indexed with FAST indexing
software after harvesting. This will allow the Library's staff and users to
search the Internet archive - both via free text as well as other indices. In
addition to cataloguing some documents, and indexing all documents, the Project
will be harvesting existing embedded metadata as well as the Internet documents
they describe, and the National Library is planning a future service that will
allow publishers to generate and deliver metadata with their documents at the
time of deposit.
Giving users access to the legal deposit Internet archive is a complex
matter, and the National Library must find satisfactory solutions in spite of
the many, and sometimes conflicting, regulations found in the Legal Deposit
Act, the Copyright Act and the Personal Data Act. The Paradigma Project is
currently trying to find answers to questions like: Which users can receive
access to different types of digital materials? Can they access the collections
from computers outside the National Library?
The Library plans to give users access to its Internet archive via the
Nordic Web Archive's (NWA) Access Tool. Today,
free text searching with Boolean operators, a search for a certain URL, and the
presentation of document history via a timeline are standard options. The
Paradigma Project will adapt this tool's interface to accommodate several
special user functions, and because it is desirable to present the archived
digital documents and metadata in an organised and structured way, IFLA's
Functional Requirements for
Bibliographic Records (FRBR) model will be used in the Internet archive's
design.
Links to more specific information about Norwegian Web archiving
initiatives are listed under Web archiving resources - Norway.
Updated June 2004
Portugal
The National Foundation for Scientific Computing (FCCN – Fundação para a Computação Científica Nacional), aims to preserve information published on the Portuguese web domain (.pt) through the Arquivo da Web Portuguesa : Portuguese Web Archive. FCCN is a non profit organisation, which manages a network connecting Portuguese educational and scientific institutions, a registry for the .pt top-level domain and an Internet Exchange Point. The first developmental stage of the Portuguese Web Archive began in January 2008 and will run for two years. However the maintenance of the system and preservation of the information that is archived is intended to continue beyond that date.
The Portuguese Web Archive follows earlier work by the XLDB research group at the University of Lisbon which has been interested in archiving the Portuguese web since 2001. A web archive prototype (Tomba) was developed to enable access to the data crawled by the tumba! search engine which crawled the Portuguese web during 2002-2006.
Links to more specific information about Portuguese Web archiving initiatives are listed under Web archiving resources - Portugal.
Updated March 2009
Sweden
From its inception in September 1996, the Swedish
Kulturarw3 (Cultural Heritage
Cubed) project has explored methods for the collection, preservation and
provision of access to Swedish online documents. A project of the Swedish Royal
Library, Kulturarw3 is engaged in the comprehensive collection of static and
dynamic Internet resources from the Swedish Web.
In a significant technological advance, Sweden was the first country to
investigate the use of harvester technology for archiving Web content. The
initial harvest in 1997 was limited to the .se domain but subsequent crawls
have been expanded to include other Swedish server addresses. Since 2002 the
Royal Library has been authorised to collect Swedish Web sites on the Internet,
and two to three harvests are carried out per year. The Royal Library is
responsible for storage and long-term preservation of the collected resources,
which may only be accessed on-site at the Library's premises. From 2003, access
to the archive content has been provided using surfing software akin to the
Internet Archive's Wayback Machine.
Research effort is now engaged in developing a tool to enable free text
searching.
The Royal Library has also been active in the
Nordic Web Archive project.
Links to more specific information about Swedish Web archiving
initiatives are listed under Web archiving resources - Sweden.
Updated June 2004
United Kingdom
Research and experimentation in digital archiving in the United Kingdom
can be characterised as beginning with pilot initiatives involving individual
institutions followed by collaboration nationally and internationally.
In 2001 the British Library carried out a six-month experiment to select
and capture (but not make publicly accessible) 100 UK Websites. The
Domain.uk project, as
it was called, was based on obtaining explicit agreement of Website publishers
and included in its scope a broad range of UK Websites of historical and
cultural significance. It also included some key current events such as the
General Election of 2001 and the foot and mouth disease epidemic of the same
year. The success of Domain.uk led to the setting up of the British Library's current Web Archiving Programme, which is focussed on the following interlinked
strands: the definition of a collection development policy for Web archiving;
participation, as a lead partner in the UK Web Archiving Consortium (UKWAC);
and participation in the International Internet Preservation Consortium (IIPC),
led by the Bibliotheque nationale de France.
In parallel with these developments, legal deposit legislation within
the UK has now become subject to the Legal Deposit Libraries Act 2003, which
reached the statute book in February 2004. The Act extends legal deposit to
non-print and currently comprises enabling legislation. A governmental Advisory
Panel is to be set up to oversee the various stages of secondary legislation
which are likely to lead to Regulations by format, e.g., offline, e-books,
e-journals, Websites. Pending secondary legislation, the British Library Web
archiving initiatives will be based entirely on a voluntary, rights-cleared
basis.
Links to more specific information about British Web archiving
initiatives are listed under Web archiving resources - United Kingdom.
Updated July 2004
United States
The Minerva (Mapping the
Internet the Electronic Resources Virtual Archive) project was established by
the Library of Congress in 2000 as an experiment to provide experience in the
archiving of 35 selected Web sites. Snapshots of three sites for closer study
were downloaded using HTTrack software and catalogued onto LC's library
management system. A Web site was created to test access, and legal issues were
examined in cooperation with the US Copyright Office. In a parallel study,
daily snapshots of approximately 200 sites relating to the 2000 presidential
election were gathered by the Internet
Archive. Based on the results of these pilots, recommendations were made
regarding future larger-scale archiving initiatives.
Today Minerva has developed into an ongoing thematic digital archiving
program, which focuses on the harvesting of topic based public Web content.
More than 35,000 sites have been archived since 2000, including the 9/11 and
9/11 Anniversary, Election 2000 and 2002, and the 2002 Winter Olympics sites,
in association with the Internet Archive which acts as the contractor for
crawling and capture. LC has also explored the legal aspects of collecting Web
content and it has been interpreted that the Library is entitled to collect and
make accessible networked materials, though this is not explicitly stated in
legal deposit legislation.
It is recognised that both comprehensive and selective harvesting will
need to be employed by LC in future if the volume and complexity of the United
States Web is to be adequately preserved. Present directions suggest that this
will best be achieved through partnerships with organisations such as the
Internet Archive. As part of this strategic path, the Library of Congress is
also leading a cross-sector cooperative online archiving venture known as the
National Digital Information
Infrastructure and Preservation Program (NDIIPP). The NDIIPP will
coordinate existing digital archiving research programs across the public and
private sectors in the United States to minimise replication, allocate
responsibilities and to share knowledge and experience.
Links to more specific information about United States Web archiving
initiatives are listed under Web archiving resources - U.S.A.
Updated May 2004
Collaborative
initiatives
International Internet
Preservation Consortium
(IIPC) In July 2003 the International Internet Preservation Consortium
(IIPC) was created as a collaboration between the national libraries of France,
Australia, Canada, Denmark, Finland, Iceland, Italy, Norway, Sweden, the
British Library, the Library of Congress and the Internet Archive. Led by the
Bibliothèque nationale de France, the Consortium has been formed to
support and foster development in the collection, archiving and provision of
access to international Internet content in the long-term. Encouraging the use
of common tools and methods to facilitate interoperability, and the raising of
awareness of Internet preservation issues the are other key objectives of the
Consortium.
Six working groups have been established to consider issues relating to
a common technical framework, researcher requirements, access tools, metrics
and the evaluation of Web crawlers, the Deep Web and content management. Until
the expiry of the initial three-year agreement, membership of the IIPC is
limited to the founding partners, but it is hoped that more national libraries
will join the Consortium after mid-2006.
Links to more specific information about the IIPC are listed under Web
archiving resources - IIPC.
Updated July 2004
Internet Archive
A San Francisco based non-profit commercial venture, the Internet
Archive is comprehensively harvesting and preserving publicly available
materials from the global Web, and has amassed the largest collection of public
Web pages in the world since it began its pioneering activity in 1996. The core
of the Internet Archive's collections are received from Alexa Internet, a
commercial crawler and research company which donates collected data to IA
immediately following each crawl period. Broad snapshot crawls are made every
two months, and the content of the archive was made publicly accessible in 2001
via the 'Wayback Machine'.
This whole domain approach is based on periodic harvesting, in which
snapshots of Web sites of all levels of quality and subject scope are collected
and stored for future reference. As many sites as possible are collected but
not all of these in their entirety, currently a total of 45 million sites. In
tandem some narrow crawls are made of selected sites to collect their entire
content.
More recently, the Internet Archive has also entered into partnerships
with other institutions to develop special collections according to the
institution's profile. The IA has also be active in trying to enter into
collaborations with national institutions in order to collect and build
interoperable national heritage collections, using IA technical expertise and
data collection based on each country's specifications.
Links to more specific information about the Internet Archive are listed
under Web archiving resources - Internet Archive.
Updated July 2004
NEDLIB
Funded by the European Commission, the NEDLIB (Networked European
Deposit Library) Project ran between 1997 and 2000 with the purpose of
developing harvesting software specifically for the collection of Web resources
for a European deposit library. Harvesting technology for indexing Web content
had been in place since the mid-1990s, but NEDLIB was one of the first crawlers
to be specifically developed for archiving purposes. The new software was
created by the Finnish IT Center for Science (CSC), to a specification
formulated by the multiple project partners (a consortium of eight European
national libraries, a national archive, two IT companies and three publishing
houses).
In January 2000, the first version of the NEDLIB harvester was released
and tested by project members and the national libraries of Estonia and
Iceland. An updated system was tested on the Icelandic Web in January 2001, and
with further modifications on the Finnish domain in 2002. The NEDLIB initiative
revealed that whilst it is a fairly simple undertaking to design a basic
harvester, it is a very complex task to develop software that is efficient,
robust, and that can be programmed to capture every nuance of selection.
Although the project is now defunct, the NEDLIB site is managed by the
Koninklijke Bibliotheek (the National Library of the Netherlands). The NEDLIB
harvester can still be found at http://web.archive.org/web/20051217014839/www.csc.fi/sovellus/nedlib/.
The latest version was published in September 2002. The application is no
longer supported or developed; in Finland, it will be replaced by Heritrix in
Summer 2004.
Links to more specific information about the NEDLIB project are listed
under Web archiving resources - NEDLIB.
Updated July 2004
Nordic Web Archive
The Nordic Web Archive is a cooperative project of the national
libraries of Denmark, Finland, Iceland, Norway and Sweden for the sharing of
experience in Web archiving. The major focus of the collaboration has been the
development of an interoperable toolset for providing long-term access to Web
archives. Funding was received in August 2000 for the first phase of the
program, which is administered from the National Library of Norway.
Digital archives exist in all the participating countries, the content
of which is transferred to the central NWA. A software package known as the NWA
toolset has been developed using expertise from each partner library to enable
navigation and searching of archived content in both time and space.
The second phase of the NWA project began in late 2002, and is focused
on the enhancement of software stability, the need for additional user
functionality, and the re-evaluation of the user interface.
Links to more specific information about the Nordic Web Archive are
listed under Web archiving resources - Nordic Web Archive.
Updated May 2004
UK Web Archiving
Consortium
Comprising six leading UK institutions, the UK Web Archiving Consortium
(UKWAC) will work, with the permission of rightsholders, on an experimental
system for archiving selected key UK Websites - ensuring that invaluable
scholarly, cultural and scientific resources remain available for future
generations.
The UKWAC - comprising The British Library, Joint Information Systems
Committee of the Higher and Further Education Councils (JISC), The National
Archives, The National Library of Wales, The National Library of Scotland and
the Wellcome Trust - will run for an initial period of two years, during which
approximately 6,000 websites will be collected and archived.
Consortium members will obtain the permission of website owners to
archive selected sites whilst working collaboratively to explore how to develop
compatible selection policies and to investigate the complex technical
challenges involved in collecting and archiving Web material.
Each consortium member will select and 'capture' content relevant to its
subject and/or domain. For example, The British Library will archive sites
reflecting national culture and events of historical importance. These could
include Web pages focusing on key events in national life, museum Web pages,
e-theses, selected blogs to support research material and Web-based literary
and creative projects by British subjects.
Wellcome will preserve a record of medicine on the Web whilst The
National Archives will focus on archiving selected material from six main
clusters of government departments. The Scottish and Welsh National Libraries
will collect material reflecting the culture and history of Scotland and Wales
and JISC will preserve Websites from leading-edge, innovative ICT projects in
UK Higher and Further Education.
Infrastructure costs, such as software, hardware, and ongoing technical
development and support will be shared equally amongst the Consortium members.
UKWAC will use HTTrack - the open source Web crawler - to acquire films for
storage. The software to carry out the archiving processes - PANDORA Digital
Archiving System (PANDAS) - has already been developed and tested by the
National Library of Australia and its partners for archiving Australian
Websites and making them accessible through PANDORA.
UKWAC members have selected Magus Research Limited to help extend the
PANDAS software for UK needs and provide the shared hardware and technical
support they require.
Updated July 2004
4. Resources (NB: This section contains lists of selected resources - see also additional resources by PADI Type listed below)
Web archiving resources -
Overviews
Archiving the Web : European Experiences
http://www.kansalliskirjasto.fi/extra/tietolinja/0203/webarchive.html
Archiving the World Wide Web
http://www.clir.org/pubs/reports/pub106/Web.html
Management of Networked Electronic Publications : A Table of Status in
Various Countries
http://epe.lac-bac.gc.ca/100/206/301/lac-bac/consultation_online_publications-ef/8/7/r7-100-e.html
Best Practices for Digital Archiving : An Information Lifecycle Approach
http://www.dlib.org/dlib/january00/01hodge.html
Digital Preservation and Permanent Access to Scientific Information :
The State of the Practice
http://cendi.dtic.mil/publications/04-3dig_preserv.html
Web Archiving : Bibliography
http://www.ifs.tuwien.ac.at/~aola/links/WebArchiving.html
Digital Electronic Archiving : The State of the Art and the State of the
Practice http://web.archive.org/web/20040715012747/http://www.icsti.org/Dig_Archiving_Report_1999.pdf
ERPANET Workshop - The Long Term Preservation of Databases : ERPANET
Workshop Report
http://www.erpanet.org/events/2003/bern/Bern_Report_final.pdf
Collecting and Preserving the World Wide Web : a Feasibility Study
Undertaken for the JISC and Wellcome Trust
http://www.jisc.ac.uk/uploaded_documents/archiving_feasibility.pdf
Legal Issues Relating to the Archiving of Internet Resources in the UK,
EU, USA and Australia : A Study Undertaken for the JISC and Wellcome Trust
http://www.jisc.ac.uk/uploaded_documents/archiving_legal.pdf
The Last Page of the Internet : the Importance of Preserving the
Dynamic Aspects of the Internet
http://www.deflink.dk/upload/doc_filer/doc_alle/1023_NBR.doc
Web archiving resources - Australia
PANDORA - Preserving and Accessing Networked Documentary Resources in
Australia
http://pandora.nla.gov.au/index.html
Web Archiving in a Web 2.0 World (2008)
http://pandora.nla.gov.au/pan/13910/20080930-1156/conferences.alia.org.au/alia2008/pdfs/124.TT.pdf
Ensuring Long-Term Access to Online Publications (2004)
http://quod.lib.umich.edu/cgi/t/text/text-idx?c=jep;view=text;rgn=main;idno=3336451.0004.405
Collecting and Managing Web Resources for Long-Term Access : Web Harvesting and Guidelines to Support Preservation (ICABS Actions 3.3 and 3.4)
http://archive.ifla.org/IV/ifla70/papers/026e-Gatenby.pdf
PANDORA, Australia's Web Archive, And The Digital Archiving System That Supports It
http://www.digicult.info/downloads/dc_info_issue6_december_20031.pdf
Archiving the Web : the PANDORA Archive at the National Library of
Australia
http://www.nla.gov.au/nla/staffpaper/2001/cathro3.html
Report On Senior Executive Fellowship to Research Digital Archiving in National Libraries
http://www.nla.gov.au/nla/staffpaper/2002/elect.html
Digital Preservation : The Australian Experience
http://www.nla.gov.au/nla/staffpaper/dw001004.html
Digital Archiving - Developing Policy And Best Practice Guidelines At The
National Library Of Australia
http://web.archive.org/web/20010303103129/www.icsti.org/icsti/2000workshop/gatenby.html
Legal Issues Relating to the Archiving of Internet Resources in the UK, EU,
USA and Australia : A Study Undertaken for the JISC and Wellcome Trust (2003)
http://www.jisc.ac.uk/uploaded_documents/archiving_legal.pdf
Web archiving resources - Austria
Austrian National Library Web Archiving
http://www.onb.ac.at/ev/about/webarchive.htm
AOLA : Austrian On-Line Archive
http://www.ifs.tuwien.ac.at/~aola/
Uncovering Information Hidden in Web Archives: a Glimpse at Web
Analysis Building on Data Warehouses
http://www.dlib.org/dlib/december02/rauber/12rauber.html Austrian
On-Line Archive : Current Status and Next Steps
http://bibnum.bnf.fr/ecdl/2001/austria/austria.pdf
Long-Term Preservation of Digital Material - Building an Archive to
Preserve Digital Cultural Heritage From the Internet
http://www.ifs.tuwien.ac.at/~aola/publications/thesis-ando.pdf Part
of Our Culture is Born Digital : On Efforts to Preserve it for Future
Generations http://www.inst.at/trans/10Nr/rauber10.htm
Archiving the Internet : Challenges, Projects, and the Austrian Perspective
http://www.ifs.tuwien.ac.at/~aola/publications/slides_reichenau_01.pdf
Web archiving resources - Canada
Government of Canada Web Archive
http://www.collectionscanada.gc.ca/webarchives/index-e.html
Library and Archives Canada Electronic Collection
http://www.collectionscanada.gc.ca/electroniccollection/003008-200-e.html
Government Web Content in Canada a National Library Web Archive Perspective (2008)
http://archive.ifla.org/IV/ifla74/papers/130-Cantello_Stegenga-en.pdf
Networked Electronic Publications : Policy and Guidelines, 1998
http://epe.lac-bac.gc.ca/100/200/301/nlc-bnc/networked_epubs-ef/9/8/index-e.html
Electronic Publications Pilot 1996/97 : Final Report
http://epe.lac-bac.gc.ca/100/200/301/statcan/electronic-e/0009711-534-RIE.pdf
Electronic Publications Pilot Project (EPPP) : Summary of the Final
Report http://epe.lac-bac.gc.ca/100/200/301/nlc-bnc/eppp_summary-e/ereport.htm
Networked Electronic Publications Policy, National Library News article May 1999
http://web.archive.org/web/20041123172033/http://www.collectionscanada.ca/bulletin/015017-9905-07-e.html
Web archiving resources - Czech Republic
WebArchiv : Archive of the Czech Web http://en.webarchiv.cz/
Archiving the Czech Web : Issues and Challenges
http://bibnum.bnf.fr/ecdl/2003/index.html
Research Intent - Digital Libraries
http://wwwdata-old.muni.cz/research/cez_item.asp?ID=23
Digital Cultural Heritage and the Cooperation of National Memory Institutes http://en.webarchiv.cz/files/dokumenty/konference/Zabickabanskabystrica2006.doc
Web archiving resources - Denmark
netarchive.dk (2005-)
http://netarchive.dk/index-en.php
Netarchive.dk : ECDL Workshop, October 19th 2002
http://bibnum.bnf.fr/ecdl/2002/Kb/Kb.html
netarchive.dk - the Pilot Project
http://web.archive.org/web/20060301064321/http://netarchive.dk/pilot-index-en.php
netarchive.dk - 2nd phase 2004
http://web.archive.org/web/20051213191653/netarchive.dk/fase2index-en.php
netarchive.dk - the Pilot Project : Project Reports
http://web.archive.org/web/20060301064315/netarchive.dk/rap/index-en.php
Legal Deposit From the Internet in Denmark - Experiences With the Law From
1997 and the Need for Adjustments
http://www.deflink.dk/upload/doc_filer/doc_alle/1023_BNH.doc
Legal Deposit In Denmark - The New Law and Electronic Products (1999)
http://web.archive.org/web/20070608091825rn_1/liber-maps.kb.nl/articles/dupont11.htm
Web archiving resources - Finland
National Library of Finland Web Archive
http://www.nationallibrary.fi/services/digitaalisetkokoelmat/webarchive.html
Finland's Internet Archive (2009)
http://www.kansalliskirjasto.fi/extra/bulletin/article4.html
EVA : The Acquisition and Archiving of Electronic Network Publications
http://web.archive.org/web/20041010005510/www.lib.helsinki.fi/eva/english.html
EVA : Elektronisen Verkkoaineiston Hankinta ja Arkistointi : Raportteja
http://web.archive.org/web/20041216011419/http://www.lib.helsinki.fi/eva/raportit.html
Harvesting the Finnish Web Space : Practical Experiences
http://bibnum.bnf.fr/ecdl/2001/finland/sld001.htm
EVA - The Acquisition and Archiving of Electronic Network Publications
In Finland http://www.lib.helsinki.fi/tietolinja/0199/evaart.html
Experiences and Conclusions From a Pilot Study : Web Archiving of the
District and County Elections 2001
http://netarchive.dk/publikationer/webark-final-rapport-2003.pdf
Legal Deposit Act (Draft)
http://www.minedu.fi/export/sites/default/OPM/Julkaisut/2003/liitteet/opm_132_Legal_Deposit_Act.pdf?lang=en
Web archiving resources - France
Digital Legal Deposit
http://www.bnf.fr/PAGES/version_anglaise/depotleg/dl-internet_intro_eng.htm
Legal Deposit of the French Web: Harvesting Strategies for a National Domain (2008)
http://iwaw.net/08/IWAW2008-Lasfargues.pdf
Towards Continuous Web Archiving : First Results and an Agenda for the
Future (2002) http://www.dlib.org/dlib/december02/masanes/12masanes.html
A First Experience in Archiving the French Web
ftp://ftp.inria.fr/INRIA/Projects/verso/gemo/GemoReport-229.pdf
The BnF's Project for Web Archiving
http://bibnum.bnf.fr/ecdl/2001/france/sld001.htm International
Web Archiving Workshops (IWAW) http://www.iwaw.net/ Web Archive (discussion
list) http://listes.cru.fr/wws/info/web-archive
Web archiving resources - Germany
Archive Server DEPOSIT.D-NB.DE
http://deposit.ddb.de/index_e.htm
Deutsche National Bibliothek Online Publications
http://www.d-nb.de/eng/netzpub/index.htm
Collection of German online resources by Die Deutsche Bibliothek
http://bibnum.bnf.fr/ecdl/2001/germany/sld001.htm
Long-term archiving of digital documents : what efforts are being made in
Germany?
http://caliban.ingentaselect.com/vl=9314169/cl=57/nw=1/fm=docpdf/rpsv/cw/ alpsp/09531513/v16n3/s9/p207
NESTOR : Network of Expertise in Long-term STOrage of Digital Resources - A
Digital Preservation Initiative for Germany
http://www.dlib.org/dlib/april04/dobratz/04dobratz.html
NESTOR : Kompetenznetzwerk Langzeitarchivierung : Network of Expertise in
Digital Preservation
http://www.langzeitarchivierung.de/index.php?newlang=eng
Long-term Preservation of Electronic Theses and Dissertations
http://edoc.hu-berlin.de/etd2003/liegmann-hans/HTML/index.html
Web archiving resources - Japan
WARP http://warp.ndl.go.jp/
Digitizing, Archiving and Preserving Japanese Cultural Heritage
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070519/viewer/file1230.html#feature1
Web Resources as Cultural Heritage: International Symposium on Web
Archiving http://www.ndl.go.jp/en/publication/proceedings/index.html Legal
Deposit System Council
http://www.ndl.go.jp/en/aboutus/deposit_council_book.html
Electronic Publications and the Legal Deposit System in Japan
http://www.ndl.go.jp/en/publication/ndl_newsletter/107/076.html
Electronic Publications and National Bibliography in Japan
http://www.ndl.go.jp/en/publication/ndl_newsletter/112/123.html
Web archiving resources - Lithuania
Harvesting and Archiving of Electronic Resources in Lithuania : Towards
Virtual Library
http://www.inforum.cz/inforum2003/prispevky/Jodelis_Remigijus.pdf
Web archiving resources - The Netherlands
Koninklijke Bibliotheek Web Archiving
http://www.kb.nl/hrd/dd/dd_projecten/webarchivering/index-en.html
e-depot and Digital Preservation
http://www.kb.nl/hrd/dd/index-en.html KB/IBM
Long Term Preservation Study
http://www.kb.nl/hrd/dd/dd_onderzoek/reports/6-webpublications.pdf
Digital Preservation in Practice: The e-Depot at the Koninklijke
Bibliotheek
http://puck.emeraldinsight.com/vl=2423084/cl=15/fm=html/nw=1/rpsv/cw/ mcb/03055728/v34n1/s3/p21
Legal Deposit of Digital Materials
http://www.kb.nl/kb/hrd/dd/dd_links_en_publicaties/publicaties/LIBER-Oltmans-input.doc
Digital Archiving : A Necessary Evil or New Opportunity?
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6W63-4BV52PK-1&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=3ab0c901795f97b1a891e20529da4e85
Digital Preservation and Permanent Access: The UVC for Images
http://www.kb.nl/hrd/dd/dd_links_en_publicaties/publicaties/uvc-ist.pdf The
Road to E-Deposit at the Koninklijke Bibliotheek
http://www.kb.nl/hrd/dd/dd_links_en_publicaties/publicaties/eskb-roadtoe-deposit3_20aug02.pdf Permanent Archiving of Electronic Publications : Research &
Practice
http://www.kb.nl/hrd/dd/dd_links_en_publicaties/publicaties/ summerschoolticer2003.pdf
Preservation Functionality in a Digital Archive
http://portal.acm.org/citation.cfm?id=996350.996416 Preserving
electronic publications
http://www.kb.nl/hrd/dd/dd_links_en_publicaties/publicaties/isu350.pdf
Web archiving resources - New Zealand
NDHA National Digital Heritage Archive
http://ndha-wiki.natlib.govt.nz/ndha/
National Library to Capture New Zealand's Digital Heritage, Media Release 30 May 2004
http://web.archive.org/web/20060307100946/http://www.natlib.govt.nz/bin/media/pr?item=1085885702
National Library of New Zealand (Te Puna Matauranga o Aotearoa) Act
2003 http://www.natlib.govt.nz/files/Act03-19.pdf
National Library to lead electronic harvesting, Media Release 26 September 2003
http://web.archive.org/web/20051025111655/http://www.natlib.govt.nz/bin/media/pr?item=1064531843
National Library of New Zealand - Digital Library Development Review, July 2003
http://www.natlib.govt.nz/files/ross_report.pdf
National Library of New Zealand Preservation Metadata Schema
http://www.natlib.govt.nz/catalogues/library-documents/preservation-metadata-revised
National Library of New Zealand Preservation Metadata Data Model
http://www.natlib.govt.nz/catalogues/library-documents/downloadpage.2007-02-15.6613783926
Web archiving resources - Norway
The Paradigma Project
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file1651.html#feature2
The Paradigma Web Harvesting Environment
http://bibnum.bnf.fr/ecdl/2003/index.html
Identification of Network Accessible Documents : Problem Areas and
Suggested Solutions http://bibnum.bnf.fr/ecdl/2003/index.html
The Paradigma Project and its Quest for Metadata Solutions and User
Services http://archive.ifla.org/IV/ifla70/papers/009e-Nuys.pdf
Web archiving resources - Portugal
Arquivo da Web Portuguesa : Portuguese Web Archive http://www.arquivo.pt (In Portuguese)http://arquivo-web.fccn.pt/(In English)
Introducing the Portuguese Web Archive Initiative http://iwaw.net/08/IWAW2008-Gomes.pdf
Tomba : Portuguese Web Archive http://tomba.tumba.pt/ (In Portuguese)http://tomba.tumba.pt/index_en.html (In English)
Design and Selection Criteria for a National Web Archive http://xldb.fc.ul.pt/daniel/docs/papers/gomes06tomba.pdf
Web archiving resources - Sweden
Kulturarw3 http://www.kb.se/english/find/internet/websites/
New Decree for Kulturarw3
http://web.archive.org/web/20020803234652/http://www.kb.se/Info/Pressmed/Arkiv/2002/020605_eng.htm
The Collection of Swedish web pages at the Royal Library - The Web Heritage
of Sweden http://archive.ifla.org/IV/ifla68/papers/111-163e.pdf
Kulturarw3, the Swedish WWW-Archive : Or, to Preserve the Swedish World
Wide Web http://bibnum.bnf.fr/ecdl/2001/sweden/sld001.htm
The Kulturarw3 Project - The Royal Swedish Web Archiw3e - An Example of
'Complete' Collection of Web Pages
http://archive.ifla.org/IV/ifla66/papers/154-157e.htm
The Kulturarw3 Project - The Royal Swedish Web Archiw3e
http://web.archive.org/web/20060623011336/http://www.kb.se/kw3/articles/article000605.pdf
Web archiving resources - United Kingdom
UK Web Archive
http://www.webarchive.org.uk/ukwa/
Web Archiving in the UK: Cooperation, Legislation and Regulation (2008)
http://liber.library.uu.nl/publish/articles/000258/article.pdf
UK Government Web Archive
http://www.nationalarchives.gov.uk/preservation/archivedwebsites.htm
Tuck, J (2006). ‘Collecting, selecting and legal deposit.’ DPC Forum on Web Archiving, 12th June 2006
http://www.dpconline.org/docs/events/060612Tuck.pdf
Digital Preservation Activities in the UK : Building the Infrastructure
http://archive.ifla.org/IV/ifla69/papers/129e-Jones.pdf
Web Archiving : a Feasibility Study for JISC and the Wellcome Trust
http://www.jisc.ac.uk/index.cfm?name=project_webarchiving
Legal Issues Relating to the Archiving of Internet Resources in the UK,
EU, USA and Australia : A Study Undertaken for the JISC and Wellcome Trust
http://www.jisc.ac.uk/uploaded_documents/archiving_legal.pdf
Domain UK : Britain on the Web
http://bibnum.bnf.fr/ecdl/2002/uk/uk.html
Web archiving resources - United States
Library of Congress Web Archiving
http://www.loc.gov/webarchiving/
MINERVA : Mapping the Internet Electronic Resources Virtual Archive
http://www.loc.gov/minerva/
Collections Policy Statement : Web Site Capture & Archiving
http://lcweb.loc.gov/acq/devpol/webarchive.html
Web Preservation Project Final Report : A Report to the Library of
Congress (2001) http://web.archive.org/web/20041106155137/http://www.loc.gov/minerva/webpresf.pdf
Preserving Our Digital Heritage : Plan for the National Digital
Information Infrastructure and Preservation Program
http://www.digitalpreservation.gov/library/resources/pubs/docs/ndiipp_plan.pdf
National Digital Information Infrastructure and Preservation Program
http://www.digitalpreservation.gov/
Editor's Interview : National Digital Information Infrastructure and
Preservation Program
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file3453.html#feature1 National
Digital Preservation Initiatives: An Overview of Developments in Australia,
France, the Netherlands, and the United Kingdom and of Related International
Activity http://www.clir.org/pubs/abstract/pub116abst.html
Web Preservation Projects at Library of Congress
http://www.archimuse.com/mw2003/papers/grotke/grotke.html
Collecting and Preserving the Web : the Minerva Prototype
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file1424.html#feature1
Building Thematic Web Collections: Challenges and Experiences from the
September 11 Web Archive and the Election 2002 Web Archive
http://bibnum.bnf.fr/ecdl/2003/index.html
Legal Issues Relating to the Archiving of Internet Resources in the UK, EU,
USA and Australia : A Study Undertaken for the JISC and Wellcome Trust
http://www.jisc.ac.uk/uploaded_documents/archiving_legal.pdf
Collaborative initiatives
Web archiving resources - IIPC
Netpreserve.org : International Internet Preservation Consortium
http://netpreserve.org/about/index.php
Web archiving resources - Internet Archive
Internet Archive http://www.archive.org/
Saving the Web http://bibnum.bnf.fr/ecdl/2002/ia/ia.html
Archiving the Internet (1996) http://web.archive.org/web/20030605113917re_/www.archive.org/ sciam_article.html
The Internet Archive, An Interview with Brewster Kahle (2002)
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file993.html#interview
Web archiving resources - NEDLIB
NEDLIB : Networked European Deposit Library
http://nedlib.kb.nl/
Collecting and Preserving the Web: Developing and Testing the NEDLIB Harvester
http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file1424.html#feature2
NEDLIB Local Situations
http://nedlib.kb.nl/results/local_situations_v2.htm
Long-Term Preservation of Electronic Publications: the NEDLIB Project
http://www.dlib.org/dlib/september99/vanderwerf/09vanderwerf.html
Setting up a Deposit System for Electronic Publications, the Nedlib
Guidelines http://nedlib.kb.nl/results/NEDLIBguidelines.pdf
Archiving the Web : European Experiences
http://www.lib.helsinki.fi/tietolinja/0203/webarchive.html
Web archiving resources - Nordic Web Archive
NWA : Nordic Web Archive http://web.archive.org/web/*/http://nwa.nb.no/ Nordic Web Archive
http://www.lib.helsinki.fi/tietolinja/0100/nwa.pdf
Nordic Web Archive http://bibnum.bnf.fr/ecdl/2003/index.html
Archiving the Web : European Experiences (2003)
http://www.lib.helsinki.fi/tietolinja/0203/webarchive.html
Articles
| Books
| Events
| Organisations and Websites
| Online tutorial
| Policies, Strategies & Guidelines
| Projects and Case Studies
| Bibliographies
| Glossaries
| Journals & Newsletters
| Discussion Lists
| Surveys
| Tools
| Evaluations
- 2007 Australian Federal Election on the Internet
Crook, Edgar
(Date Created: Dec 2007)
(Australia)
- This paper describes the challenge of archiving web sites for the Australian 2007 Federal Election in the PANDORA web archive.
- http://www.nla.gov.au/nla/staffpaper/2007/documents/Election2007.pdf
-
2nd ECDL Workshop on Web Archiving
Day, Michael
In: Cultivate Interactive
(Date Created: Nov 2002)
- A report from the 2nd ECDL Workshop on Web Archiving held in Rome on the 19th September 2002 organised by the Bibliothèque nationale de France (BnF) and the Vienna University of Technology.
- http://www.cultivate-int.org/issue8/ecdlws2/
-
3rd ECDL Workshop on Web Archiving
Day, Michael
In: Ariadne
(Date Created: 30 Oct 2003)
- A report on the 3rd ECDL Workshop on Web Archives held in Trondheim in August 2003, describing work presented on identifiers, web archiving tools and experiences in web archiving from various countries.
- http://www.ariadne.ac.uk/issue37/ecdl-web-archiving-rpt/
- 4th International Web Archiving Workshop
Day, Michael
(Date Created: 30 Oct 2004)
- The 4th International Web Archiving Workshop was held at the University of Bath in September as part of ECDL 2004. This workshop report features a paragraph summary of each paper.
- http://www.ariadne.ac.uk/issue41/ecdl-web-archiving-rpt/
- In Ariadne, Issue 4, October 2004
-
Access to Web Archives: the Nordic Web Archive Access Project
Brygfjeld, Svein Arne
(Date Created: 22 Aug 2002)
(Scandinavia)
- Presented at the 68th IFLA General Conference and Council held in Glasgow, 18-24th August, 2002, this paper describes a prototype access system for access to large-scale web archives, as developed by the Nordic Web Archive Access Project. The project is an initiative of the National Libraries of Denmark, Finland, Iceland, Norway and Sweden.
- http://archive.ifla.org/IV/ifla68/papers/090-163e.pdf
- Also available in French at http://archive.ifla.org/IV/ifla68/papers/090-163f.pdf
- Architecture of DAMP : a System for Harvesting and Archiving Web Publications
Milinovic, Miroslav ; Topolscak, Nebojsa
(Date Created: Jul 2005)
(Croatia)
- This article discusses the DAMP project (Digital Archive for Web Publications). DAMP is used to archive Croatian websites. It was developed by The University Computing Centre based at the University of Zagreb (SRCE) and the National and University Library (NUL) in Zagreb. The article discusses the first version of DAMP, including its architecture, the gathering process and future developments such as the instigation of an authentication and authorisation system.
- http://widwisawn.cdlr.strath.ac.uk/Issues/Vol3/issue3_3_1.html
- WIDWISAWN, Vol. 3, no. 3 (July 2005)
-

Archival Data Format Requirements
Christensen, Steen S.
(Date Created: Jul 2004)
(Denmark)
- This document discusses the requirements of the Netarkivet in terms of both file formats and metadata. The document outlines the basic requirements for storage formats that can be used for long term preservation. Different storage formats are then evaluated in relation to Netarkivet specifications and a recommendation for a suitable long term storage format is made.
- http://netarkivet.dk/publikationer/Archival_format_requirements-2004.p
df
-
Archiving and Accessing Web Pages : the Goddard Library Web Capture Project
Allen, Robert B. ; Anderson, Nikkia; Hodge, Gail; Senserini, Alessandro ; Smith, Daniel
In: D-Lib magazine
(Date Created: Nov 2004)
(United States of America)
- This is a report on capturing intranet web pages by the NASA Goddard Space Flight Center. The aim of the Web Capture project was to ensure the long term preservation of this information, and to extract and index metadata in a format that is searchable. The article discusses how this was achieved including descriptions of system flow, web site selection, and the capture process. Technical limitations of the process are also discussed.
- http://dx.doi.org/10.1045/november2004-hodge
- http://www.dlib.org/dlib/november04/hodge/11hodge.html
- Archiving Foreign Government Statistical Web Sites for All at Indiana University Libraries
Singer, Andrea
(Date Created: 26 May 2008)
(United States of America)
- This paper delivered at the World Library and Information Congress: 74th IFLA General Conference and Council (10-14 August 2008, Québec, Canada), focuses on a government information web site archiving project at Indiana University called National Government Statistical Web Sites (GIMSS). Web sites of national level governmental statistical agencies excluding the
European Union, Australia, Canada, and the USA are archived using Archive-It. Statistical agencies from countries in Africa, Asia, Europe, Latin America, the Middle East, Oceania, South Asia, and Southeast Asia are included, with collection comprising approximately 200 sites archived since June 2006.
- http://archive.ifla.org/IV/ifla74/papers/130-Singer-en.pdf
- Also available in French http://archive.ifla.org/IV/ifla74/papers/130-Singer-trans-fr.pdf
- Archiving Temporal Web Information: Organization of Web Contents for Fast Access and Compact Storage
JaJa, Joseph; Song, Sangchul
(Date Created: 2008)
- This technical report (11 p) address the problem of archiving dynamic web contents over significant time spans through the development of a scheme that stores unique temporal web contents in containers using the ARC/WARC format, and that provides quick access to the archived contents for arbitrary temporal queries.
- http://www.umiacs.umd.edu/~joseph/temporal-web-archiving-final-umiacs-
tr-2008-08.pdf
- Technical Report UMIACS-TR-2008-08
- Archiving the Canadian Web : Experiences at Library and Archives Canada
Lilleniit, Roselyn
(Date Created: Aug 2007)
(Canada)
- This article describes current practices for selection, acquisition, management and accessibility of Canadian web sites at Library and Archives Canada and outlines future plans.
- http://www.informaworld.com/smpp/content~content=a904273506~db=all~ord
er=page
- Available in The Serials Librarian, Vol 53, No 1/2, 20 Aug 2007, pp. 139-149
- Archiving the Greek Web
Eirinaki, Magdalini; Jevtuchova, Darija; Lampos, Charalampos; Vazirgiannis, Michalis
(Date Created: 2004)
(Greece)
- This paper describes a project to archive Greek orientated web materials using the domain name (.gr), the Greek language and Greek content as selection criteria. It describes the methodology for archiving, including details on the web crawler and semantic analysis, and presents the results of the experiments.
- http://www.iwaw.net/04/Lampos.pdf
- Archiving the Icelandic Web
Sigurðsson, Kristinn
(Date Created: 16 Sep 2006)
(Iceland)
- This presentation given at ICA-SUV Seminar in Reykjavik, Iceland on 16 September 2006 gives an update on web archiving in Iceland.
- http://www2.hi.is/solofile/1009995
-

Archiving the Internet
Kahle, Brewster
In: Report on the WWW 8 Conference
(Date Created: 11 Apr 1996)
- Brewster Kahle discusses his rationale for an Internet Archive of public documents. He gives insights into some of the logistical issues including those of data gathering, cost, access and collecting mandate.
It is intended that the Archive will provide resources for research and data analysis.
- http://web.archive.org/web/20030605113917re_/www.archive.org/sciam_art
icle.html
- Archiving the Slovenian Web : Recent Experiences
Grobelnik, Marko; Kavcic-Colic, Alenka
(Date Created: 2004)
(Slovenia)
- This paper reports on a two year project ending in September 2004 which aimed to develop a national repository for the long term preservation of the Slovenian web. The two main goals of the project were to develop a methodology to archive the Slovenian web and to develop a web crawler to facilitate this.
- http://www.iwaw.net/04/Kavcic.pdf
- This workshop was held in conjunction with the 8th European Conference on Research and Advanced Technologies for Digital Libraries, September 2004.
-
Archiving the Web - Some Legal Aspects
Kavcic-Colic, Alenka
(Date Created: 21 Aug 2002)
- A paper presented at the 68th IFLA General Conference and Council held in Glasgow, 18-24 August, 2002, discussing some legal aspects of archiving publications from the web. Copyright and legal deposit legislation are discussed with regard to rights implications for harvesting, public access and long-term preservation.
- http://archive.ifla.org/IV/ifla68/papers/116-163e.pdf
- Also available in French at http://archive.ifla.org/IV/ifla68/papers/116-163f.pdf
- Archiving the Web : Does Whole-of-Domain Archiving = Information Overload?
Pymm, Bob; Wallis, Jake
(Date Created: Jan 2009)
(Australia)
- This paper presented at Information Online 2009 (Sydney Convention Centre, 20-22 January 2009) presents a study comparing results of searching the whole of Australian domain harvest 2007 undertaken by the National Library of Australia in 2007 and selective archiving in the PANDORA web archive. The authors explore the question of the value of whole domain harvests compared to selective archiving.
- http://www.information-online.com.au/sb_clients/iog/data/content_item_
files/000001/PresentationB8.pdf
-
Archiving the Web : European Experiences
Hakala, Juha
In: Tietolinja News
(Date Created: Oct 2003)
- This paper was presented at CONSAL XII, 20-23 October 2003, Brunei, and is an updated version of Collecting and Preserving the Web: Developing and Testing the NEDLIB Harvester, published in 2001. It outlines the outcomes of the NEDLIB Harvester Project for the archiving of Web resources. Some of the key issues in using this form of technology for preserving materials on the Web are reviewed. New developments are discussed including the Nordic Web Archive II project and the establishment of the International Internet Preservation Consortium (IIPC) to develop standards and tools for web archiving.
- http://www.kansalliskirjasto.fi/extra/tietolinja/0203/webarchive.html
-
Archiving the Web : Political Party Web Sites in the Netherlands
Druiven, Henk; Keyzer, André; Voerman, Gerrit; den Hollander, Frank
(Date Created: Jun 2003)
(Netherlands)
- This article describes the Dutch project Archipol, which archives Dutch political Web sites, and examines why this information may be important to researchers in the future. Also discussed are various international initiatives to archive similar Web sites.
- http://iospress.metapress.com/content/dmqdln7vt3vp27b9/?p=17748463c0ce
482ea54b6a0106c89ae1&pi=0
- Available in Information Services and Use, vol. 23, issue 1 (2003), ISSN 0167-5265
-

Archiving the Web: The PANDORA Archive at the National Library of Australia
Cathro, Warwick; Webb, Colin; Whiting, Julie
(Date Created: Jun 2001)
(Australia)
- This paper describes the National Library of Australia's PANDORA web archive. It includes description of the Library's framework for digital collection management, the reasons for a selective approach to web archiving, the archiving processes, persistent identifiers, access control, preservation plans and opportunities for global cooperation and information sharing.
- http://www.nla.gov.au/nla/staffpaper/2001/cathro3.html
-

Archiving the World Wide Web
Lyman, Peter
In: Building a National Strategy for Digital Preservation: Issues in Digital Media Archiving
(Date Created: Apr 2002)
(United States of America)
- A paper in the series of six environmental scans commissioned as part of the US National Digital Information Infrastructure and Preservation Program (NDIIPP), this article examines archiving the web from several perspectives. As well as technical issues, such as those involved in defining the digital object, authenticity and provenance and the technologies needed for preservation, the article considers the roles of librarians and computer scientists in managing digital material and organisational issues such as copyright and the responsibility to preserve.
- http://www.clir.org/pubs/reports/pub106/web.html
- The collected papers are also available in PDF format via: http://www.clir.org/pubs/abstract/pub106abst.html
- Archiving Web Site Resources : a Records Management View
Kelly, Brian; Pennock, Marueen
(Date Created: 2006)
- This brief paper forms part of the poster exhibition at the 15th International World Wide Web Conference in 2006. The paper describes a records management approach to the archiving and maintenance of web sites so that enough metadata is collected on the resource to ensure legal and archival requirements. The authors propose several ways to facilitate this including collaboration between record creators and managers, education and training.
- http://www.dcc.ac.uk/docs/WWW2006_Archiving_Web_Site_Resources.pdf
- Poster presented at the WWW 2006 conference, Edinburgh Scotland
- Archiving Web Sites for Preservation and Access : MODS, METS and MINERVA
Guenther, Rebecca; Myrick, Leslie
(Date Created: Apr 2007)
(United States of America)
- This article examines the technical challenges in harvesting and managing web archives and metadata strategies using as an example the Library of Congress' Minerva project.
- http://www.informaworld.com/smpp/content~content=a902777368~db=all~ord
er=page
- Journal of Archival Organization, April 2007, Vol 4 No1-2, p 141-166 and through InformaWorld
-

Archiving Websites
Boudrez, Filip; Van den Eynde, Sofie
(Date Created: Jul 2002)
(Belgium)
- A report (95 p., 1.8 MB) from the Flemish DAVID Project examining the need for archiving of web sites, approaches for collection and management, and a range of legal issues including copyright. The report is available in PDF format.
- http://www.expertisecentrumdavid.be/davidproject/teksten/Rapporten/Rep
ort5.pdf
- Archiving, Indexing and Accessing Web Materials: Solutions for Large Amounts of Data
Cowart, Charles; Minor, David; Moore, Reagan; Zhu, Bing
(Date Created: 23 Jun 2007)
(United States of America)
- This paper given at IWAW 2007(7th International Web Archiving Workshop, Vancouver Canada, 23 June 2007) describes two different technologies applied by the San Diego Super Computer Centre for archiving, indexing and accessing web archived materials for two projects: Library of Congress/San Diego Supercomputer Center and the National Science Digital Library.
- http://iwaw.net/07/IWAW2007_minor.pdf
- Paper presented at IWAW 2007.
- Australian Web Domain Harvests: A Preliminary Quantitative Analysis of the Archive Data
Koerbin, Paul
(Date Created: 15 Apr 2008)
(Australia)
- This report provides a preliminary quantitative analysis of content from three Australian web domain harvests conducted between 2005 and 2007. It reports on the size of the Australian web harvests; number and percentage of hosts and URLs by 2nd level domain; MIME types and changes between the 2006 and 2007 harvests.
- http://pandora.nla.gov.au/documents/auscrawls.pdf
-

Austrian Online Archive Processing: Analyzing Archives of the World Wide Web
Aschenbrenner, Andreas; Rauber, Andreas; Witvoet, Oliver
(Date Created: Sep 2002)
(Austria)
- A paper describing the use of OLAP technologies (On-Line Analytical Processing) to build a data warehouse of a web archive. Data warehousing allows exploration and analysis of technical aspects of the archive, such as the use of operating systems, scripts and file types, from which inferences of change over time may be drawn.
- http://www.ifs.tuwien.ac.at/ifs/research/pub_pdf/rau_ecdl02.pdf
-

Best Practices for Digital Archiving: An Information Lifecycle Approach
Hodge, Gail
In: D-Lib magazine
(Date Created: Jan 2000)
- This report is based on a study commissioned by the International Council for Scientific and Technical Information (ICSTI), in March 1999 to highlight the importance of digital archiving (long-term storage, preservation and access to information that is "born digital"). Project managers from a selection of projects including web archives were surveyed about the methods they were using in their project regarding: Creation; Acquisition and Collection Development; Identification and Cataloguing; Storage; Preservation; Access.
- http://dx.doi.org/10.1045/january2000-hodge
- http://www.dlib.org/dlib/january00/01hodge.html
-

Blog Today, Gone Tomorrow? Preservation of Weblogs
Entlich, Richard
In: RLG DigiNews
(Date Created: 15 Aug 2004)
- Blogs (weblogs) are a recent development on the Internet and they are becoming more popular as a communication tool. The article outlines the nature and ephemeral life of weblogs and how content can be lost, and also looks at the nascent efforts to preserve them.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file1
789.html#article3
-
Brewster Kahle on the Internet Archive and People's Technology
Rein, Lisa
(Date Created: 22 Jan 2004)
(United States of America)
- In this interview, Brewster Kahle, the founder of the Internet Archive, discusses the philosophy and direction of the Archive. Also discussed are projects in collaboration with the Library of Congress, the implementation of new technologies and some common difficulties in archiving certain files and websites.
- http://www.openp2p.com/lpt/a/4563
-
Capturing the Basque Web
Marcos Maciá, Sonia; Pulgar Vernalte, Francisca
(Date Created: 25 May 2009)
(Spain)
- This paper given at LIDA 2009 (May 25-30 2009, Dubrovnik and Zadar, Croatia) describes the Basque Web capture project launched in 2007 by the Basque Government Department of Culture and the Basque Government IT services company. After describing the international context the authors go on to discuss the context, tools and selection policies of ONDARENET: the Basque web archive.
- http://eprints.rclis.org/16256/
- Available as PDF from E-Lis: eprints in Library and Information Science
-
Challenge of Web Site Records Preservation
Phillips, John T.
(Date Created: Jan 2003)
- This article explains the need for web site records management and retention, examines the challenges of web site records preservation management and discusses solutions to these problems. The article also describes studies and documents that attempt to quantify and describe the problems of web site preservation.
- http://www.arma.org/bookstore/productdetail.cfm?ProductID=1222
- Information Management Journal, (ISSN 1535-2897), Vol. 37, no. 1, January/February 2003 and by subscription online at www.ebscohost.com
- Characteristics of .au Websites: An Analysis of Large-Scale Web Crawl Data from 2005
Ackland, Robert; Bailey, Peter; Spink, Amanda
(Date Created: 02 Jul 2007)
(Australia)
- This paper provides a preliminary analysis of .au domain websites collected by the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in 2005. It is a first step in the planned ".au Census" project.
- http://ausweb.scu.edu.au/aw07/papers/refereed/spink/paper.html
- Paper presented at AusWeb07, the thirteenth Australasian World Wide Web Conference, Coffs Harbour NSW 30 June - 4 July 2007
-

Collecting and Managing Web Resources for Long-Term Access : Web Harvesting and Guidelines to Support Preservation (ICABS Actions 3.3 and 3.4)
Gatenby, Pam
(Date Created: Aug 2004)
(Australia)
- This paper outlining the National Library of Australia’s recent contributions to the advancement of understanding of international Web archiving issues, was presented at the 70th IFLA Conference in August 2004. The NLA’s work on web harvesting and the preservation of digital materials as part of its ICABS responsibilities are detailed, together with its involvement in the International Internet Preservation Consortium’s (IIPC) Deep Web Working Group.
- http://archive.ifla.org/IV/ifla70/papers/026e-Gatenby.pdf
-

Collecting and Preserving the Web : Developing and Testing the NEDLIB Harvester
Hakala, Juha
In: RLG DigiNews
(Date Created: Apr 2001)
- This article by Juha Hakala outlines the outcomes of the NEDLIB Harvester Project for the archiving of Web resources. Some of the key issues in using this form of technology for preserving materials on the Web are reviewed.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file1
424.html#feature2
-

Collecting and Preserving the Web : The Minerva Prototype
Arms, William Y.
In: RLG DigiNews
(Date Created: Apr 2001)
(United States of America)
- This article, by William Arms (Cornell University), Roger Adkins, Cassy Ammen and Allene Hayes (Library of Congress), describes the Library of Congress' Minerva Project (Mapping the INternet the Electronic Resources Virtual Archive). The main features of Minerva are described, and suggestions for a full-scale preservation system explored.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file1
424.html#feature1
-

Collecting and Preserving the World Wide Web : a Feasibility Study Undertaken for the JISC and Wellcome Trust
Day, Michael
(Date Created: 25 Feb 2003)
(United Kingdom)
- The study (91 p.) is an analysis of the task of archiving the web, and includes a review of current web archiving activities and approaches. One of the major recommendations of this study is to establish a pilot medical web archiving project using the National Library of Australia's PANDAS software.
- http://www.jisc.ac.uk/uploaded_documents/archiving_feasibility.pdf
-
Collecting, Indexing and Archiving Net Publications
Schwens, Ute
(Last Updated: 12 Jun 2006)
(Germany)
- These pages outlined the legal mandate and deposit arrangements of the Die Deutsche Bibliothek (DDB) for acquiring digital publications, both for physical formats such as DVDs and for online publications such as websites and electronic periodicals prior to the introduction of the new law which came into effect on 22 June 2006. The current situation is explained on the later web pages about online publications.
- http://web.archive.org/web/20070128202450/http://www.d-nb.de/eng/wir/u
eber_dnb/netzpubl.htm
-
Collection of Swedish web pages at the Royal Library - The Web Heritage of Sweden
Arvidson, Allan
(Date Created: 22 Aug 2002)
(Sweden)
- A paper presented by Allan Arvidson of the Royal Library of Sweden at the 68th IFLA Council and General Conference, Glasgow, 18-24th August, 2002, discussing the evolution of the Swedish web since the Royal Library began harvesting "snapshots" in 1997. The paper questions whether a true picture of Sweden is achieved, from both geographical and technical viewpoints, and comments on the possible influences of future web technologies.
- http://archive.ifla.org/IV/ifla68/papers/111-163e.pdf
-

Concerning Etags and Datestamps
Clausen, Lars R.
(Date Created: Jul 2004)
(Denmark)
- In order to avoid archiving of many copies of the same content, this paper reports on an investigation of a scheme for reliably predicting whether the content has changed, without having to download it. This involved an investigation of the reliability of the Etag and Last-Modified-Date fields of HTTP-headers as indicators of changed content. The investigation examined the front pages of 361,408 websites on the .dk-domain, which were harvested every other night over a period of one month. A checksum of the content was then compared to the Etag and Last-Modified Date fields. This comparison was used for calculating the reliability and usefulness of these fields in determining content changes.
- http://netarkivet.dk/publikationer/Etags-2004.pdf
- Conference report : Archiving Web Resources International Conference: Issues for Cultural Heritage Organisations
Phillips, Margaret E.
In: RLG DigiNews
(Date Created: 15 Feb 2005)
- This article reports on the Archiving Web Resources International conference which was held on 9-11 November 2004 at the National Library of Australia in Canberra. The main themes and papers of the conference are summarised.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file1
298.html#article2
- Creating Preservation-Ready Web Resources
Smith, Joan A.; Nelson, Michael
In: D-Lib magazine
(Date Created: Jan 2008)
(United States of America)
- This article describes a simple model for archiving "everyday websites" at the time of dissemination using the web server to prepare the site's resources for preservation.
- http://www.dlib.org/dlib/january08/smith/01smith.html
-
CRL Political Communications Archiving Investigation
D-Lib Magazine
In: D-Lib magazine
(Date Created: Jan 2003)
- A brief overview of a Center for Research Libraries (CRL) project on developing methodological, conceptual and organisational frameworks for the systematic, sustainable preservation of political Web materials.
- http://www.dlib.org/dlib/january03/01inbrief.html#REILLY
- Current Issues in Web Archiving in Australia
Koerbin, Paul
(Date Created: Jul 2005)
(Australia)
- This paper discusses the curatorial, technical and legal issues currently problematic for web archiving activities by PANDORA at the National Library of Australia.
- http://www.nla.gov.au/nla/staffpaper/2005/koerbin1.html
- Paper given at the Open Publish Conference, Sydney, Australia, 27-29 July 2005.
- Czech Web Archive Analysis
čabiška, P.; Matĕjka, L
(Date Created: 2007)
(Czech Republic)
- This paper reviews the current activities of WevArchiv, a project of the National Library in Prague. It also considers strategies that could help to reduce costs.
- http://www.informaworld.com/smpp/content~content=a780451540~db=all~ord
er=page
- Available in New Review of Hypermedia & Multimedia 2007, v. 13 no 1 pp27-37
- Data Quality in Web Archiving
Spaniol, Marc; Senellart, Pierre; Denev, Dimitar; Mazeika, Aturas; Weikum, Gerhard
(Date Created: 20 Apr 2009)
- This paper given at WICOW'09 (3rd Workshop on Information Credibility on the Web, April 20 2009, Madrid Spain) provides an overview of strategies that help to overcome, or at least identify the temporal diffusion of Web crawls that last from a few hours to several days.The paper introduces a model for identifying coherent sections of a website and thus measuring data quality and a crawling strategy for minimizing Web site captures.
- http://www.dl.kuis.kyoto-u.ac.jp/wicow3/papers/p19-spaniolA.pdf
- DCC Digital Curation Manual : Instalment on "Archiving Web Resources"
Thompson, Dave
(Date Created: Dec 2008)
(United Kingdom)
- This instalment of the DCC Digital Curation Manual prepared by Dave Thompson of the Wellcome Trust provides an introduction to web archiving and digital curation. Points covered include Automation of harvesting, Deposit approaches, Selection, negotiation and capture, Issues associated with the "deep" web, Existing initiatives (e.g. Internet Archive, NWA, PANDAS), Legal implications, Collaboration and responsibility and Non standard media types.
- http://www.dcc.ac.uk/resource/curation-manual/chapters/archiving-web-r
esources/
-
Deposit for Digital Collections
Borbhina, Jose; Campos, Joao P.; Gomes, Daniel; Noronha, Norman; Silva, Mario J
(Date Created: Sep 2001)
(Portugal)
- This paper given at ECDL 2001 (Research and Advanced Technology for Digital Libraries , 5th European Conference, Darmstadt, Germany, September 4-9, 2001) describes an architecture for a system which manages the workflow for deposit of digital publications from the Web, developed by Faculty of Science, University of Lisbon in collaboration with the National library of Portugal as part of the NEDLIB project. Two subsystems Digital Deposit and PURL.PT are described and an evaluation after a trial harvest is presented.
- http://xldb.fc.ul.pt/xldb/publications/paper_ecdl01.pdf
- Also available Springer-Verlag, Lecture Notes in Computer Science Vol. 2163/2001, pp 200-212.
- Design and Selection Criteria for a National Web Archive
Freitas, Sergio; Gomes, Daniel; Silva, Mario J.
(Date Created: Sep 2006)
(Portugal)
- This paper presented at ECDL 2006 conference in September, describes the architecture of the Tomba prototype. The paper also discusses strategies to populate a web archive, selection, and access. Tomba : Portuguese Web Archive was developed by the Faculdade de Ciências, Universidade de Lisboa and has been crawling the Portuguese web since 2002.
- http://xldb.fc.ul.pt/daniel/docs/papers/gomes06tomba.pdf
- Available in Research and Advanced Technology for Digital Libraries, pp 196-207 http://www.springerlink.com/content/e313463036m04246/
- Developing National Taiwan University Web Archiving System
Chen, Kuang-hua ; Chen, Yen-liang ; Ting, Peng-fung
(Date Created: Sep 2008)
(Taiwan)
- This paper given at IWAW 2008 (International Workshop on Web Archiving, Aarhus Denmark 18-19 Sept, 2008) discusses the development and implementation of the National Taiwan University Web Archiving System (NTUWAS), which was developed by the National Taiwan University Library and also gives a brief survey of international web archiving activities.
- http://iwaw.net/08/IWAW2008-Chen.pdf
-

Digital Archiving
National Library of Australia
(Regularly Updated)
(Australia)
- A Web page summarising the National Library of Australia's digital archiving initiatives. Links are provided to key documents which describe the various components of the program.
- http://www.nla.gov.au/initiatives/digarch.html
-

Digital Archiving- Developing Policy and Best Practice Guidelines at The National Library of Australia
Gatenby, P.
(Date Created: 30 Jan 2000)
(Australia)
- This paper looks at the National Library of Australia's business model for online publications. There is a focus on PANDORA including selection, collection, metadata and cataloguing.
- http://web.archive.org/web/20010303103129/www.icsti.org/icsti/2000work
shop/gatenby.html
- Digital Cultural Heritage and the Cooperation of National Memory Institutes
Žabička, Petr
(Date Created: May 2006)
(Czech Republic)
- This paper discusses and current and future developments of the WebArchiv, a project to selectively archive the Czech web cultural heritage. It outlines the acquisition and access process, the hardware and software infrastruce and aspects of national and international cooperation for the project.
- http://en.webarchiv.cz/files/dokumenty/konference/Zabickabanskabystric
a2006.doc
- Paper presented at the the 1st Colloquium of Library Information Employees of the V4+ Countries. State Science Library, Banská Bystrica, Slovakia, 11 - 13 May, 2006. Libraries in the Knowledge Society
- Digital Preservation using the WARC file format
Kristiansen, Mads Alhof
(Date Created: 30 Jul 2006)
(Denmark)
- This paper was prepared as a graduate student project in collaboration with the Danish Royal Library (Det Kongelige Bibliotek) and has been published as a technical paper at University of Copenhagen, Department of Computer Science. It considers the potential of WARC as an archival medium through small scale experiments testing the preservation strategies of migration, emulation and durable encoding using the case study of the Royal Library's Digital Object Management System.
- ftp://ftp.diku.dk/diku/semantics/papers/D-548.pdf
- Technical paper 2006 D-548, TOPPS, DIKU (Datalogisk Institut på Københavns Universitet)
-

Digital Preservation: The Australian Experience
Woodyard, Deborah
(Date Created: Oct 2000)
(Australia)
- This paper, presented at the 3rd Digital Library Conference: Positioning the Fountain of Knowledge, discusses how and what is being digitally preserved in Australia. Current projects include standardizing metadata and preserving audio collections and electronic publications. Cooperative projects, such as PANDORA and PADI are two of the projects mentioned.
- http://www.nla.gov.au/nla/staffpaper/dw001004.html
- DK Domain : in Words and Figures
Andersen, Bjarne
(Date Created: Feb 2006)
(Denmark)
- In July 2005, new legal deposit laws in Demark gave the Royal Library and the State and University Library the mandate to gather and preserve the Danish domain of the Internet. This paper discusses the initial cross section harvesting that occurred from July to October 2005. Also discussed are the technical specifications of the harvest, harvest limitations, statistics on file types and the size of Danish websites.
- http://netarkivet.dk/publikationer/DFrevy_english.pdf
-

Dominos : a New Web Crawler's Design
Djeraba, Chabane; Hafri, Younès
(Date Created: Sep 2004)
- This article describes 'Dominos', a high performance crawler developed as a real-time distributed system running on a cluster of machines in order to facilitate the French Web legal deposit. Various general web crawling strategies are described, including repetitive crawling, targeted crawling, and deep web crawling and the differences between Dominos and other web crawlers are outlined. A detailed technical picture of the functions and processes of the Dominos crawler and results from performance testing are included.
- http://www.iwaw.net/04/Hafri.pdf
- This workshop was held in conjunction with the 8th European Conference on Research and Advanced Technologies for Digital Libraries, September 2004
-

DPC Forum - Web-archiving: Managing and Archiving Online Documents and Records
Joint Information Systems Committee Distributed National Electronic Resource (JISC/DNER)
(Last Updated: 25 Apr 2002)
(United Kingdom)
- Organised by the Digital Preservation Coalition (DPC) and held at the Institution of Professionals, Managers and Specialists, London, on 25 March, 2002, this event brought together key institutions involved in web-archiving, in order to share experience and to highlight issues in archiving and long-term preservation. Links to the summary report of the meeting and presenters' notes are provided.
- http://www.dpconline.org/graphics/events/webforum.html
-
ECDL 2001 Workshop - What's next for Digital Deposit Libraries? Preserving Online Content for Future Generations - Papers
Lupovici, Catherine; Masanès, Julien
(Date Created: 26 Sep 2001)
(France)
- This web site contains links to presentations given at the ECDL 2001 Workshop on September 8, 2001 in Darmstadt, Germany. Topics focus on legal deposit and the internet, web harvesting and archiving. Papers in Powerpoint format include - Austrian On-Line Archive : Current Status and Next Steps ; Harvesting the Finnish Web Space : Practical Experiences ; BnF's Project for Web Archiving ; Collection of German Online Resources by Die Deutsche Bibliothek ; Kulturarw the Swedish WWW Archive.
- http://bibnum.bnf.fr/ecdl/2001/index.html
-
ECDL 2002 Workshop Report: Web Archiving
Masanès, Julien; Rauber, Andreas
(Date Created: Oct 2002)
- A brief report on the 2nd ECDL Workshop on Web Archiving, held in Rome on 19 September, 2002, in conjunction with ECDL 2002. The Workshop had three sessions dealing with technical issues and strategies, the progress of several web archiving projects, and two international consortia initiatives in web archiving.
- http://www.dlib.org/dlib/october02/10inbrief.html#RAUBER
- Electronic Publication: Problems of Archiving and Access to Archived Information: Legal Deposit, Data Protection and Related Topics
von Hielmcrone, Harald
(Date Created: 16 Jul 2008)
(Denmark)
- This paper given at World Library and Information Congress: 74th IFLA General Conference and Council (10-14 August 2008, Québec, Canada) discusses the challenges for libraries of electronic publications. It focuses on the situation in Denmark looking in particular at the effect of the revision of the Legal Deposit Act and the legal aspects of web harvesting.
- http://archive.ifla.org/IV/ifla74/papers/087-Hielmcrone-en.pdf
-
Ephemeral to Enduring : The Internet Archive and Its Role in Preserving Digital Media
Edwards, Eli
(Date Created: 2004)
(United States of America)
- This article focuses on the role and functions of the Internet Archive (IA) in the USA. As well as providing background information on the development of the Archive, the article looks at the Archive's several website collections as well as gaps in the collection, the role of the IA in relation to copyright and compares and contrasts the IA's approach to archiving with internet and archiving projects undertaken by national libraries around the world.
- http://www.ala.org/ala/mgrps/divs/lita/ital/volume23a/number1a/v23n1.c
fm#edwards
- Also available in Information Technology and Libraries, Vol 23, No 1, March 2004, pp. 3-8, ISSN 07309295
- Erfaringer med høstning af det danske net 2005-2008
Jacobsen, Grethe
(Date Created: Dec 2008)
(Denmark)
- This article (in Danish) provides an update on the activities of Netarkivet.dk and its efforts to archive the Danish Web. Included are statistics comparing the 2005 and 2008 crawls.
- http://ej.lib.cbs.dk/index.php/dfrevy/article/viewFile/2291/2276
- DF Revy, ISSN 0106-0503, No. 8, December 2008, pp12-15
- Ethical Issues in Web Archive Creation and Usage - Towards a Research Agenda
Kaiser, Max; Rauber, Andreas; Wachter, Bernhard
(Date Created: Sep 2008)
- This paper delivered at IWAW 2008, discusses key ethical concerns raised about web archiving and proposes directions to pro-actively address the concerns particularly through IT methods.
- http://iwaw.net/08/IWAW2008-Rauber.pdf
-

EVA : The Acquisition and Archiving of Electronic Network Publications In Finland
Lounamaa, Kirsti; Salonharju, Inkeri
In: Tietolinja News
(Date Created: 15 Jan 1999)
(Finland)
- This article presents the current status of a project to capture and preserve electronic documents published in the Finnish Internet. The article describes the policy used to select the material to be included in the collection, the capturing process, the storage technology and accessing method. Electronic documents are not yet a part of the Legal Deposit in Finland, but the revision of the Act is going on and the aim is to also preserve electronic publications for the coming generations.
- http://www.lib.helsinki.fi/tietolinja/0199/evaart.html
- Tietolinja News 1/1999
- Fast Browsing of Archived Web Contents
JaJa, Joseph; Song, Sangchul
(Date Created: Sep 2008)
- This paper given at IWAW 2008 (International Workshop on Web Archiving, Aarhus Denmark 18-19 Sept, 2008) considers the problem of archiving a linked set of web objects into web containers in such a way as to minimize the number of containers accessed during a browsing session. The paper describes the development of a methodology and simulation results which are compared to the common scheme used to organize web objects.
- http://iwaw.net/08/IWAW2008-Song.pdf
- Finding and Archiving the Internet Footprint
Cox, David; Garfinkel, Simson
(Date Created: 10 Feb 2009)
- This paper given at the First Digital Lives Conference (British Library Conference Centre, London, 9-11 February 2009) explores the range of information that an originator may have left on computers that form part of the Internet. It discusses methods for finding the content and legal and ethical issues faced by archivists.
- http://simson.net/clips/academic/2009.BL.InternetFootprint.pdf
- Finland's Internet Archive
Jansson, Leena
(Date Created: Aug 2009)
(Finland)
- This article briefly describes Finland's Internet Archive which was opened by the National Library of Finland on 2 April 2009. It includes information on the Finnish web archive's background, content and collection, use of the archive, its users and usability.
- http://www.kansalliskirjasto.fi/extra/bulletin/article4.html
- The National Library of Finland Bulletin 2009
-
First Experience in Archiving the French Web
Abiteboul, Serge; Cobéna, Gregory; Masanès, Julien; Sedrati, Gerald
(Date Created: Sep 2002)
(France)
- This paper describes preliminary work and experiments by the Bibliothèque nationale de France and INRIA in archiving the French web under legal deposit legislation. Defining the perimeter of the French web and versioning issues are also discussed.
- ftp://ftp.inria.fr/INRIA/Projects/gemo/gemo/GemoReport-229.pdf
- Formal Analysis of Recovery in a Preservational Data Grid
Christensen, Niels H.
(Date Created: Jun 2006)
(Denmark)
- This paper describes the processes of data recovery in the Netarkivet data grid. Netarkivet is the Danish national web archive. The mathematical model includes five recovery operations and scenarios are developed which illustrate the use of these operations.
- http://netarchive.dk/publikationer/nhc-kb-dk-msst2006.pdf
- Paper presented at the Conference on Mass Storage Systems and Technologies, 15-18 May 2006
- Future Proofing Web Sites
Pennock, Maureen
In: Ariadne
(Date Created: 08 Feb 2006)
- This is a report on a two-day workshop on Future-Proofing Web Sites, organised by the Digital Curation Centre (DCC) and the Wellcome Library at the Wellcome Library, London, over 19-20 January 2006. The report covers practical approaches to future proofing web sites, tools and current archiving activities and international activities and legislation.
- http://www.ariadne.ac.uk/issue46/dcc-fpw-rpt/
- Government Web Content in Canada a National Library Web Archive Perspective
Cantello, Gillian; Stegenga, John
(Date Created: 27 May 2008)
(Canada)
- This paper delivered at the World Library and Information Congress: 74th IFLA General Conference and Council (10-14 August 2008, Québec, Canada) discusses web harvesting of government produced information by Library and Archives Canada and its relationship to legal deposit which was broadened to include online publications in early 2007.
- http://archive.ifla.org/IV/ifla74/papers/130-Cantello_Stegenga-en.pdf
- Also available in French http://archive.ifla.org/IV/ifla74/papers/130-Cantello_Stegenga-trans-fr.pdf
-

Handling File Formats
Clausen, Lars. R.
(Date Created: May 2004)
(Denmark)
- This report describes a part of “Internetbevaringsprojektet” (the Internet preservation project), which is a joint project between Statsbiblioteket (The State Library) and Det Kongelige Bibliotek (The Royal Library) in Denmark. The aim of the project is to archive the Danish part of the Internet. As part of this project, this report specifically deals with strategies in handling file formats to ensure long term preservation and accessibility. Some of the main ways to preserve access to information stored as digital objects are discussed, including capture, sequential conversion to new formats, conversion on demand, emulation and hardware preservation. The report concludes with a suggested strategy for dealing with file formats in an archival system, file preservation workflows and general recommendations for handling file formats.
- http://netarchive.dk/publikationer/FileFormats-2004.pdf
- Harvester Results in a Digital Preservation System
Steinke, Tobias
(Date Created: 29 Sep 2008)
(Germany)
- This paper given at IPRES 2008 (29 - 30 Sep 2008, British Library Conference Centre), compares existing approaches to use METS and Web harvesting results in archival systems. It describes the advantages and disadvantages of treating Web harvests like other digital publications in dedicated preservation systems.
A side by side comparison of containers based on METS and the possibilities for WARC are made.
- http://www.bl.uk/ipres2008/presentations_day1/25_Steinke.pdf
-
Harvesting and Archiving of Electronic Resources in Lithuania : Towards Virtual Library
Jodelis, Remigijus
(Date Created: 2003)
(Lithuania)
- In November 2002 the National Library of Lithuania started the LIBIS Electronic Resources Subsystem, consisting of NEDLIB harvester and supplementary modules in LIBIS (the Lithuanian Integrated Library Information System), designated to create the Archive of Electronic Resources. This paper gives an overview of the project, types of resources archived, harvester problems and future directions.
- http://www.inforum.cz/inforum2003/prispevky/Jodelis_Remigijus.pdf
- Paper presented at Inforum 2003 : 9th Conference on Professional Information Resources
- Harvesting the Danish Internet - the First Two Years
Jacobsen, Grethe
(Date Created: 02 May 2007)
(Denmark)
- This paper describes experiences related to harvesting the Danish Internet in the first two years of the netarchive.dk project.
- http://netarchive.dk/publikationer/CollectingTheDanishInternet_2007.pd
f
- Harvesting the Web, Preserving Chinese Voices : the Digital Archive for Chinese Studies in Heidelberg
Gross, Jennifer; Wagner, Rudolf G.
(Date Created: Nov 2004)
(China)
- This paper was given at the International Conference on Sinological Resources in the Digital Era, Taipei, December 7-9, 2004. It provides detailed information on the origins, technical infrastructure and format, copyright issues and the collection development focus of DACHS (Digital Archive for Chinese Studies). Future plans and current usability problems are also discussed.
- http://www.sino.uni-heidelberg.de/dachs/publications/taipei2004.rtf
- Historical Infrastructures for Web Archiving: Annotation of Ephemeral Collections for Researchers and Cultural Heritage Institutions
Dougherty, Meghan; van den Heuvel, Charles
(Date Created: Apr 2009)
- This paper given at MiT6: (Media in Transition International Conference Stone and Papyrus Storage and Transmission, April 24-26 2009, Cambridge, Mass.) considers the gaps in the requirements of, and development of access tools for the creators and users of web archives. The authors suggest that there is a need for discovery and enrichment tools in addition to retrieval tools to enhance access and usability. Institutions and researchers will need to work together to develop tools and to understand each others ongoing requirements. Annotation as a way of adding value is explored as an historically used method which has potential application to web archives.
- http://web.mit.edu/comm-forum/mit6/papers/Dougherty_Heuvel.pdf
- Identification and Archiving of the Czech Web Outside the National Domain
Vlcek, Ivan
(Date Created: Sep 2008)
(Czech Republic)
- This paper delivered at IWAW '08 (September 18–19, 2008, Aarhus, Denmark) presents work supporting the comprehensive archiving of the Czech Web by the WebArchiv Project through the integration of WebAnalyzer in to the Heritrix Crawler. The paper describes the functionality and testing of WebAnalyzer and harvesting of Czech websites in other countries.
- http://iwaw.net/08/IWAW2008-Vlcek.pdf
- Integration of Non-harvested Web Data into an Existing Web Archive
Andersen, Bjarne
(Date Created: Nov 2007)
(Denmark)
- This paper describes a software prototype for transforming non-harvested web data into ARC-files. It provides an analysis of the problems associated with a test conducted on real data from a Danish web site and use of the Wayback tool to index the data.
- http://netarchive.dk/publikationer/IntegrationOfNonHarvestedData.pdf
-

Internet Archive : an Interview with Brewster Kahle
Kahle, Brewster
In: RLG DigiNews
(Date Created: 15 Jun 2002)
- Brewster Kahle discusses the organisational infrastructure, policies and processes of the Internet Archive and the Wayback Machine and touches on various aspects of technical, financial and legal issues in digital archiving.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file9
93.html#interview
- Introducing the Portuguese Web Archive Initiative
Costa, Miguel; Gomes, Daniel; Miranda, João; Nogueira, André
(Date Created: Sep 2008)
(Portugal)
- This paper delivered at IWAW '08 introduces the Portuguese Web Archive initiative which began in January 2008. It describes the project's infrastructure and characterization of the Portuguese Web.
- http://iwaw.net/08/IWAW2008-Gomes.pdf
-

Introduction to Heritrix : an Open Source Archival Quality Web Crawler
Avery, Dan; Kimpton, Michele; Mohr, Gordon; Ranitovic, Igor; Stack, Michael
(Date Created: 2004)
- This article provides an introduction to Heritrix, the open source archival web crawler developed by the Internet Archive. Major components of the crawler are described, its features and limitations are discussed and its performance is evaluated. Future plans, including collaboration with other institutions, are also noted.
- http://www.iwaw.net/04/Mohr.pdf
- This workshop was held in conjunction with the 8th European Conference on Research and Advanced Technologies for Digital Libraries, September 2004
- Just in Time Recovery of Web Pages
Harrison, Terry L.; Nelson, Michael L.
(Date Created: Sep 2006)
- This paper was presented at the HYPERTEXT 2006 conference held at Odense, Denmark on 23-25 August 2006. This paper describes Opal, a framework for locating missing (404) web pages using web archives, search engines, and other research projects. The paper covers the architecture of the Opal framework, the implementation of the framework, and a quantitative analysis of the framework that indicates the strong potential of Opal as an effective way to locate missing web pages.
- http://www.cs.odu.edu/~mln/pubs/ht06/ht10-harrison.pdf
-
Kulturarw3 Project - The Royal Swedish Web Archiw3e - An example of "complete" collection of web pages
Arvidson, Allan; Mannerheim, Johan; Persson, Krister
(Date Created: Aug 2000)
(Sweden)
- This paper describes the Royal Library of Sweden's project Kulturarw3 to test methods of collecting, preserving and providing access to Swedish electronic online documents.
- http://archive.ifla.org/IV/ifla66/papers/154-157e.htm
-

Language Engineering Techniques for Web Archiving
Coch, Jose; Masanès, Julien
(Date Created: 2004)
(France)
- This article describes the WATSON project and the use of language engineering techniques for facilitating both web archiving and archive mining by researchers. These techniques include pre-filtering of less important sites, categorisation of sites and characterisation of sites via a WATSON working station, which shows key words, key sentences and metadata information.
- http://www.iwaw.net/04/Coch.pdf
- This workshop was held in conjunction with the 8th European Conference on Research and Advanced Technologies for Digital Libraries, September 2004
-

Last Page of the Internet : the Importance of Preserving the Dynamic Aspects of the Internet
Brugger, Niels
(Date Created: Jun 2001)
- This paper, presented at the Preserving the Present for the Future : Strategies for the Internet conference, discusses the complications involved in the preservation of the dynamic features of the internet, as identified by media scholar Niels Brugger.
- http://www.deflink.dk/upload/doc_filer/doc_alle/1023_NBR.doc
- Lazy Preservation : Reconstructing Websites by Crawling the Crawlers
Bollen, Johan; McCown, Frank; Nelson, Michael L. ; Smith, Joan A.
(Date Created: Sep 2006)
(United States of America)
- This paper was presented at the Workshop on Web Information and Data Management (WIDM 2006) in November 2006. The paper describes the results of two experiments, the caching behavior of search engines and reconstructing websites using the Warrick web-repository crawler, which was developed at Old Dominion University. The authors stress that this approach should not be a substitute for a digital preservation infrastruture and policy but could be used as a last resort approach.
- http://www.cs.odu.edu/~fmccown/pubs/lazyp-widm06.pdf
- Lecture Meeting and Discussion "Present and Future Prospects of Web Archiving - For International Partnerships"
National Diet Library (Japan)
(Date Created: Apr 2008)
(Japan)
- This article reports on a meeting held at the National Diet Library in Japan on 23 January 2008. Speakers included Julien Masanes (European Archive), Kris Carpenter (Internet Archive), Dr Masaru Kitsuregawa (Univ. of Tokyo). A link is provided to slides from the lecture.
- http://www.ndl.go.jp/en/publication/ndl_newsletter/160/602.html
- National Diet Library Newsletter April 2008, no. 160.
- Legal Aspects of Web Archiving from a Dutch Perspective : Report Commissioned by the National Library in the Hague
Buenen, Annemarie; Schiphof, Tjeerd
(Date Created: 04 Oct 2006)
(Netherlands)
- This report commissioned by the Koninklijke Bibliotheek provides an overview of intellectual property rights in relation to web archiving in the Netherlands.
- http://www.kb.nl/hrd/dd/dd_projecten/webarchivering/documenten/KB_Lega
l_Aspects_WebArchiving_EN.pdf
- Legal Deposit and Collection Development in a Digital World
Joint, Nicholas
(Date Created: 2006)
(United Kingdom)
- This editorial article compares and contrasts management of national collections of hard copy and digital deposit materials. The author argues that while print collections are comparatively comprehensive, selective collecting of digital material requires intelligent selection to be meaningful .
- http://dx.doi.org/10.1108/00242530610689310
- http://www.emeraldinsight.com/Insight/viewContentItem.do?contentType=A
rticle&contentId=1571426
- Available in HTML and PDF. Published in Library Review 2006 Vol 55 no 8 pp.468-473
-

Legal Deposit from the Internet in Denmark : Experiences with the Law from 1997 and the Need for Adjustments
Henriksen, Birgit
(Date Created: Jun 2001)
(Denmark)
- This paper, presented at the Preserving the Present for the Future : Strategies for the Internet conference, reviews the 1997 amendments to Danish legal deposit legislation to include static internet publications. The author examines the limitations of the updated statute for the collection of dynamic web resources, and argues for the further extension of the law to allow for harvesting of a broader representation of the Danish web.
- http://www.deflink.dk/upload/doc_filer/doc_alle/1023_BNH.doc
-

Legal Deposit In Denmark : The New Law and Electronic Products
Dupont, Henrik, Det Kongelige Bibliothek
(Date Created: 1999)
(Denmark)
- This article, originally published in LIBER Quarterly, details the provisions of the Act on Copyright Deposit of Published Works, which came into effect in Denmark on 1 January 1998. The amendments change the emphasis of the legislation from printed to published works, regardless of the medium in which they are produced. Importantly, the law extends to physical format digital materials, and to those published Internet materials which are issued in a static and final form.
- http://liber-maps.kb.nl/articles/dupont11.htm
-

Legal Deposit of Digital Materials
Oltmans, Erik
(Date Created: Jun 2003)
(Netherlands)
- This is the text of a paper delivered at the LIBER conference in Rome, June 2003. Using the Digital Information Archiving System (DIAS) developed jointly with IBM, the Koninklijke Bibliotheek (KB) implemented electronic deposit (the 'e-Depot') in 2002. This paper explains the fully operational e-Depot at the KB with a focus on the data flow of processing deposited digital publications and the management of long-term preservation aspects of the stored digital objects.
- http://www.kb.nl/hrd/dd/dd_links_en_publicaties/publicaties/LIBER-Oltm
ans-input.doc
- Legal Deposit of the French Web: Harvesting Strategies for a National Domain
Lasfargues, France; Oury, Clément; Wendland, Bert
(Date Created: Sep 2008)
(France)
- This paper delivered at IWAW '08 (September 18–19, 2008, Aarhus, Denmark) describes the Bibliothèque nationale de France (BnF) web harvesting activities in the context of the French Copyright Law August 1 2006, which extended legal deposit to the Internet. It compares the .fr domain harvests conducted over the last four years and discusses collaboration with the Internet Archive.
- http://iwaw.net/08/IWAW2008-Lasfargues.pdf
-

Legal Issues Relating to the Archiving of Internet Resources in the UK, EU, USA and Australia : a Study Undertaken for the JISC and Wellcome Trust
Charlesworth, Andrew
(Date Created: 25 Feb 2003)
- Key legal issues such as copyright, data protection and defamation are discussed with emphasis on their application to the United Kingdom legal framework. The article also examines other approaches to web archiving, including several EU countries, the US and Australia. EU experience suggests that establishing legal deposit is highly desirable, US experience reveals that pragmatic approach is successful, but potentially full of legal controversies and Australian licensing approach although limited in its ability to deal with general harvesting of the web provides an acceptable degree of legal risk and allows selective archiving of both ‘shallow’ and ‘deep’ web resources. Also included is a series of recommendations for the JISC and Wellcome Trust regarding the archiving of UK materials.
- http://www.jisc.ac.uk/uploaded_documents/archiving_legal.pdf
-

Long-Term Preservation of Digital Material - Building an Archive to Preserve Digital Cultural Heritage from the Internet
Aschenbrenner, Andreas
(Date Created: Dec 2001)
(Austria)
- Submitted as a Diploma thesis, this report presents a detailed case study of the Austrian On-Line Archive and a prototype for automatic retrieval of interactive online documents. A review of long-term digital archiving initiatives, issues and proposed strategies in digital preservation are also included.
- http://www.ifs.tuwien.ac.at/~aola/publications/thesis-ando/
- Also available in PDF or gnu-zipped PostScript ( http://www.ifs.tuwien.ac.at/~aola/publications.html ).
- L’archivage d’Internet, un défi pour les décideurs et les bibliothécaires : scénarios d’organisation et d’évaluation L’expérience du consortium IIPC et de la BnF
Illien, Gildas
(Date Created: 10 Jul 2008)
(France)
- This paper delivered at World Library and Information Congress: 74th IFLA General Conference and Council (10-14 August 2008, Québec, Canada) outlines the challenge of introducing web archiving to library stakeholders and staff at the National Library of France (BnF). It also discusses metrics and organisation and BnF's experience with the IIPC (International Internet Preservation Consortium).
- http://archive.ifla.org/IV/ifla74/papers/107-Illien-fr.pdf
- Metrics and Strategies for Web Heritage Management and Preservation
Bermes, Emmanuelle; Illien, Gildas
(Date Created: 24 Aug 2009)
- This paper presented at the World Library and Information Congress: 75th IFLA General Conference and Assembly (23-27 August 2009, Milan, Italy) reviews the challenges of web archive measurement. It considers metric scenarios for web archives including collection development, processing and management and the core metrics for technical monitoring and cost analysis at all levels of the workflow.
- http://www.ifla.org/files/hq/papers/ifla75/92-bermes-en.pdf
- mod_oai : An Apache Module for Metadata Harvesting
Harrison, Terry L.; Liu, Xiaoming; McFarland, Nathan; Nelson, Michael L.; Van de Sompel, Herbert
(Date Created: 2005)
(United States of America)
- This article discusses mod_oai, an Apache 2.0 module that optimizes web content harvesting by building Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) capability into the Apache server. The article includes a comparison between web crawling and OAI-MPH harvesting and presents the results of experimentation using the authors' university web site as a testbed.
- http://arxiv.org/ftp/cs/papers/0503/0503069.pdf
- Modelling Information Persistence on the Web
Gomes, D.; Silva, Mario J.
(Date Created: 2006)
(Portugal)
- This paper will be presented at the Sixth International Conference on Web Engineering in
July 2006. Over a three year period the researchers measured URL and content persistence using Tomba, the Portuguese web archive. The paper discusses what characterises the persistent URLs and gives reasons for lack of persistence.
- http://xldb.fc.ul.pt/daniel/docs/papers/gomes06urlPersistence.pdf
-
Moral Perspective on South-North Web Archiving
Britz, Johannes J.; Lor, Peter
(Date Created: Dec 2004)
- This paper examines moral issues of web archiving. Financial and technological constraints make it difficult for organisations such as national libraries or governments in developing countries to undertake archiving, and the transient nature of the web means that these resources will be lost. The authors argue that developed countries have a moral obligation to undertake archiving on behalf of developing countries so that the principles of social justice such as equal access to information and freedom of communication can be achieved. Legal issues arising from web archiving, such as ownership and republishing, are briefly discussed.
- http://jis.sagepub.com/cgi/content/abstract/30/6/540
- Journal of Information Science, Vol. 30, no. 6 (December 2004), pp. 540-549. Available online by subscription from Sage Journals online and in hardcopy
-

National Digital Preservation Initiatives: An Overview of Developments in Australia, France, the Netherlands, and the United Kingdom and of Related International Activity
Beagrie, Neil
(Date Created: Apr 2003)
- This report, copublished by CLIR and the Library of Congress, examines national and international preservation initiatives in Australia, France, Netherlands and the United Kingdom. Although the report focuses on libraries, other organisations are also examined. The information was gathered over January and February 2002 by interviews, questionnaires and research. The report identifies underlying trends influencing preservation worldwide and includes several recommendations from national libraries to the NDIIPP.
- ISBN: 1932326006
- http://www.clir.org/pubs/abstract/pub116abst.html
- Available online as full text or .pdf, or for purchase in print
-

National Library of Australia: Ensuring Long-Term Access to Online Publications
Phillips, Margaret
In: JEP - The Journal of Electronic Publishing
(Date Created: Jun 1999)
(Australia)
- This article gives an overview of archiving networked digital material in the PANDORA project and covers topics such as collection, selection, metadata, access, costs, permanent naming, technological obsolescence and preservation.
- http://hdl.handle.net/2027/spo.3336451.0004.405
-

New Data Storage and Service Model of China Web InfoMall
Chen, Chong; Huang, Lianen; Xie, Zhengmao; Yan, Hongfei
(Date Created: 2004)
(China)
- This paper describes the Chinese web archiving project, InfoMall, which began in 2001. The paper concentrates on three main aspects of implementation strategies, how the materials are taken from the web, storage strategies and availability of the pages to the public. Detailed descriptions of the service model and the Tianwang archival storage format for web pages are also available.
- http://www.iwaw.net/04/Hongfei.pdf
- This workshop was held in conjunction with the 8th European Conference on Research and Advanced Technologies for Digital Libraries, September 2004
-
New Legal Deposit Law Saves Non Print Materials and Electronic Publications for Future Generations
Finlayson, Catriona
(Date Created: Feb 2004)
(United Kingdom)
- This news release from the British Library describes the implementation of the Legal Deposit Libraries Act 2003 (UK). The ground-breaking law extends current legal deposit legislation by requiring publishers to deposit electronic and other non-print materials (CD-Roms, websites, microform) to the British Library and other legal deposit libraries in the UK. The article also contains background information on how the scheme works, the types of electronic materials covered and international trends in the legal deposit of electronic resources.
- http://www.sconul.ac.uk/publications/newsletter/30/23.PDF
- In News from Member Libraries, SCONUL Newsletter 30 Winter 2003
- New Methodologies for Quantifying Licence-Based Commons on the Web
Bildstein, Ben
(Date Created: Aug 2008)
(Australia)
- This paper delivered at the First Interdisciplinary Research Workshop on Free Culture, at iSummit '08, 30 July – 1 August 2008, describes the challenges of quantifying online commons, proposes using raw web crawler data and reports on some experiments using the Australian web domain harvests.
- http://law.bepress.com/unswwps/flrps08/art52/
- University of New South Wales FAculty of Law Research Series, 2008. Working Paper 52
- New Zealand Web Harvest 2008
National Library of New Zealand
(Date Created: Oct 2008)
(New Zealand)
- This web page provides an overview of the New Zealand Domain Web Harvest which was conducted in October 2008 by the National Library of New Zealand.
- http://www.natlib.govt.nz/about-us/current-initiatives/past-initiative
s/web-harvest-2008
- Our Digital Heritage as Source Material to End-users: Collection of and Access to Net Publications in the National Library of Norway
Rustad, Kjerste
(Date Created: 01 May 2006)
(Norway)
- This paper describes how the National Library of Norway collects net publications under the Norwegian Legal Deposit Act and how they may be made accessible to users. It considers the limitations of the existing legislation.
- http://www.palgrave-journals.com/dam/journal/v2/n3/abs/3650032a.html
- Available in Journal of Digital Asset Management ISSN1743-6559 (2006) 2, 172-177
- Our Digital Heritage as Source Material to End-users: Collection of and Access to Net Publications in the National Library of Norway
Rustad, Kjersti
(Date Created: Apr 2006)
(Norway)
- This article provides an overview of the collection and access of Web documents at the National Library of Norway. It reviews end user access in the light of the legislative framework.
- http://www.emeraldinsight.com/10.1108/01604950610677567
- Collection Building ISSN 0160-4953, 25/3 (2006) pp:89-94
- PADICAT : Realitat i Reptes 3 Anys de L'arxiu Web de Catalunya
Cócera, Daniel; Llueca, Ciro
(Date Created: May 2008)
(Spain)
- This paper (in Spanish) given at 11es Jornades Catalanes d'Informació i Documentació, Barcelona (Spain), 22-23 May 2008 provides an overview and update of the operation of Padicat, the web archive project of the Library of Catalonia.
- http://eprints.rclis.org/13562/1/llueca_padicat_jornades_2008.pdf
- In Actes 11es Jornades Catalanes d’Informació i Documentació, 2008 pp 163-178 http://www.cobdc.org/jornades/11JCD/actes11jcid/comunicacions/pag_163.pdf
-

PANDORA Digital Archiving System (PANDAS): Managing Web Archiving in Australia : A Case Study
Koerbin, Paul
(Date Created: 16 Sep 2004)
(Australia)
- This is a detailed case study of the processes and workflows of the PANDORA and how they are supported by the underlying PANDAS digital archiving system. Processes examined include metadata, access restrictions, harvesting, quality assurance, reporting and archiving. The paper also includes a section on future directions for PANDAS and web archiving in Australia.
- http://www.nla.gov.au/nla/staffpaper/2004/koerbin2.html
-

PANDORA, Australia's Web Archive, And The Digital Archiving System That Supports It
Phillips, Margaret E
In: DigiCULT.Info Newsletter
(Date Created: Dec 2003)
(Australia)
- This article looks at Australia’s Web Archive, PANDORA – led by the National Library of Australia in co-operation with eight other partners - and its supporting digital archiving system, PANDAS. The article not only explores the role of national libraries in relation to web harvesting and describes their different approaches, but provides an overview of the selection criteria used for PANDORA, the processes of digital archiving and the main functions of the PANDAS software. Future directions of PANDAS as well as digital archiving initiatives, such as the “Deep Web”, are also discussed.
- http://www.digicult.info/downloads/dc_info_issue6_december_20031.pdf
- PANDORA, Australia's Web Archive. How Much Metadata is Enough?
Koerbin, Paul; Philips, Margaret E.
(Date Created: 2007)
(Australia)
- This article discusses the development of the PANDORA Web Archive, policy and practices and metadata workflows and issues.
- http://www.informaworld.com/smpp/content~content=a909291621~db=all~ord
er=page
- Journal of Internet Cataloging, Feb 2004 (NB: Cover date 2004, published 2007), Vol 7, no 2 p. 19-33
- PANDORA: Collecting in a Digital World - Where is the Artefact?
Koerbin, Paul
(Date Created: 10 Oct 2007)
(Australia)
- This paper was given at the The ALIA Acquisitions and The Bob Hawke Prime Ministerial Library Symposium: The Acquisition of Cultural Artefacts October 10, 2007, Bradley Forum, Hawke Building, University of South Australia. It discusses whether the Web and archived web pages can be considered as cultural artefacts.
- http://pandora.nla.gov.au/documents/artifacts_symposium_pandora.pdf
-

Paradigma Project
van Nuys, Carol
In: RLG DigiNews
(Date Created: 15 Apr 2003)
(Norway)
- Project Paradigma is designed to collect and preserve Norway's digital cultural heritage. The Project began in August 2001 and is scheduled to end in December 2004. Its goals are to develop and establish routines for the selection, collection, description, identification, and storage of all types of digital documents and to give users access to these publications in compliance with the Legal Deposit Act.
The article provides an overview of the Project, including selection criteria, legal and technical issues. The article also provides a background of legal deposit in Norway, especially in regard to digital publications.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070513/viewer/file1
651.html#feature2
-
Paradigma Project and its Quest for Metadata Solutions and User Services
Albertsen, Ketil; Nuys, Carol van; Pedersen, Linda; Stenstad, Asborg
(Date Created: 22 Aug 2004)
(Norway)
- A paper examining the National Library of Norway’s Paradigma Project, which was presented at the 70th IFLA Conference in Buenos Aires in August 2004. The initiative (which is engaged in the harvesting and archiving of the Norwegian domain as part of the NLN’s legal deposit responsibilities) is described, and its work in exploring appropriate metadata solutions is analysed. Other topics surveyed include the incorporation of the Functional Requirements for Bibliographic Records (FRBR) model into the archive design, and the implementation of a proposed authentication service for document verification.
- http://archive.ifla.org/IV/ifla70/papers/009e-Nuys.pdf
-

Part of Our Culture is Born Digital : On Efforts to Preserve it for Future Generations
Aschenbrenner, Andreas ; Rauber, Andreas
(Date Created: Jun 2001)
(Austria)
- This English version of a paper presented at the 'Knowledge Networking in Cultural Studies' conference in May 2001, examines the challenges for the long-term preservation of digital cultural heritage. The authors discuss some of the main technical issues involved in archiving the Internet, and evaluate possible solutions such as emulation and migration of data. The paper concludes with a report on the AOLA pilot project to create an Austrian On-Line Archive.
- http://www.ifs.tuwien.ac.at/%7Eaola/publications/trans10.html
- Published in Trans - Internet-Zeitschrift für Kulturwissenschaften, 10. Nr. / Juni 2001
-
Political Communications in Web Archiving : a Proposal to the Andrew W. Mellon Foundation from the Centre for Research Libraries
Reilly, Bernard F.
(Date Created: 26 Jul 2002)
(United States of America)
- This proposal describes the Center for Research Libraries (CRL) project to develop effective methodologies for the systemic, sustainable preservation of web based political communications. The project covers three main areas of political Web communications archiving- curatorship, technology and long-term resource management. The project has a fifteen month time-frame. Web based political communications has been identified as lacking preservation strategies. Yet this area provides an important source of information about political trends, attitudes and activities.
- http://www.library.cornell.edu/iris/research/WebPolCom.pdf
-
Preservation Functionality in a Digital Archive
Oltmans, Erik; van Diessen, Raymond; van Wijngaarden, Hilde
(Date Created: 2004)
(Netherlands)
- This paper describes the design, rationale and use of the Preservation Manager, a component of the e-Depot digital archiving system at the Koninklijke Bibliotheek, the National Library of the Netherlands. The function of the Preservation Manager is to define and manage the technical environments needed to support digital resources stored in the e-Depot.
- http://doi.acm.org/10.1145/996350.996416
- http://portal.acm.org/citation.cfm?id=996350.996416&coll=portal&dl=ACM
& type=series&idx=996350&part=Proceedings&WantType=Proceedings&titl e=International%20Conference%20on%20Digital%20Libraries
- Paper presented at the International Conference on Digital Libraries, 2004. Full text available via subscription from the ACM Portal - http://portal.acm.org/portal.cfm?CFID=26225830&CFTOKEN=69467023
-

Preservation Risk Management for Web Resources: Virtual Remote Control in Cornell's Project Prism
Botticelli, Peter; Entlich, Richard; Kenney, Anne R; Lagoze, Carl; McGovern, Nancy Y; Payette, Sandra
In: D-Lib magazine
(Last Updated: 21 Jan 2002)
(United States of America)
- This article provides an overview of Project Prism at Cornell University, through which a non-custodial, distributed model for archiving is being explored. Among its aims is to use automated strategies to monitor and validate resource integrity, in a risk-based preservation management program.
- http://www.dlib.org/dlib/january02/kenney/01kenney.html
- D-Lib Magazine , Vol. 8, No. 1
- Preserving Access to Government Websites : Development and Practice in the CyberCemetery
Hoffmann, Starr
(Date Created: 26 May 2008)
(United States of America)
- This paper delivered at World Library and Information Congress: 74th IFLA General Conference and Council (10-14 August 2008, Québec, Canada) provides an overview of the development of the CyberCemetery and the process of identifying, capturing, and publishing content in the archive. The Cybercemetery is a collaboration between the University of North Texas Library and the US Government Printing Office.
- http://archive.ifla.org/IV/ifla74/papers/130-Hoffman-en.pdf
- Also available in French translation http://archive.ifla.org/IV/ifla74/papers/130-Hoffman-trans-fr.pdf
- Preserving Government and Political Information : The Web-At-Risk Project
Glenn, Valerie D.
In: First Monday
(Date Created: Jul 2007)
(United States of America)
- This article describes the Web–at–Risk Project to harvest & preserve born–digital government and political information and analyses the issues, tools and services involved in web harvesting.
- http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view
/1917/1799
-
Preserving the Chinese Internet: The DACHS Project
Lecher, Hanno
In: D-Lib magazine
(Date Created: Jan 2003)
- A brief overview of the Digital Archive for Chinese Studies (DACHS) project at the University of Heidelberg, which aims to identify, archive and ensure long-term access to Internet resources relevant for Chinese Studies, including resources drawn from the Chinese web.
- http://www.dlib.org/dlib/january03/01inbrief.html#LECHER
- Preserving the Scholarly Side of the Web
Green, Cody; Leggett, John; Maslov, Alexey; Mikeal, Adam; Phillips, Scott
(Date Created: 26 Oct 2006)
(United States of America)
- This paper given at LA Webb 2006 (4th Latin American Web Conference, 25-27 October 2006, Puebla Cholula Mexico), presents results of a case study (migration of the Journal of Digital Information- JoDI hosted by the Texas Digital Library) addressing the issues around preservation in a digital library. It focuses on issues applying to the preservation of scholarly articles encoded in current web standards, discusses emulation and migration, and compares the strengths and weaknesses of automatic and manual migration.
- http://txspace.tamu.edu/bitstream/handle/1969.1/5680/LA%20WEB%202006%2
0Paper.pdf?sequence=1
-
Preserving Web History
Mitchell, Robert L
(Date Created: 08 May 2002)
(United States of America)
- Focusing on the USA, this article looks at the challenges faced by a number of companies in relation to website archiving. With companies having statutory and legal obligations to preserve records of past transactions and activities, the need to preserve information on websites has posed a number of difficulties for these organisations. The article also includes a case study of web archiving techniques employed at Corporate Express Inc in the USA.
- http://www.computerworld.com.au/index.php/id;1965761530
- Available in Computerworld, Vol 36, Issue 32, 8 May 2002, pp. 26-28, ISSN 00104841 and by subscription online at http://www.ebscohost.com/
- Proceedings of the International Web Archiving Workshop 2006 : IWAW 06
(Date Created: Oct 2006)
(Spain)
- The IWAW workshop was held at the University of Alicante, Alicante, Spain, 21-22 September 2006. The proceedings include presentations on the acquisition, maintenance and preservation of digital objects for long-term access, web archiving and studies on effective usage of this type of archives.
- http://www.iwaw.net/06/PDF/iwaw06-proceedings.pdf
- Prototypes Related to IIPC Access Working Group Use Cases
IIPC Access Working Group
(Date Created: May 2006)
- This report develops prototypes encompassing the functional requirements as suggested by the use studies in the companion report Use Cases for Access to Internet Archives. The prototypes were used to ascertain what interfaces may be required in the development of a common architecture for web archives.
- http://netpreserve.org/publications/iipc-r-004.pdf
- Quantitative Evaluation of Dissemination-Time Preservation Metadata
Nelson, Michael L.; Smith, Joan A.
(Date Created: 15 Sep 2008)
(United States of America)
- This paper presented at ECDL 2008 (Aarhus, Denmark 14-20 September 2008) discusses the feasibility of using a web server for just in time preservation metadata generation and selection of utilities to improve performance.
- http://www.cs.odu.edu/~mln/pubs/ecdl-2008/preservation-metadata-ecdl-2
008.pdf
- Also available in Lecture Notes in Computer Science, Volume 5173/2008, pp 346-357
- Recent Developments in Digital Archiving and Preservation
Gatenby, Pam
(Date Created: Nov 2006)
(Australia)
- This paper which was prepared for the CDNL Meeting, Seoul, 23 August 2006 provides an overview of developments in digital archiving and preservation of interest to National Libraries in particular the National Library of Australia.
- http://www.nla.gov.au/nla/staffpaper/2006/documents/pgatenby_CDNL.pdf
-

Report on Senior Executive Fellowship to Research Digital Archiving in National Libraries
Gatenby, Pam
(Date Created: 09 Dec 2002)
- A report prepared by the National Library of Australia's Assistant Director-General, Collections Management, of a digital archiving research trip undertaken in August-September 2002. Her observations of digital archiving initiatives at the national libraries of Denmark, Sweden and Canada, the Library of Congress, and the Online Computer Library Center in Ohio are described, and the differing approaches are compared. In conclusion, the implications for the National Library of Australia's own archiving responsibilities are also discussed.
- http://www.nla.gov.au/nla/staffpaper/2002/elect.html
- Role and Justification of Web Archiving by National Libraries: A Questionnaire Survey
Eisenschitz, Tamara ; Shiozaki, Ryo
(Date Created: Jun 2009)
- This article present the findings of a survey of 16 national libraries (Australia, Canada, Czech Republic, Denmark, Finland, France, Iceland, New Zealand, Norway, Slovenia, Sweden, Swiss, Netherlands, UK, USA, Japan) into how they justify their web archiving programs. The cost benefits and legal justifications are considered.
- http://lis.sagepub.com/cgi/content/abstract/41/2/90
- Journal of Librarianship and Information Science (UK) ISSN 0961-0006 Jun 2009 Vol 41 No 2 pp:90-107
-

Sampling the Umich.edu Domain
Lyle, Jared
(Date Created: 2004)
(United States of America)
- The article compares sampling methodologies, such as systematic and random sampling, to the approaches of whole domain and selective downloading as an appraisal mechanism for web archiving. The author uses the whole domain of umich.edu as an example. He argues that sampling methodologies would provide a less biased and more manageable approach to web archiving. Some disadvantages of this method are also briefly discussed.
- http://www.iwaw.net/04/Lyle.pdf
- This workshop was held in conjunction with the 8th European Conference on Research and Advanced Technologies for Digital Libraries, September 2004.
- Selective Archiving of Web Resources : A Study of Acquisition Costs at the National Library of Australia
Phillips, Margaret E.
In: RLG DigiNews
(Date Created: 16 Jun 2005)
(Australia)
- This article examines the costs of maintaining PANDORA : Australia's web archive at the National Library of Australia. In outlining the costs, the article also describes the purpose, history, activities and staffing of PANDORA. It finds that acquiring Web publications is considerably more expensive than printed publications and describes ways in which the costs could be reduced.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file5
58.html#article0
- Selective Archiving of Web Resources: A Study of Processing Costs
Buzina, Tanja; Holub, Karolina; Milinovic, Miroslav; Topolšcak, Nebojša; Willer, Mirna; Zajec, Jasenka
(Date Created: 2008)
(Croatia)
- This article describes a study undertaken at the National and University Library of Croatia to assess cost of web archiving of Croatian web resources. The results were compared to the results of a similar exercise undertaken at the National Library of Australia.
- http://www.ingentaconnect.com/content/mcb/280/2008/00000042/00000004/a
rt00001
- Program: electronic library & information systems, Volume 42, Number 4, 2008 , pp. 341-364(24)
- Separating the Wheat from the Chaff: Identifying the Key Elements in the NLA .au Domain Harvest
Fellows, Geoff; Harvey, Ross; Lloyd, Annemaree; Pymm, Bob; Wallis, Jake
(Date Created: Sep 2008)
- This paper by the Charles Sturt University's POA (Preservation for Ongoing Accessibility) research group describes the 2006 National Library of Australia whole domain harvest and uses the example of blogs to address how to identify material within the harvest and determine issues that need further investigation.
- http://eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailm
ini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=EJ814499&ERICExtSearc h_SearchType_0=no&accno=EJ814499
- Australian Academic & Research Libraries, v39 n3 p137-148, Sep 2008 in print to subscribers only.
- Strategies and Approaches to Building Thematic Collections in WebArchiv
Coufal, Libor; Žabička, Petr
(Date Created: Nov 2007)
(Czech Republic)
- This paper delivered at InFuture 2007 describes experiences of building thematic collections in WebArchiv, a joint web-archiving project run by the National Library of Czech Republic in cooperation with the Moravian Library and Institute of Computer Science of Masaryk University in Brno. It also considers the potential use of web based alerting systems and presents findings of a preliminary study.
- http://en.webarchiv.cz/files/dokumenty/konference/InFuture07.pdf
- InFuture 2007, 7-9 Nov 2007, Zagreb
-
TechNotes -- The Internet Archive: an End to the Digital Dark Age
Panos, Patrick
(Date Created: 22 Mar 2003)
(United States of America)
- A short introduction to the Internet Archive with details of how to use the "Wayback Machine." Concludes that it could be important tool for social work education research and that it may assist many social work authors in keeping their Internet references valid and alive.
- http://www.accessmylibrary.com/coms2/summary_0286-23714134_ITM
- Journal of Social Work Education, 39(2), 343-347.
- Terminology Evolution in Web Archiving: Open Issues
Iofciu, Tereza; Niederee, Claudia ; Risse, Thomas; Siberski, Wolf; Tahmasebi, Nina
(Date Created: 18 Sep 2008)
- This paper given at IWAW 2008 (18-19 September, Aarhus Denmark) discusses the problem of terminology evolution in web archive queries and potential solutions for dealing with long term "semantic" accessibility.
- http://iwaw.net/08/IWAW2008-Tahmasebi.pdf
-

Test Bed Taxonomy for Crawler
Boyko, Andrew
(Date Created: Jul 2004)
- This paper reports on the problems and issues that automated archival web crawlers encounter in the surface web within the context of harvesting, content retrieval and link detection and presentation. The 15-page document covers issues such as non-typical URIs, cookies and forms, as well as synthetic and non-HTML content and links.
- http://netpreserve.org/publications/iipc-r-002.pdf
- Version 1
- For the Record : Assessing the Impact of Archiving on the Archived
Crook, Edgar
In: RLG DigiNews
(Date Created: 15 Aug 2006)
(Australia)
- The article describes a study on the effect of PANDORA archiving on digital publishers and publications. The study was based on a survey and supplemented by a selected range of archived material which was examined to discover publication patterns pre- and post-archiving. Publications examined include blogs, e-journals and commercial websites. The broad conclusion of this study is that PANDORA had not had a detrimental effect on publications.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file3
438.html#article0
-
On the Web, Research Work Proves Ephemeral
Weiss, Rick
(Date Created: 24 Nov 2003)
(United States of America)
- Using citations and footnotes as examples, this article reports on a recent study of scientific journals that found that that after 27 months of publication, 13 per cent of internet references were inactive. With many scholarly articles now citing Uniform Resource Locators (URLs) in their footnotes and references, there is a growing concern amongst scientists and scholars that this medium is becoming increasingly more ephemeral than archival in nature. The article also highlights the continual importance of the work of the Internet Archive and the development of the DOI (Digital Object Identifier) system – which traces a web page, even if it has moved to a new URL - in helping to make web pages permanently available for future generations.
- http://www.washingtonpost.com/ac2/wp-dyn/A8730-2003Nov23?language=prin
ter
- Also available in hardcopy.
-

And Then There's the Rest of the Stuff : the Role of the PANDORA Archive
Koerbin, Paul
(Date Created: 13 May 2004)
(Australia)
- This paper gives an overview of the PANDORA Archive at the National Library of Australia. It describes PANDORA and its management system, PANDAS, and the advantages and disadvantages of selective archiving, as well as various technical and legal issues. Specific problems relating to the harvesting and archiving of multimedia, sound and video are also briefly examined.
- http://www.nla.gov.au/nla/staffpaper/2004/koerbin1.html
- Paper given at the Australasian Sound Recording Association 2004 Conference 'Sounding Out a Cultural Heritage'
-

Towards Continuous Web Archiving: First Results and an Agenda for the Future
Masanès, Julien
In: D-Lib magazine
(Date Created: Dec 2002)
(France)
- As part of its Web archiving activities, the Bibliothèque nationale de France (BnF) has investigated various ways in which the crawler programs used for the automatic collection of Web pages can be improved. In particular, the BnF initiative has tried to implement and evaluate sophisticated crawler programs that use the link structure of the Web to provide an automatic estimation of a Web page's importance. This paper describes the result of experiments whereby the automatic ranking of a sample set of Web sites (using Xyleme's ranking algorithm) was compared with manual evaluations of the same sites by librarians. There was a sufficient level of correlation from these to suggest that automatic importance evaluation could help national libraries build large-scale collections of Web resources.
- http://www.dlib.org/dlib/december02/masanes/12masanes.html
- Towards format repositories for web archives
Christensen, Niels H.
(Date Created: Aug 2004)
(Denmark)
- This paper was presented at the International Web Archiving Workshop in 2004 and describes the challenges of file formats for web archives.
- http://netarkivet.dk/publikationer/FormatRepositories-2004.pdf
- UK Government Web Continuity : Persisting Access Through Aligning Infrastructures
Spencer, Amanda; Sheridan, John; Thomas, David; Pullinger, David
(Date Created: Dec 2008)
(United Kingdom)
- This paper describes a study by a working group established by the UK Cabinet Office to ensure that parliamentarians would be able to access information referenced by URLs particularly those referenced in Hansard in perpetuity. The Web Continuity solution was developed after investigation, analysis and comparison of options. In November 2008 The National Archives began comprehensive archiving of the government Web Estate.
- http://www.dcc.ac.uk/events/dcc-2008/programme/papers/UK%20Government%
20Web%20Continuity.pdf
- Paper of the 4th International Digital Curation Conference, 1-3 Dec 2008, Edinburgh Scotland
- UK Web Archiving Consortium
Jones, Maggie
(Date Created: Mar 2006)
(United Kingdom)
- Maggie Jones of the Digital Preservation Coalition (DPC) interviews Philip Beresford, Web archiving project manager for the UK Web Archiving Consortium. The interview covers various aspects of the UK Web Archiving Consortium, including project partners, funding and what is being collected.
- http://www.dpconline.org/graphics/join/ukwac.html
- UKWAC : Building the UK's First Public Web Archive
Bailey, Steve; Thompson, Dave
In: D-Lib magazine
(Date Created: Jan 2006)
(United Kingdom)
- This article describes the UK Web Archiving Consortium project. This project consists of six UK institutions who aim to establish a test bed for the selective archiving of UK web sites. The article discusses the background, methodology, benefits and issues arising as the project develops.
- http://dlib.org/dlib/january06/thompson/01thompson.html
-

Uncovering Information Hidden in Web Archives: A Glimpse at Web Analysis Building on Data warehouses
Aschenbrenner, Andreas; Bruckner, Robert M.; Kaiser, Max; Rauber, Andreas; Witvoet, Oliver
In: D-Lib magazine
(Date Created: Dec 2002)
- Web archives hold a wealth of information that can be used to analyse various aspects of the Web. In order to investigate this, the authors established a data warehouse as a key to this information in the context of the Austrian On-Line Archive (AOLA). This made it possible to analyse the characteristics of the Web in a flexible and interactive manner using on-line analytical processing (OLAP) techniques. Specifically, these were used to investigate technological aspects of the archive, e.g. the operating systems and Web servers used, the variety of file types, forms or scripting languages encountered, as well as the link structure within domains. These can then be analysed to infer the characteristics of technology maturation and impacts on community structures. [Description edited from original abstract].
- http://www.dlib.org/dlib/december02/rauber/12rauber.html
- Use Cases for Access to Internet Archives
IIPC Access Working Group
(Date Created: May 2006)
- This report develops a number of use case scenarios for a web archive. The scenarios include cases relating to data usage, user interface, data mining and presentation.
- http://netpreserve.org/publications/iipc-r-003.pdf
- Using Page Histories for Improving Browsing the Web
Jatowt, Adam; Kawai, Yukiko; Tanaka, Katsumi
(Date Created: Sep 2008)
- This paper given at IWAW 2008 (International Workshop on Web Archiving, Aarhus Denmark 18-19 Sept, 2008), outlines and classifies interaction methods with web page histories with a view to improving browsing in web archives.
- http://iwaw.net/08/IWAW2008-Jatowt.pdf
-
VidArch: Preserving Video Objects and Context: Final Report
Marchionini, Gary; Tibbo, Helen
(Date Created: 11 Sep 2009)
(United States of America)
- This is the final report of the VidArch Project, supported by NSF and the Library of Congress NDIIPP. The project contributed to development of policies and tools to facilitate preservation of digital video from the internet (such as YouTube) and explored context related aspects of curation. Tools include ContextMiner, TubeKit and VidArch Exchange.
- http://sils.unc.edu/research/publications/reports/TR-2009-01.pdf
-

Virtual Remote Control : Building a Preservation Risk Management Toolbox for Web Resources
Buckley, Ellie; Entlich, Richard; Kehoe, William R.; Kenney, Anne R.; McGovern, Nancy Y.
In: D-Lib magazine
(Date Created: Apr 2004)
(United States of America)
- Importance of Internet archiving was brought to public attention after material archived by the Wayback Machine was accepted in the US court. Cultural institutions face legal as well as technical challenges when they attempt to archive the Internet and much of the content is therefore lost. Cornell University Library's Virtual Remote Control (VRC) initiative differs from most web archiving projects by its emphasis on monitoring websites over time, with capture being used only as a last resort. The process is envisaged to be fully automated in the future via a Toolbox software that is being developed, but at present considerable human input is necessary.
- http://www.dlib.org/dlib/april04/mcgovern/04mcgovern.html
-

Web Archive Activities in Denmark
Christensen-Dalsgaard, Birte
In: RLG DigiNews
(Date Created: Jun 2004)
(Denmark)
- This article outlines the history of development of legal deposit for digital material in Denmark. It included the gradual change in the legislation influenced by the conference on the topic, harvesting experiments, Parliament hearing and a resulting report followed by government funding. Harvesting experiments are examined in detail, including: their selection policy, hybrid strategy (involving bulk harvesting four times a year combined with selective harvesting of approximately 80 sites, combined with 2-3 yearly event-based harvests), harvesting tools used, testing different archive formats for storing the data, preservation issues, various other technical issues, as well as making sure that 'look and feel' of the archived material is retained.
- http://worldcat.org/arcviewer/1/OCC/2007/08/08/0000070511/viewer/file3
628.html#article0
-

Web Archiving : a Feasibility Study for JISC and the Wellcome Trust
JISC (Joint Information Systems Committee)
(Date Created: 09 Mar 2003)
(United Kingdom)
- An overview of a feasibility study by UKOLN into web archiving. The aims of the study were to provide both the JISC and the Wellcome Trust with an analysis of existing web archiving arrangements and how they address the needs of the UK research community, plus recommendations on how the JISC and the Wellcome Trust could develop web archives initiatives. This site includes links to two PDF reports, "Collecting and Preserving the World Wide Web" and "Legal Issues Relating to the Archiving of Internet Resources in the UK, EU, US and Australia".
- http://www.jisc.ac.uk/index.cfm?name=project_webarchiving
- Web Archiving at BnF
IFLA Core Activity on Preservation and Conservation
(Date Created: Dec 2006)
(France)
- This article in International Preservation News describes web archiving activities at the Bibliotheque nationale de France.
- http://archive.ifla.org/VI/4/news/ipnn40.pdf
- Available in International Preservation News, Vol no 40 December 2006 pp. 27-34 in English with summary in French & Spanish
- Web Archiving at the National Library of New Zealand
Lala, Vanita; Joe, Susanne
(Date Created: Oct 2006)
(New Zealand)
- This paper given at the LIANZA Conference 2006 (8-11 October 2006, Wellington, NZ) discusses web archiving at the National Library of New Zealand and the changes resulting from the 2003 National Library of New Zealand Act that extended legal deposit to electronic publications. It illustrates it with a case study on the selection, harvesting and appraisal of websites and blogs in the 2005 election.
- http://opac.lianza.org.nz/cgi-bin/koha/opac-detail.pl?bib=113
- Web Archiving in a Web 2.0 World
Crook, Edgar
(Date Created: Sep 2008)
(Australia)
- This paper delivered at ALIA Biennial Conference, 2-5 September 2008, Alice Springs Australia gives an update on the activities of the PANDORA Archive, the current state of web archiving in Australia and adaption to web 2.0 applications.
- http://pandora.nla.gov.au/pan/13910/20080930-1156/conferences.alia.org
.au/alia2008/pdfs/124.TT.pdf
- Web Archiving in Denmark August 2009: A Fact Sheet
Netarchive.dk
(Date Created: Aug 2009)
(Denmark)
- This fact sheet provides a brief overview of web archiving activities in Denmark in August 2009 by Netarchive.dk (Netarkivet.dk). Included are brief details of the organization's resources, strategies, statistics and access details.
- http://netarchive.dk/publikationer/Fact%20sheet%20Webarchiving%20in%20
Denmark%202009.pdf
- Web Archiving in the UK: Cooperation, Legislation and Regulation
Tuck, John
(Date Created: Dec 2008)
(United Kingdom)
- This article in Liber Quarterly provides an overview of web archiving in the UK since 2001. It considers the international context, collaboration and the impact of the 2003 legal deposit legislation.
- http://liber.library.uu.nl/publish/articles/000258/article.pdf
- Liber Quarterly ISSN 1435-2005, Vol 18 (2008) Issue 3/4
- Web Archiving Internationally: Interoperability in the future?
Jacobsen, Grethe
(Date Created: 14 May 2007)
- This paper delivered at the World Library and Information Congress: 73RD IFLA General Conference and Council (Durban South Africa, 19-23 August 2007) reports the results of a questionnaire sent to 95 National Libraries regarding their web archiving activities and intentions. Partnering was considered of key importance. Legal Deposit, copyright and other legislation important challenges.
- http://archive.ifla.org/IV/ifla73/papers/073-Jacobsen-en.pdf
- Text also available in Spanish and French from coference website
- Web Archiving System of the National Library of Korea : OASIS
National Library of Korea
(Date Created: Mar 2007)
- This article describes the National Library of Korea's Online Archiving & Searching Internet Sources (OASIS) system which was developed in 2005 to preserve online web resources . The NLK OASIS project is developing a selective collection approach and the basic workflow and future developments are outlined.
- http://www.ndl.go.jp/en/publication/cdnlao/058/583.html
- CDNLAO Newsletter: Forum for the Conference of Directors of National Libraries in Asia and Oceania No 58, March 2007
- Web Archiving: Issues and Problems in Collection Building and Access
Jacobsen, Grethe
(Date Created: Dec 2008)
(Denmark)
- This article in Liber Quarterly, focuses on collection-building and access issues based on the experiences of web archiving by Netarkivet.dk in Denmark since 2005.
- http://liber.library.uu.nl/publish/articles/000273/article.pdf
- Liber Quarterly ISSN 1435-2005, Vol 18 (2008) Issue 3/4
- Web Archiving: Organizing Web Objects into Web Containers to Optimize Access
JaJa, Joseph; Song, Sangchul
(Date Created: 2007)
- This technical report (14 p.) considers the problem of archiving a linked set of web objects into web containers such as WARC so as to minimize the number of containers accessed during a typical browsing session.
- https://wiki.umiacs.umd.edu/adapt/images/c/cd/OrganizingWebObjects_UMI
ACS-TR-2007_42.pdf
- Technical Report UMIACS-TR-2007-42
- Web at Risk : a Distributed Approach To Preserving Our Nation's Political Cultural Heritage
California Digital Library
(Date Created: 2005)
(United States of America)
- Following a grant from the NDIIPP (National Digital Information Infrastructure and Preservation Program), the California Digital Library and its partners are developing a distributed approach to preserving political and government web pages. The 4 and a half year project is developing open source web archiving tools to capture, curate, and preserve collections of web-based government and political information. The project has four key areas - Content identification and selection, Content acquisition, Content retention and transfer and Partnership building.
- http://www.cdlib.org/inside/projects/preservation/webatrisk/
- Web Curator Tool
Beresford, Philip
In: Ariadne
(Date Created: Jan 2007)
(New Zealand; United Kingdom)
- This paper describes the development of the Web Curator Tool, a Web harvesting management system by the National Library of New Zealand, British Library and Sytec.
- http://www.ariadne.ac.uk/issue50/beresford/
- Web Harvesting for Nuclear Knowledge Preservation
International Atomic Energy Association
(Date Created: 21 May 2008)
- This technical report (50 p.) provides a contemporary technical overview of activities in the domain of web harvesting, document archiving and Internet access technology and its relevance to the preservation of online publications and documents in the nuclear field as background for a pilot project. The document considers the need for web archives and roles for nuclear knowledge organisations such as the International Atomic Energy Association (IAEA), methods for distributed access facilitation, international examples of ongoing web archiving initiatives, web harvesting technologies, and makes recommendations for project management.
- ISBN: 9789201112071
- http://www.iaea.or.at/inisnkm/nkm/documents/publ_web_harvesting.pdf
- IAEA Nuclear Energy Series, ISSN 1995–7807, No. NG-T-6.6
-

Web Harvesting Survey
Ashenfelder, Michael; Boyko, Andrew ; Marill, Jennifer
(Date Created: Jul 2004)
- This survey rates the harvestability of various types of web documents and websites within both the surface and deep web. The results are presented in tabular form and include types of documents, examples, formats, and difficulty ratings for harvest in terms of acquisition, parsing and presentation. More detail on problems in harvesting may be found in the complementary document, Test Bed Taxonomy for Crawler.
- http://netpreserve.org/publications/iipc-r-001.pdf
- Version 1
- Web Harvesting White Paper
(Date Created: 14 Feb 2007)
(United States of America)
- The white paper reports on the context of the results of the EPA Web Publication Harvesting Pilot Project, lesson learned about automated web publication discovery and harvesting technologies and planned further steps.
- http://www.fdlp.gov/home/repository/doc_details/543-web-harvesting-whi
te-paper
-
Web Preservation Projects at Library of Congress
Grotke, Abigail; Jones, Gina
(Date Created: 15 Mar 2003)
(United States of America)
- This paper, presented at the Museums and the Web 2003 conference, provides an overview of the Library of Congress (LC) web preservation projects to date. It reports on various aspects the LC's web harvesting activities including selection, capturing, cataloguing and access using topic based collections as examples, including September 11, 2002 Winter Olympics and the 2002 presidential election collections.
- http://www.archimuse.com/mw2003/papers/grotke/grotke.html
-

Web Resources as Cultural Heritage : International Symposium on Web Archiving : Proceedings
National Diet Library (Japan)
(Date Created: 30 Jan 2002)
- The proceedings of the symposium held on January 30, 2002 features a keynote speech from Brewster Kahle, a report on MINERVA and papers on Australian, Danish and Japanese web archiving activities.
- http://www.ndl.go.jp/en/publication/proceedings/index.html
-
Web Site Archiving : an Approach to Recording Every Materially Different Response Produced by a Website
Fitch, Kent
(Date Created: 2003)
(Australia)
- This paper, presented at the AusWeb 2003 conference, discusses an approach to archiving dynamically generated responses on a web site. The approach is based on archiving the end results of changes to content and content generation systems, inspecting and archiving materially different responses as they are delivered by a web site. The author argues that response archiving is feasible because only a small amount of responses are unique.
- http://ausweb.scu.edu.au/aw03/papers/fitch/
- Web Spam: A Survey with Vision for the Archivist
Benczúr, András A.; Bíró, István; Fekete, Zsolt; Siklósi, Dávid; Szabó, Jácint
(Date Created: Sep 2008)
- This paper delivered at IWAW '08 (September 18–19, 2008, Aarhus, Denmark) discusses the problem of web spam in web archives, reviews the state of the art in web spam filtering and envisions a shared solution for web archives based on work of the Living Web Archives project (LiWA).
- http://iwaw.net/08/IWAW2008-Benczur.pdf
- WebArchiv : A Digital Archive of Czech Documents Published on the Internet
National Library of the Czech Republic
(Date Created: 2007)
(Czech Republic)
- This brochure provides an introduction to WebArchiv - archive of Czech web resources, a collaboration between the National Library of the Czech Republic, Moravian Library and the Institute of Computer Science at Masaryk University. It outlines the history of WebArchiv, its content and searching.
- http://en.webarchiv.cz/files/dokumenty/letak/letak07_eng.pdf
-

Webarchivierung : Eine Aufgabe für Nationalbibliotheken ?!
Rauber, Andreas ; Schmidt, Alfred
(Date Created: 14 May 2001)
(Austria)
- This report, which was presented to the May 2001 meeting of the Federal Internet Advisory Board, discusses the challenges of archiving the Internet for the Austrian National Library. The legal deposit implications for the collection and control of online materials are also examined in the paper.
- http://www.ifs.tuwien.ac.at/~aola/publications/InternetBeirat_14-5-01_
1.doc
- Webarchiving Internationally: Interoperability in the Future? Results of a Survey of Webarchiving Activities of National Libraries, March 2007.
Jacobsen, Grethe
(Date Created: Sep 2007)
(Denmark)
- This paper is a revised version of a paper published on IFLANET prior to the World Library & Information Congress 73rd IFLA General Conference August 2007. It presents the results of a survey of national libraries in March 2007 on the interoperability of web archives and considers some of the associated challenges.
- http://netarchive.dk/publikationer/InteroperabilityInTheFuture_IFLA200
7.pdf
-
Webarian
Mackintosh, Hamish
(Date Created: 21 Feb 2002)
- An interview with Brewster Kahle about the Internet Archive published in The Guardian of 21 February 2002.
- http://www.guardian.co.uk/Archive/Article/0,4273,4359816,00.html
- Webarkivering
Henriksen, Birgit
(Date Created: Apr 2005)
(Denmark)
- 'This paper describes web archiving as a discipline and Netakivet as a project.' This article is in Danish.
- http://netarkivet.dk/publikationer/webarkivering-webarchiving.pdf
-
Webwatch : Surfing Historical UK University Websites
Kelly, Brian
In: Ariadne
(Date Created: Apr 2003)
(United Kingdom)
- This article is a description of a survey of archived university web sites in the Internet Archive accessed using the WayBack Machine. An automated view and comparison of the earliest and latest sites is available by a "Web Tour". The article also notes problems in automatically capturing the sites over time, frequent occurrences of images not displaying, accessibility issues caused by sites being designed for a specific browser and third party content not being functional on the archived version of the website.
- http://www.ariadne.ac.uk/issue35/web-watch/
- What Should We Preserve : the Question for Heritage Libraries in the Digital World
Phillips, Margaret E.
(Date Created: 2005)
(Australia)
- This article discusses the advantages and disadvantages of the various approaches to web archiving - whole domain harvesting, the hybrid approach and selective archiving. The National Library of Australia's PANDORA archive is discussed in detail as an example of the selective approach.
- http://muse.jhu.edu/demo/library_trends/v054/54.1phillips.html
- Also available in pdf. Library Trends, E ISSN 1559-0682, Vol. 54, no.1 (Summer 2005)
- Year of selective Web Archiving With the Web Curator at the National Library of New Zealand
Joe, Susanna; Lala, Vanita; Lee, Gillian; Paynter, Gordon
In: D-Lib magazine
(Date Created: May 2008)
(New Zealand)
- This article describes the Web Curator Tool and its use for selective archiving at the National Library of New Zealand in its first year.
- http://www.dlib.org/dlib/may08/paynter/05paynter.html
Top
- Archiving Websites : A Practical Guide for Information Management Professionals
Brown, Adrian
(Date Created: Jul 2006)
- Drawing on the author’s experience of managing the National Archives’ web archiving programme, together with lessons learned from other international initiatives, this book provides a comprehensive overview of current best practice. It assumes only a basic understanding of IT and web technologies, but also offers much to more technically-oriented readers. Contents include the development of web archiving, selection policies, collection methods, quality assurance, preservation, delivery to users, optimizing websites for archiving, legislation, managing a web archiving programme and future trends. [description from abstract]
- ISBN: 1856045536
- http://www.facetshop.co.uk/mm5/merchant.mvc?Screen=PROD&Store_Code=1&P
roduct_Code=553-7&Category_Code=
-

Archiving Websites : General Considerations and Strategies
Brugger, Niels
(Date Created: Jan 2005)
- The topic of this monograph is the micro archiving of websites. Micro archiving is defined as small scale archiving by individuals who have limited space, such as a desk top computer, and require the archiving for study or research. Other topics covered include the testing of currently available archiving software and the development of micro archiving strategies. The website associated with the electronic version of this monograph has additional information including detailed test descriptions and results, recommendations for using individual programmes and links to resources.
- ISBN: 8799050706
- http://www.cfi.au.dk/publikationer/archiving/guide.pdf
- Also available as an identical electronic verson from http://cfi.imv.au.dk/pub/boeger/bruegger_archiving.pdf
- Integrating Preservation Functions Into the Web Server
Smith, Joan A.
(Date Created: Aug 2008)
(United States of America)
- This PHD thesis submitted to Old Dominion University, presents research in which preservation functions have been integrated into the web server itself to produce archive-ready versions of the website’s resources. It considers current practices in digital preservation, roles of the web server and search engines, enumeration problems, and resource description including the CRATE model for self description.
- http://www.joanasmith.com/node/47
-

National Digital Preservation Initiatives: An Overview of Developments in Australia, France, the Netherlands, and the United Kingdom and of Related International Activity
Beagrie, Neil
(Date Created: Apr 2003)
- This report, copublished by CLIR and the Library of Congress, examines national and international preservation initiatives in Australia, France, Netherlands and the United Kingdom. Although the report focuses on libraries, other organisations are also examined. The information was gathered over January and February 2002 by interviews, questionnaires and research. The report identifies underlying trends influencing preservation worldwide and includes several recommendations from national libraries to the NDIIPP.
- ISBN: 1932326006
- http://www.clir.org/pubs/abstract/pub116abst.html
- Available online as full text or .pdf, or for purchase in print
- PoWR : The Preservation of Web Resources Handbook : Digital Preservation for the UK HE/FE Web Management Community
JISC-PoWR
(Date Created: Nov 2008)
(United Kingdom)
- This handbook was produced by the JISC-PoWR (Preservation of Web Resources ) project and is based on information gathered at the JISC-PoWR workshops held in 2008 and expertise of team members. The handbook deals with web resource management, capture, selection, appraisal and preservation, institutional drivers and policies for web archiving.
- http://jiscpowr.jiscinvolve.org/files/2008/11/powrhandbookv1.pdf
- Web Archiving
Masanes, Julien
(Date Created: 2006)
- In this book Julien Masanes, Director of the European Archive, has edited contributions from computer scientists and librarians on tools, tasks and processes used in web archiving. This book is for practitioners, researchers and students interested in methods, tools, standards and issues in web archiving.
- http://www.springer.com/east/home/computer/database+management+%26+inf
ormation+retrieval?SGWID=5-153-22-72040423-0
- Book available in print can be ordered from www.springer.com
- Web Modeling for Web Warehouse Design
Gomes, Daniel Coelho
(Date Created: 2007)
(Portugal)
- This PHD thesis (240 p.) explores requirements for developing a "Web Warehouse" an archive for the portuguese web. It uses the case study of the Tomba project but also gives an overview of web characterisation, web crawling and other web archive projects.
- http://xldb.fc.ul.pt/daniel/docs/papers/webModellingWebWarehouse.pdf
- WWW as a Challenge and
as a Chance for Parliamentary
and Party Archives : Beiträge der Tagung:
SPP/ICA: Annual Meeting 2.-4.11.2006
in Bonn
Archiv der sozialen Demokratie der Friedrich-Ebert-Stiftung
(Date Created: 2008)
(Germany)
- This interesting conference proceedings (161 p.) includes papers and presentations on political archives on the internet, the standards for structuring descriptive information
on archival holdings, dealing with copyright for archives on the internet
and as an appendix Guidelines for Archiving Websites in German, French and English.
- ISBN: 9783898929387
- http://library.fes.de/pdf-files/adsd/05657.pdf
- ISSN 1431-6080, Heft 5
Top
-
2nd ECDL Workshop on Web Archiving - in conjunction with the 6th European Conference on Digital Libraries (ECDL 2002)
19 Sep 2002
, Pontifical Gregorian University, Rome
(Italy)
Reviewed by: ECDL 2002 Workshop Report: Web Archiving
; 2nd ECDL Workshop on Web Archiving
- An overview, programme and links to presentations given at the Workshop are available. Presentations include - Saving the Web ; Archiving the Deep Web ; Using METS for Web Archiving ; Domain UK : Britain and the Web.
- http://bibnum.bnf.fr/ecdl/2002/index.html
-
3rd ECDL Workshop on Web Archives : in Conjunction with the 7th European Conference on Research and Advanced Technologies for Digital Libraries (ECDL 2003)
21 Aug 2003
, Trondheim, Norway
(Date Created: Aug 2003)
(Norway)
Reviewed by: 3rd ECDL Workshop on Web Archiving
Has papers: Implementing Preservation Strategies for Complex Multimedia Objects
- An overview, programme and links to presentations given at the Workshop. The full Workshop proceedings are available as a PDF file. Individual papers available in word format from the website include- Nordic Web Archive by Þorsteinn Hallgrímsson and Sverre Bang ; Building Thematic Web Collections : Challenges and Experiences from the September 11 Web Archive and the Election 2002 Web Archive by Steven M. Schneider, Kirsten Foot, Michele Kimpton and Gina Jones ; Paradigma Web Harvesting Environment by Ketil Albertsen ; Characterization of the Portuguese Web by Daniel Gomes and Mário J. Silva ; Political Communications Web Archiving by Bernard Reilly ; Archiving the Czech Web : Issues and Challenges by Petr Žabicka; Identification of Network Accessible Documents: Problem Areas and Suggested Solutions (based on the Paradigma Project) by Carol van Nuys, Ketil Albertsen.
- http://bibnum.bnf.fr/ecdl/2003/index.html
- 4th International Web Archiving Workshop (IWAW 04)
16 Sep 2004
, University of Bath, England
(United Kingdom)
Reviewed by: 4th International Web Archiving Workshop
Has papers: Archiving the Slovenian Web : Recent Experiences
; Archiving the Greek Web
; Introduction to Heritrix : an Open Source Archival Quality Web Crawler
; PANDORA Digital Archiving System (PANDAS): Managing Web Archiving in Australia : A Case Study
; Dominos : a New Web Crawler's Design
; Sampling the Umich.edu Domain
; Language Engineering Techniques for Web Archiving
; New Data Storage and Service Model of China Web InfoMall
- This workshop was part of the 8th European Conference on Research and Advanced Technologies for Digital Libraries. The conference website contains links to the presented papers.
- http://www.iwaw.net/04/
- Archiving Web Resources : Issues for Cultural Heritage Institutions : International Conference
09 - 11 Nov 2004
, National Library of Australia, Canberra
(Australia)
Reviewed by: Conference report : Archiving Web Resources International Conference: Issues for Cultural Heritage Organisations
- Links to presentations are available from the website.
- http://www.nla.gov.au/webarchiving/
- AusWeb 2004
03 - 07 Jul 2004
, Seaworld Nara Resort, Gold Coast, Queensland
(Australia)
- Includes some papers on web archiving, repositories
- http://ausweb.scu.edu.au/aw04/papers/index.html
- Some full text papers and presentations available from conference website.
- DPC Forum on Web Archiving
12 Jun 2006
, British Library Conference Centre, London
(United Kingdom)
- This one day forum showcased the UK Web Archiving project as well as some European initiatives (IIPC and European Web Archive). Technical solutions and legal issues were examined and debate and discussion centered around different strategies and methodologies. Presentations in PDF are available from the website.
- http://www.dpconline.org/graphics/events/060612web-archiving.html
- Future of Today’s Scholarship: A Symposium in Honor of Bob Oakley
25 Jul 2009
Georgetown Law Library
(United States of America)
- This one day symposium held at the Georgetown Law Library focused on the research value of blogs, the reliability of legal documents uploaded to blogs , and blog preservation. It concluded with a session on guidelines for working with blogs now and in the future for librarians, bloggers, preservationists and researchers. Video webcasts and presentations available from the schedule link.
- http://www.ll.georgetown.edu/ftls/
- IIPC Open day: Active Solutions for Preserving Internet Content
07 Oct 2009
, California Mission Bay Conference Center, San Francisco, California
International Internet Preservation Consortium (IIPC)
(United States of America)
- This open day to be held in conjunction with IPRES 2009 (5-6 October, San Francicsco, California) is organised by the the International Internet Preservation Consortium (IIPC) will highlight areas of interest to the IIPC and work on web archiving solutions.
- http://netpreserve.org/events/ActiveSolutions.php
- International Conference on Preservation of Digital Objects (iPRES) 2005
15 - 16 Sep 2005
, Zentrales Hörsaalgebäude (ZHG), Room 103, 1st floor, Platz der Goettinger Sieben 5, 37073 Goettingen, Germany
(Germany)
- Focus areas of the conference were preservation policies, technical workflows and web archiving as well as recent developments. Links to abstracts, slides and video streams of the presentations are now available from programme link.
- http://rdd.sub.uni-goettingen.de/conferences/ipres05/home
- IWAW 05 : International Web Archiving Workshop and Digital Preservation
22 - 23 Sep 2005
, Vienna
(Austria)
- This was held in conjunction with the 9th European Conference on Research and Advanced Technologies for Digital Libraries. There are links to full papers on the IIPC (International Internet Preservation Consortium), digital preservation, audio and video web archiving and current projects and issues.
- http://www.iwaw.net/05/
- IWAW 2007: 7th International Web Archiving Workshop
23 Jun 2007
, Vancouver, Canada
(Canada)
- Held in conjunction with ACM IEEE Joint Conference on Digital Libraries, this workshop included papers on web archiving, access and case studies. It also included updates and demonstrations from the Internet Archive, The California Digital Library and Hanzo Archive.
- http://www.iwaw.net/07/
- Papers and presentations available from the conference web site.
- IWAW 2008 : 8th International Web Archiving Workshop
18 - 19 Sep 2008
, Aarhus Denmark
(Denmark)
Has papers: Legal Deposit of the French Web: Harvesting Strategies for a National Domain
; Identification and Archiving of the Czech Web Outside the National Domain
; Ethical Issues in Web Archive Creation and Usage - Towards a Research Agenda
; Developing National Taiwan University Web Archiving System
; Introducing the Portuguese Web Archive Initiative
; Terminology Evolution in Web Archiving: Open Issues
; Fast Browsing of Archived Web Contents
; Using Page Histories for Improving Browsing the Web
; Web Spam: A Survey with Vision for the Archivist
- Workshop held in conjunction with ECDL 2008 to provide "a cross domain overview on active research and practice in all domains concerned with the acquisition, maintenance and preservation of digital objects for long-term access, with a particular focus on web archiving and studies on effective usage of this type of archives". Papers and presentations are available from the website.
- http://www.iwaw.net/08/
- IWAW 2009: 9th International Web Archiving Workshop
30 Sep - 01 Oct 2009
, Corfu
(Greece)
- This workshop to be held in conjunction with ECDL 2009 (13th European Conference on Digital Libraries, September 27 to October 2 2009,Corfu Greece) and will provide a cross domain overview on active research and practice in all domains concerned with the preservation of the Web.
- http://iwaw.net/09/
- Joint Workshop on Future-proofing Institutional Websites
19 - 20 Jan 2006
, Wellcome Library, London
(United Kingdom)
- Key themes include an overview of international curation and preservation activities; tools and techniques to create and persistently identify website content for curation and preservation; and examples of real-life experiences in web archiving. Powerpoint presentations available from conference web page.
- http://www.dcc.ac.uk/events/fpw-2006/
- PICNIC 06 : Avoiding the Digital Memory Loss
27 Sep 2006
, Westergasfabriek, Amsterdam
(Netherlands)
- A half day seminar held in conjunction with PICNIC 06.
- http://www.europarchive.org/picnic06.php
- Present Becomes the Past: - Harvesting Archiving, Presenting Today’s Digitally Produced Newspapers: IFLA Newspapers Conference at the National Library of Sweden
19 - 20 Aug 2009
, National Library of Sweden, Stockholm
Kungliga Biblioteket (Sweden)
(Sweden)
- This two day conference is organised by the IFLA Newspapers Section and the National Library of Sweden (Kungliga Biblioteket). It aims to share knowledge about how libraries can tackle the challenges of harvesting, archiving and presenting today’s digitally produced newspapers, and to open up opportunities for libraries, librarians and associated publishing industry players to interact and develop suitable partnerships to improve the provision of newspaper services at libraries. It will consider born digital newspaper production and newspapers on the web.
- http://www.kb.se/english/about/news/Present-past/
Top
Organisations and Websites
|
- 2004 Presidential Term Web Harvest
National Archives and Records Administration
(Date Created: Jan 2005)
(United States of America)
- This web archive project conducted by the National Archives and Records Administration documents United States government web sites at the time of the ending of the Presidential term 2004/2005. The approximately 75 million web pages in the archive were gathered between October and November 2004.
- http://www.webharvest.gov/collections/peth04/
-
2nd ECDL Workshop on Web Archiving - in conjunction with the 6th European Conference on Digital Libraries (ECDL 2002)
19 Sep 2002
, Pontifical Gregorian University, Rome
(Italy)
Reviewed by: ECDL 2002 Workshop Report: Web Archiving
; 2nd ECDL Workshop on Web Archiving
- An overview, programme and links to presentations given at the Workshop are available. Presentations include - Saving the Web ; Archiving the Deep Web ; Using METS for Web Archiving ; Domain UK : Britain and the Web.
- http://bibnum.bnf.fr/ecdl/2002/index.html
-
3rd ECDL Workshop on Web Archives : in Conjunction with the 7th European Conference on Research and Advanced Technologies for Digital Libraries (ECDL 2003)
21 Aug 2003
, Trondheim, Norway
(Date Created: Aug 2003)
(Norway)
Reviewed by: 3rd ECDL Workshop on Web Archiving
Has papers: Implementing Preservation Strategies for Complex Multimedia Objects
- An overview, programme and links to presentations given at the Workshop. The full Workshop proceedings are available as a PDF file. Individual papers available in word format from the website include- Nordic Web Archive by Þorsteinn Hallgrímsson and Sverre Bang ; Building Thematic Web Collections : Challenges and Experiences from the September 11 Web Archive and the Election 2002 Web Archive by Steven M. Schneider, Kirsten Foot, Michele Kimpton and Gina Jones ; Paradigma Web Harvesting Environment by Ketil Albertsen ; Characterization of the Portuguese Web by Daniel Gomes and Mário J. Silva ; Political Communications Web Archiving by Bernard Reilly ; Archiving the Czech Web : Issues and Challenges by Petr Žabicka; Identification of Network Accessible Documents: Problem Areas and Suggested Solutions (based on the Paradigma Project) by Carol van Nuys, Ketil Albertsen.
- http://bibnum.bnf.fr/ecdl/2003/index.html
- Archive-it
Internet Archive
(Regularly Updated)
- Archive-it is a subscription service established by the Internet Archive together with various partners from state archives and university libraries. It allows subscribers to build, manage and search their own unique web archive containing only the content they are interested in harvesting. This allows these collections to be more in depth and focussed on a particular area than the general Web archive. Archive-It collections have more functionality as they can be searched in full text, whereas the Internet Archive via the Wayback Machine, cannot be searched in this way. The website provides tours of the user interface and applications, an extensive list of FAQs and access to the archived collections of the pilot partners.
- http://www.archive-it.org/
- Archiving the Web Sites of Political Parties in Germany : a Joint Project of the Archives of Political Foundations Funded by the Deutsche Forschungsgemeinschaft : Politisches Internet Archiv
Archiv de sozialen Demokratie der Friedrich-Ebert-Stiftung
(Date Created: 2006)
(Germany)
- Although this site is mostly in German there are some links to English reosources and similar projects.
- http://www.fes.de/archiv/spiegelung/default.htm
- BOA: Baden-Württembergisches Online-Archiv
(Regularly Updated)
(Germany)
- Established in 2006, the Baden-Württemberg Online Archive (BOA), collects and catalogues web resources on the German State of Baden-Württemberg, comprising a large collection of government websites. All resources are available online. The system is run by the Bibliotheksservice-
Zentrum (Library Service Centre) Baden-Württemberg (BSZ), a support institution for libraries and archives. Two state libraries the Badische Landesbibliothek in Karlsruhe, and the Württembergische Landesbibliothek in Stuttgart, and the archive Landesarchiv Baden-Württemberg cooperate in archiving websites in this system.
- http://www.boa-bw.de
- CyberCemetery
Government Documents Dept., University of North Texas Libraries
(Regularly Updated)
(United States of America)
- This website was established in 2000 as a partnership between the University of North Texas Libraries and the U.S. Government Printing Office. It aims to provide permanent public online access to the web sites and publications of defunct U.S. government agencies and commissions. The site is arranged alphabetically and by category.
- http://govinfo.library.unt.edu/
- Da Blog (ULCC Digital Archives Blog)
University of London Computer Centre Digital Archives Department
(Regularly Updated)
(United Kingdom)
- This blog by the University of London Computer Centre Digital Archives Department includes discussion on recent developments and events and thoughts on digital preservation and archiving.
- http://dablog.ulcc.ac.uk/
- Documenting Internet2 : a Collaborative Model for Developing Electronic Records Capabilities in the Small Archival Repository
Charles Babbage Institute
(United States of America)
- This 18 month project, which began in January 2004, administered by the Charles Babbage Institue (CBI) in conjunction with the University of Minnesota (UMN) and supported by the National Historical Publications and Records Commission, the funding arm of the National Archives. The purpose of the project was to develop feasible methods for selection, description, and long term preservation of historically significant born-digital records in the subject area of history of information technology. Internet2 is a non-profit consortium of over 200 universities working with industry and government to provide high speed networking capabilities to the research community, and relies heavily on electronic media. The final report was issued in March 2006 and is available from this website.
- http://www.cbi.umn.edu/documentinginternet2/index.html
- European Archive
European Archive
(Regularly Updated)
- The European Archive is a non profit initiative. It is a digital library of European material such as sound, motion pictures and web pages. It provides free access to researchers, historians, scholars, and the general public. The Eurpean Archive will develop a large-scale archiving architecture and infrastructure for the storage, access and preservation of digital objects. The European Archive has a partnership with the Internet Archive.
- http://www.europarchive.org/
- Federal Web Harvests
U.S. National Archives & Records Adminstration
(Regularly Updated)
(United States of America)
- This site currently has links to the web harvests of the United States 110th Congress (2008), 109th Congress (2006) and the Presidential Term 2004. NARA's intention is to harvest Federal web sites at the end of each presidential term (4 years) and each Congress (2 years).
- http://www.webharvest.gov/collections/
- Government of Canada Web Archive
Library and Archives Canada
(Regularly Updated)
(Canada)
- Library and Archives Canada (LAC) launched this website in Fall 2007. The website allows searching of web pages harvested since December 2005 from the web domain of the Federal Government of Canada.
- http://www.collectionscanada.gc.ca/webarchives/index-e.html
- English and French versions available.
- HANZO Web : a Social Web Archiving Service
HANZO Archives Limited
(Regularly Updated)
- Up to this time the archiving of websites is mainly in the sphere of national and international organisations. This is an interesting site as it allows personal web archiving at nominated intervals of selected pages, links and websites via registration. The site guarantees permanent access and content is stored in open web archive standards-based formats.
- http://www.hanzoweb.com/
- Internet Archive
Kahle, Brewster
(Last Updated: 3 Mar 1998)
(United States of America)
- A non-profit commercial venture, the Internet Archive is collecting and storing public materials from the Internet such as the World Wide Web, Netnews, and downloadable software which have been donated by Alexa Internet. This collection will include all publicly accessible World Wide Web pages, the Gopher hierarchy, the Netnews bulletin board system, and downloadable software. Archived web pages may be accessed using the Wayback Machine interface. Also preserved at this site is access to films documenting 20th century North American life and culture, digitised from the Prelinger Archives of ephemeral films in San Francisco.
- http://www.archive.org/
- Israeli Internet Sites Archive
Jewish National & University Library
(Regularly Updated)
(Israel)
- This project is part of a research project at the Department of Information Studies of Bar-Ilan University and is supported by the Jewish National & University Library of Israel (JNUL). The first phase of the Pilot Project utilised the selective approach to archiving the Israeli web. Approximately twenty sites were initially selected and these sites were captured monthly between December 2003 and May 2004 by using off-the-shelf capturing software. The results will form the basis of a larger scale pilot which will include selection guidelines and concentrate on providing long term access to these archived sites. A second collection the 17th Knesset Elections 2006 has been added.
- http://www.jnul.huji.ac.il/IA/ArchivedSites/IA/firstpage.html
- IWAW 05 : International Web Archiving Workshop and Digital Preservation
22 - 23 Sep 2005
, Vienna
(Austria)
- This was held in conjunction with the 9th European Conference on Research and Advanced Technologies for Digital Libraries. There are links to full papers on the IIPC (International Internet Preservation Consortium), digital preservation, audio and video web archiving and current projects and issues.
- http://www.iwaw.net/05/
- JISC-PoWR : Preservation of Web Resources : A JISC-sponsored project
JISC-PoWR
(Date Created: 23 Apr 2008)
(United Kingdom)
- This blog was set up to support the JISC-PoWR project which was initiated by the JISC Integrated Information Environment Committee and ran from April to September 2008. It was undertaken jointly by UKOLN at Bath University and ULCC Digital Archives department. The project aimed to “organise workshops and produce a handbook that specifically addresses digital preservation issues that are relevant to the UK HE/FE web management community”. Links to the handbook and presnetations at events which were held June, July & September 2008 are included. The blog has continued after the finalisation of the project with posts relevant to the the project aims.
- http://jiscpowr.jiscinvolve.org/
-

KB/IBM Long Term Preservation Study
Koninklijke Bibliotheek
(Date Created: Dec 2002)
(Netherlands)
- This website documents the Koninklijke Bibliotheek's Long Term Preservation Study, which was undertaken as part of the Project Depot van Nederlandse Elektronische Publicaties (DNEP) in association with IBM. The Study, which ran from November 2000 to December 2001, examined issues relating to the long term preservation of digital objects, particularly as they impact upon the DNEP (the KB's deposit system for electronic materials which is currently under construction). In December 2002 six reports describing the results of the study were published - The Long-Term Preservation Study of the DNEP Project - an Overview of the Results ; Authenticity in a Digital Environment ; Preservation Requirements in a Deposit System ; The UVC: a Method for Preserving Digital Documents – Proof of Concept ; Managing Media Migration in a Deposit System ; Archiving Web Publications. All are available from the website.
- http://www.kb.nl/hrd/dd/dd_onderzoek/dnep_ltp_study-en.html
- Reports also available from the IBM website - http://www-5.ibm.com/nl/dias/preservation.html
- MetaArchive
MetaScholar Initiative
(Regularly Updated)
(United States of America)
- This project commenced in 2004 and was aimed at the preservation of digital content with the subject focus of southern United States culture and history. The original project had six partner institutions in collaboration with the National Digital Information Infrastructure and Preservation Program (NDIIPP) and Library of Congress. The goals of the project included the creation of a conspectus of digital content held by partner sites, a harvested body of the most critical content to be preserved, development of a model cooperative agreement for ongoing collaboration and the implementation of a distributed preservation network infrastructure based on the LOCKSS system. The type of content to be preserved included digital masters of scanned images, research databases and websites. Since 2007 the MetaArchive Cooperative membership has been extended. It is now "responsible for preserving Member organizations’ content in a decentralized, distributed preservation network comprised of subject-based archives (e.g., Southern Digital Culture, Electronic Theses and Dissertations, etc.), as well as maintaining and extending its methodology and approach to distributed digital preservation."
- http://www.metaarchive.org/
-
MINERVA : Mapping the INternet Electronic Resources Virtual Archive
Library of Congress
(Date Created: 2001)
(United States of America)
- The MINERVA Web Preservation Project, developed by the US Library of Congress, was established to initiate a broad program to collect and preserve primary resources that are 'born digital' and not available in any other format. The site has links to various subject based archives such as the September 11 archive. The final report was issued in 2002. Minerva was the early development project for the Library of Congress Web Archives.
- http://www.loc.gov/minerva/
- mod_oai
Old Dominion University
(Regularly Updated)
(United States of America)
- Mod-oai is a joint project of the Old Dominion University Digital Library Research Group, the Los Alamos National Laboratory and is funded by the Mellon Foundation. The aim of the project is to make the content from Apache Web servers accessible via the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). Web servers and Web crawlers would be able to selectively harvest from data repositories, for instance data gathering could be limited to files of a certain mimetype. The website provides an overview of the project including links to relevant organisations and describes members of the project team.
- http://www.modoai.org/
- Netpreserve.org : International Internet Preservation Consortium
International Internet Preservation Consortium (IIPC)
(Regularly Updated)
- The International Internet Preservation Consortium (IIPC) began in July 2003 with a group of twelve members including the national libraries of Australia, Canada, Denmark, Finland, France, Iceland, Italy, Norway, Sweden, The British Library (UK), The Library of Congress (USA) and the Internet Archive (USA) and was coordinated by the Bibliotheque nationale de France. The membership has since expanded to include additional libraries, archives, museums and cultural institutions from around the world. The Consortium goals include advocacy for internet archiving initiatives and legislation, fostering development of common standards and internet archiving tools, and raising international awareness of internet preservation issues.
- http://netpreserve.org/about/index.php
-
NWA : Nordic Web Archive
Nordic Web Archive (NWA)
(Last Updated: 31 Aug 2005)
- The Nordic Web Archive (NWA) was a cooperative venture between the Nordic National Libraries to develop open source software to access archived web documents. The article includes links to more detailed descriptions of WERA (which replaced the NWA toolset) and Nutchwax search engine.
- http://nwa.nb.no/
- OASIS (Online Archiving & Searching Internet Sources)
National Library of Korea
(Regularly Updated)
(Korea)
- Website of the National Library of Korea's web archive, OASIS.
- http://www.oasis.go.kr/
- Our Digital Island: A Tasmanian Web Archive
State Library of Tasmania
(Regularly Updated)
(Australia)
- Our Digital Island provides access to Tasmanian Web sites that have been preserved for posterity by the State Library of Tasmania, as required by the legal deposit provisions of the Libraries Act 1984. It includes sections on: Selection policy, Metadata development, and a description of the project.
- http://odi.statelibrary.tas.gov.au/
-

PANDORA - Preserving and Accessing Networked Documentary Resources in Australia
National Library of Australia
(Regularly Updated)
(Australia)
- PANDORA began as a project initiated by the National Library of Australia to investigate strategies and implement procedures for the capture, storage, preservation and access to digital data by creating an electronic archive of library materials. It is interested in capturing the full range of materials published online in Australia - including serials, newspapers and books, as well as unique online formats such as homepages (see also Digital Services Project). PANDORA is Australia's web archive managed by the Library with contributions from partners.
- http://pandora.nla.gov.au/
- Tomba : Portuguese Web Archive
Faculdade de Ciências, Universidade de Lisboa
(Regularly Updated)
(Portugal)
- This prototype was developed as a research project at the University of Lisbon. The tumba! search engine has been crawling the Portuguese web since 2002. Tomba archives and provides access to all new pages and the different versions of previous pages.
- http://tomba.tumba.pt/index_en.html
- UK Government Web Archive
The National Archives (UK)
(Regularly Updated)
(United Kingdom)
- This archive started as a one year project comprised of a selection of 50 UK government websites archived since August 2003 at varying intervals by the Internet Archive on behalf of The National Archives. Evaluation of the results of the project were used to inform policy and best practice for future archiving of government websites by The National Archives. Since 2005 archiving has been undertaken by contract to the European Archive. In November 2008 a program to archive UK central government websites 3 times per year was started. Additionally sites reflecting topical themes and issues of public interest have been collected into themed collections.
- http://www.nationalarchives.gov.uk/preservation/archivedwebsites.htm
- UK Web Archiving Consortium
United Kingdom Web Archiving Consortium (UKWAC)
(Date Created: Jun 2004)
(United Kingdom)
- The UK Web Archiving Consortium (UKWAC) comprises 6 UK institutions including the British Library, JISC (Joint Information Systems Committee), National Archives, Wellcome Trust and the National Libraries of Scotland and Wales. The project will run for 2 years during which approximately 6,000 websites will be collected and archived. PANDAS software, developed by the National Library of Australia, will be used to carry out the archiving process. The purpose of the project is not only to collect and archive web sites that are relevant to the Consortium’s interests, but to develop compatible selection policies as well as to investigate the complex technical challenges involved in collecting and archiving web material.
- http://www.webarchive.org.uk/
- Vefsafn.is
National and University Library of Iceland
(Date Created: 30 Sep 2009)
(Iceland)
- The Icelandic Web Archive owned and maintained by the National and University Library of Iceland, has been collecting snapshots of Icelandic websites since 2004. It was opened for public access in September 2009. Access to the collection is via a Wayback Machine like the one used by the Internet Archive.
- http://vefsafn.is/index.php?page=english
-

Virtual Remote Control
Cornell University
(United States of America)
- Virtual Remote Control (VRC) was a research initiative to develop a risk management approach to providing access to Web resources over time, including setting up a model, framework as well as identifying tools that can help institutions deal with the web material. VRC model can include the capturing of web sites, but that is not its essential component, since organisations using it may wish only to monitor web sites due to possible legal technical and other limitations. Site includes evaluations of available web tools. VRC emerged from Cornell University's participation in Project Prism.
- http://irisresearch.library.cornell.edu/VRC/index.html
- WARP
National Diet Library (Japan)
(Regularly Updated)
(Japan)
- This is a National Diet Library of Japan project. WARP is an experimental project that collects and preserves Japanese websites and online periodicals on the Internet. Most of the information is available in Japanese only.
- http://warp.ndl.go.jp/
- Web Archive Collection Service (WAX)
Harvard University Library
(Date Created: 04 Feb 2009)
(United States of America)
- The Web Archive Collection Service ( WAX) at Harvard University Library began as a pilot service in 2006 and became a production service in February 2009. The service supports the collection of selected web content to ensure its long term preservation and accessibility for teaching and research. Harvard libraries, museums and archives are eligible to use the WAX Service. The public service was made available September 2009. More information available from http://hul.harvard.edu/ois/systems/wax
- http://wax.lib.harvard.edu
- Web Archive Singapore (WAS)
National Library Board Singapore
(Regularly Updated)
(Singapore)
- The website of Web Archive Singapore was officially launched in October 2006 by the National Library Board of Singapore. Phase 1 of the project was aimed at archiving .sg websites meeting selection criteria of national and historical significance, authoritative and research value and social and cultural significance.
- http://was.nl.sg/
- Web Archivering / Web Archiving
Koninklijke Bibliotheek
(Netherlands)
- This web site describes the Koninklijke Bibliotheek's web archiving program. The KB began selective archiving in 2006. The project is described, and there are links to information about selection, technical and legal aspects and related documents.
- http://www.kb.nl/hrd/dd/dd_projecten/webarchivering/index.html
- Available in Dutch and English versions (http://www.kb.nl/hrd/dd/dd_projecten/webarchivering/index-en.html)
-
Web Archives : Yesterday's Web; Today's Archives
California Digital Library
(Regularly Updated)
(United States of America)
- This website (which was opened to the public in July 2009) includes web archives created with the California Digital Library's Web Archiving Service, and includes web archives and documents from the State of California archived by ten institutions.
- http://webarchives.cdlib.org/
- Web Archiving
Library of Congress
(Date Created: May 2006)
(United States of America)
- This web site, developed and maintained by the Library of Congress, includes information on its program to capture and preserve historically important Web sites such as US congressional website, Hurricane Katrina and the Iraq War. The website has links to technical information, guidelines for the harvesting process, FAQs and the Project partners. The Web Archiving Program, formerly called the Web Capture Program, and its partners are developing a common set of Web capture tools in four areas: curator selection, verification and permissions; acquisition; collection storage and maintenance; and access.
- http://www.loc.gov/webarchiving/
- Web Archiving @ the University of Melbourne
University of Melbourne
(Regularly Updated)
(Australia)
- This is the University of Melbourne's web site which describes its corporate web archiving activities. Included are links to related policies, an overview of the program, technical aspects and a search facility for available content.
- http://www.unimelb.edu.au/records/web-archiving/
- Web InfoMall
Computer Networks and Distributed Systems Laboratory , Beijing University
(Regularly Updated)
(China)
- InfoMall is the Chinese web archiving project. It currently holds 0.7 billion pages (10.6 terabyte) together with 5 terabyte of digital web resources other than web pages. It has a storage capacity to hold more than 10 billion pages (150 terabytes). Its main tasks are to permanently collect web pages into a repository, provide free access to the archived data and provide an open testbed to encourage multidisciplinary study. Most of the site is in Chinese, although the search interface has an English version.
- http://www.infomall.cn/index-eng.htm
- WebArchiv - Archive of the Czech Web
National Library of the Czech Republic
(Regularly Updated)
(Czech Republic)
- The Archive of Czech web resources, known as WebArchiv, was established to selectively archive the Czech domain based on a voluntary deposit agreement with authors and publishers. There are links to documentation (mainly in Czech) and selection criteria.
- http://en.webarchiv.cz/
- WebArchivist.org
University of Washington and SUNY Institute of Technology
(Regularly Updated)
(United States of America)
- Based at the University of Washington and the SUNY Institute of Technology this organisation focuses on research and software development. The organisation develops tools and strategies for studying the ephemeral Web.
- http://www.webarchivist.org/about.htm
Top
Policies, Strategies & Guidelines
|
-
Archival Preservation of Smithsonian Web Resources: Strategies, Principles, and Best Practices
Dollar Consulting
(Last Updated: 18 Oct 2001)
(United States of America)
- Commissioned by the Smithsonian Institution Archives, this report provides guidelines to help ensure the capture, management and long-term preservation of Smithsonian Institution web sites. Incorporating an integrated records life cycle process model, it provides a set of recommendations for best practice and follow up, plus appendices on documentation of web sites and a preservation metadata model.
- http://siarchives.si.edu/pdf/dollar_report.pdf
-
Archiving Smithsonian Websites : an Evaluation and Recommendation for a Smithsonian Institution Archives Pilot Project
Smithsonian Institution Archives Records Management Team
(Date Created: May 2003)
(United States of America)
- This report outlines a proposal for a pilot project to archive Smithsonian websites and is the result of the evaluation and testing of Dollar Consulting's report Archival Preservation of Smithsonian Web Resources: Strategies, Principles, and Best Practice Archival Preservation of Smithsonian Web Resources: Strategies, Principles, and Best Practice, published in 2001. It includes proposed recommendations for archival management, access to archived websites, processing dynamic websites and cost estimates, and notes the results of test migrations of HTML to XHTML, which were based on Dollar Consulting's earlier recommendations.
- http://siarchives.si.edu/pdf/websitepilot.pdf
-

Archiving Web Resources : a policy for keeping records of web-based activity in the Commonwealth Government
National Archives of Australia
(Last Updated: Jan 2001)
(Australia)
- A document setting out the National Archives of Australia's policy on the status and management of Commonwealth Government online resources, including websites, as Commonwealth records. It includes Best Practice recommendations plus listings of additional sources to assist Commonwealth agencies in establishing mechanisms for creating, managing and retaining web-based records. Ver 2 revised Jan 2001 re-endorsed 2007.
- ISBN: 06423442213
- http://www.naa.gov.au/records-management/publications/Archiving-web-po
licy.aspx
- PDF of ver. 2 available from this URL
-

Archiving Web Resources: Guidelines for keeping records of web-based activity in the Commonwealth Government
National Archives of Australia
(Date Created: Mar 2001)
(Australia)
- The companion guidelines to the policy "Archiving Web Resources: A policy for keeping records of web-based activity in the Commonwealth Government", this document offers specific strategic and technical advice to help Commonwealth agencies establish good record-keeping strategies for web-based activities.
- ISBN: 06423442213
- http://www.naa.gov.au/records-management/publications/Archiving-web-gu
idelines.aspx
- Available as a .pdf from this URL
- Collection Development Policy for UK Websites
British Library
(Date Created: 03 Sep 2004)
(United Kingdom)
- This document outlines the British Library's policy for collection of UK websites including the context, the aims and how the aims will be achieved.
- http://www.bl.uk/reshelp/pdfs/modbritcdpwebsites.pdf
- Collections Policy Statement : Web Site Capture & Archiving
Library of Congress
(Date Created: 09 Mar 2004)
(United States of America)
- This is the US Library of Congress collections development policy on web sites. It provides a broad overview of aspects of collection development including definitions, scope, criteria for selection and retention.
- http://www.loc.gov/acq/devpol/webarchive.html
- Definition of an Event in Terms of Net Harvesting
Netarchive.dk
(Date Created: Apr 2007)
(Denmark)
- This document provides guidelines for NetArchive.dk undertaking harvesting of internet materials, covered by the Danish Legal Deposit Law. NetArchive.dk uses three strategies: 1. Snapshot harvesting (all relevant domains) quarterly; 2. Selective harvesting (appr. 80 domains) – frequency from one hour to 1 month; and 3. Event harvesting. This document provides a definition of what constitutes an event.
- http://netarchive.dk/publikationer/Event-definition-final.pdf
-

e-Depot and Digital Preservation
Koninklijke Bibliotheek
(Netherlands)
- This web page is an introduction to, and the entry point for, the e-Depot at the Koninklijke Bibliotheek (KB), the National Library of the Netherlands, via which the KB aims to preserve and provide long-term access to Dutch electronic publications. There are also links to the broader topic of digital preservation.
- http://www.kb.nl/hrd/dd/index-en.html
- Guide to Developing Recordkeeping Strategies For Websites : Record Keeping Guide G20.
Archives New Zealand
(Last Updated: Jun 2006)
(New Zealand)
- "This guide provides best practice advice for public offices and local authorities to develop strategies to manage their websites as public or local authority records. It articulates key principles for managing websites as records and, based on an analysis of website technology, assesses a range of recordkeeping strategies that organisations can use."
- ISBN: 0478182120
- http://continuum.archives.govt.nz/files/file/guides/g20/g20-introducti
on.html
- Available in HTML and as PDF (2.9 MB)
-

Handling File Formats
Clausen, Lars. R.
(Date Created: May 2004)
(Denmark)
- This report describes a part of “Internetbevaringsprojektet” (the Internet preservation project), which is a joint project between Statsbiblioteket (The State Library) and Det Kongelige Bibliotek (The Royal Library) in Denmark. The aim of the project is to archive the Danish part of the Internet. As part of this project, this report specifically deals with strategies in handling file formats to ensure long term preservation and accessibility. Some of the main ways to preserve access to information stored as digital objects are discussed, including capture, sequential conversion to new formats, conversion on demand, emulation and hardware preservation. The report concludes with a suggested strategy for dealing with file formats in an archival system, file preservation workflows and general recommendations for handling file formats.
- http://netarchive.dk/publikationer/FileFormats-2004.pdf
- Highways Agency Web Archive Policy : Implementation of Web Archive Policy for the Highways Agency
Highways Agency
(Date Created: 2008)
(United Kingdom)
- This report explores archiving requirements for information published on the UK Highways Agency corporate website. It was produced in response to guidance from The National Archives in 2006 and Freedom of Information legislation requirements.
- http://www.highways.gov.uk/aboutus/documents/Highways_Agency_Web_Archi
ve_Policy.pdf
- ISO 28500:2009 - Information and Documentation - WARC File format
International Organization for Standardization
(Date Created: 13 May 2009)
- This standard specifies the WARC file format which is an extension of the ARC format which has been used since 1996 to store files harvested on the web. The WARC format enables the recording of HTTP request headers, the recording of arbitrary metadata, the allocation of an identifier to every contained file, the management of duplicates and migrated records and the segmentation of the records. WARC files are intended to store every type of digital content, either retrieved by HTTP or another protocol.
- http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm
?csnumber=44717
- Draft version available from the BnF at http://bibnum.bnf.fr/WARC/index.html
- Kriteriji odabira obveznog primjerka mrežne građe za obradu i arhiviranje
Nacionalna i sveučilišna knjižnica u Zagrebu
(Croatia)
- Selection guidelines by the National and University Library in Zagreb, Croatia
- http://www.nsk.hr/DigitalLib.aspx?id=83
- NARA Guidance on Managing Web Records
National Archives and Records Administration (NARA)
(Date Created: Jan 2005)
(United States of America)
- These guidelines are intended to assist U.S. agencies manage web records.
- http://www.archives.gov/records-mgmt/policy/managing-web-records-index
.html
- Available in HTML and also in PDF.
- National Library of New Zealand (Te Puna Mātauranga o Aotearoa) Act 2003
New Zealand Government
(Date Created: May 2003)
(New Zealand)
- As part of the legal deposit provisions of this revised legislation, the definition of public documents was extended to include Internet documents, including the whole or parts of web sites. As such the National Library of New Zealand (NLNZ) is authorised to obtain copies of Internet documents as part of its wider responsibility to collect and preserve New Zealand documentary heritage.
- http://www.natlib.govt.nz/files/Act03-19.pdf
- Preservation of Web Resources
Pinsent, Ed
(Date Created: 24 Mar 2009)
(United Kingdom)
- This JISC briefing paper outlines the need for preservation of web resources by UK higher and further education institutions including institutional web sites, collections of learning objects, blogs and wikis. It considers who is and should be undertaking web preservation activities in the UK and the need for preservation policies and coordination between web and digital preservation activities.
- http://www.jisc.ac.uk/publications/documents/bpwebpreservation.aspx
- Also available as PDF JISC Briefing Paper. Document no. 529 http://www.jisc.ac.uk/media/documents/publications/bpwebpres.pdf
- Recordkeeping Strategies for Websites and Web Pages : State Records Guideline No. 15
Archives Office of Tasmania
(Date Created: 14 Dec 2005)
(Australia)
- "This guideline articulates key principles for managing records of websites, webpages (public or internal) and supporting records and provides a range of recordkeeping strategies that agencies can use to manage these as State records." in Tasmania.
- http://www.archives.tas.gov.au/legislative/staterecords/guidelines_lis
t/guideline_15
- Selection Criteria for Web Resources
WebArchiv
(Date Created: 2008)
(Czech Republic)
- This page sets out the selection criteria for web resources collected by WebArchiv, web archive of the Czech Republic.
- http://en.webarchiv.cz/criteria/
- Web-At-Risk Web Collection Plans
Web-At-Risk, California Digital Library
(Regularly Updated)
(United States of America)
- This page on the Web-at-Risk Wiki contains a links to a variety of web collection plans or statements for various organisations. The statements give an overview of the mission & scope, selection acquisition, description, metadata, preservation and access intentions for web harvesting for particular collections.
- https://wiki.cdlib.org/WebAtRisk/tiki-index.php?page=WebCollectionPlan
s
-

Website Archiving Policy : National Library of Australia
National Library of Australia
(Date Created: 23 Jan 2004)
(Australia)
- This document describes the strategies used to ensure that the content accessible through the National Library of Australia's websites is archived in accordance with government policy and guidelines. The strategies cover content made accessible through the Library's public website, intranet, websites hosted by the Library and websites mirrored by the Library. The document also includes the web record archiving workflow.
- http://www.nla.gov.au/policy/webarchive/archstrat.html
Top
Projects and Case Studies
|
-

AOLA : Austrian On-Line Archive
Department of Software Technology, Vienna University of Technology
(Regularly Updated)
(Austria)
- Website of the AOLA (Austrian On-Line Archive) pilot project, a joint initiative of the Austrian National Library and the Technical University of Vienna's Department of Software Technology. AOLA is being constructed as an archive of snapshots of Austrian webspace, based upon periodic harvesting of publicly available websites. The project was established to investigate the challenges associated with the collection and archiving of online publications, which were not covered by the 2000 amendments to legal deposit legislation. This pilot project ran 2001-2002
- http://www.ifs.tuwien.ac.at/~aola/
- ARCHIPOL : Archive of Web Sites of Political Parties in the Netherlands
ARCHIPOL
(Date Created: 2003)
(Netherlands)
- The aim of this project was to archive the web sites of political parties in the Netherlands. There are links to project information such as software and progress reports. The project was designed as a pilot study so that the results could be used to develop a model to help other institutions archive web sites. Included are parties websites for 2002, 2003 and 2006.
- http://www.archipol.nl/english/index.html
- ArchivePress
ULCC (University of London Computer Centre)
(Date Created: Jun 2009)
(United Kingdom)
- This joint project by ULCC (University of London Computer Centre and the British Library is financed by JISC (1 June 2009 to 30 Nov 2009). It will explore practical issues around the archiving of weblog content and will test the viability of using RSS feeds and blog APIs to harvest blog content (including comments, embedded content and metadata) instead of web crawling & harvesting techniques.
- http://archivepress.ulcc.ac.uk/
-
Archiving Smithsonian Websites : an Evaluation and Recommendation for a Smithsonian Institution Archives Pilot Project
Smithsonian Institution Archives Records Management Team
(Date Created: May 2003)
(United States of America)
- This report outlines a proposal for a pilot project to archive Smithsonian websites and is the result of the evaluation and testing of Dollar Consulting's report Archival Preservation of Smithsonian Web Resources: Strategies, Principles, and Best Practice Archival Preservation of Smithsonian Web Resources: Strategies, Principles, and Best Practice, published in 2001. It includes proposed recommendations for archival management, access to archived websites, processing dynamic websites and cost estimates, and notes the results of test migrations of HTML to XHTML, which were based on Dollar Consulting's earlier recommendations.
- http://siarchives.si.edu/pdf/websitepilot.pdf
- Archiving the Web Sites of Political Parties in Germany : a Joint Project of the Archives of Political Foundations Funded by the Deutsche Forschungsgemeinschaft : Politisches Internet Archiv
Archiv de sozialen Demokratie der Friedrich-Ebert-Stiftung
(Date Created: 2006)
(Germany)
- Although this site is mostly in German there are some links to English reosources and similar projects.
- http://www.fes.de/archiv/spiegelung/default.htm
- Arquivo Da Web Portuguesa : Portuguese Web Archive
Fundação para a Computação Científica Nacional
(Date Created: Jan 2008)
(Portugal)
- The Portuguese Web Archive is a project of the National Foundation for Scientific Computing - Fundação para a Computação Científica Nacional (FCCN) in Portugal. The first stage of development began in January 2008 and it is expected to to finish within two years. It was preceded by the Tomba project which collected sites from 2002-2006.
- http://arquivo-web.fccn.pt/
- BOA: Baden-Württembergisches Online-Archiv
(Regularly Updated)
(Germany)
- Established in 2006, the Baden-Württemberg Online Archive (BOA), collects and catalogues web resources on the German State of Baden-Württemberg, comprising a large collection of government websites. All resources are available online. The system is run by the Bibliotheksservice-
Zentrum (Library Service Centre) Baden-Württemberg (BSZ), a support institution for libraries and archives. Two state libraries the Badische Landesbibliothek in Karlsruhe, and the Württembergische Landesbibliothek in Stuttgart, and the archive Landesarchiv Baden-Württemberg cooperate in archiving websites in this system.
- http://www.boa-bw.de
- Chesapeake Project : Legal Information Archive
(Date Created: 2007)
(United States of America)
- The Chesapeake Project began in early 2007 as a two year (2007-2008) digital preservation program established to preserve and provide ongoing access to legal information available on the World Wide Web. It began as a collaboration between the Legal Information Preservation Alliance (LIPA), an independent organization of law libraries supported by the American Association of Law Libraries (AALL), and three LIPA-member libraries: the Georgetown Law Library and the State Law Libraries of Maryland and Virginia. The 3 libraries have expanded the program beyond the pilot phase.
- http://cdm266901.cdmhost.com/
- DACHS: Digital Archive for Chinese Studies
Institute of Chinese Studies, University of Heidelberg
(Regularly Updated)
- Operating since August 2001 and based at the University of Heidelberg, DACHS "aims at identifying, archiving and making accessible Internet resources relevant for Chinese Studies, with special emphasis on social and political discourse as reflected by articulations on the Chinese Internet." Collected resources include websites, discussion boards, journals, newsletters and single documents. On overview of the archive's collection policy, workflow and technical infrastructure is available.
- http://www.sino.uni-heidelberg.de/dachs/
- Delivering Coordinated UK Web Archives to User Communities
The National Archives (UK)
(Date Created: May 2009)
(United Kingdom)
- This study is being undertaken by The National Archives and UK Web Archiving Consortium with funding from JISC and is expected to run until July 2009 with results published in August 2009. It aims to: Investigate how UK web archives are delivered to users now and in the future; Define the long-term historical and research value of online content in the UK; and Examine web archive collecting organisations , their interests and selection practices.
- http://www.nationalarchives.gov.uk/news/stories/308.htm?WT.lp=n-33642
- Documenting Internet2 : a Collaborative Model for Developing Electronic Records Capabilities in the Small Archival Repository
Charles Babbage Institute
(United States of America)
- This 18 month project, which began in January 2004, administered by the Charles Babbage Institue (CBI) in conjunction with the University of Minnesota (UMN) and supported by the National Historical Publications and Records Commission, the funding arm of the National Archives. The purpose of the project was to develop feasible methods for selection, description, and long term preservation of historically significant born-digital records in the subject area of history of information technology. Internet2 is a non-profit consortium of over 200 universities working with industry and government to provide high speed networking capabilities to the research community, and relies heavily on electronic media. The final report was issued in March 2006 and is available from this website.
- http://www.cbi.umn.edu/documentinginternet2/index.html
-

Experiences and Conclusions From a Pilot Study : Web Archiving of the District and County Elections 2001 : Final Report for the Pilot Project "netarkivet.dk"
Brugger, Niels; Carlsen, Soren Vejrup ; Christensen-Dalsgaard, Birte; Finnemann, Niels Ole; Fonss-Jorgensen, Eva; Hendriksen, Birgit; von Hielmcrone, Harald
(Date Created: Feb 2003)
(Denmark)
- This is the final report of a project which began in 2001. The report discusses the issues arising from using existing software to harvest and test material relating to Danish elections, including costs and financing, storage and archiving, methods of harvesting and legal issues.
- http://netarkivet.dk/publikationer/webark-final-rapport-2003.pdf
- JISC-PoWR : Preservation of Web Resources : A JISC-sponsored project
JISC-PoWR
(Date Created: 23 Apr 2008)
(United Kingdom)
- This blog was set up to support the JISC-PoWR project which was initiated by the JISC Integrated Information Environment Committee and ran from April to September 2008. It was undertaken jointly by UKOLN at Bath University and ULCC Digital Archives department. The project aimed to “organise workshops and produce a handbook that specifically addresses digital preservation issues that are relevant to the UK HE/FE web management community”. Links to the handbook and presnetations at events which were held June, July & September 2008 are included. The blog has continued after the finalisation of the project with posts relevant to the the project aims.
- http://jiscpowr.jiscinvolve.org/
-

Kulturarw3
National Library of Sweden
(Regularly Updated)
(Sweden)
- The aim of this project, inaugurated by the The National Library of Sweden, is to test methods of collecting, preserving and providing access to Swedish electronic documents which are accessible on line. This site includes brief descriptions of the scope of collecting and collecting methods, preservation and access considerations, and progress of the Kulturarw3 Heritage Project.
- http://www.kb.se/english/find/internet/websites/
- Also see http://www.kb.se/soka/internet/sv-webbsidor/ for more information in swedish.
- Latin American Government Documents Archive, LAGDA
Latin American Network Information Center at The University of Texas at Austin
(Regularly Updated)
(United States of America)
- The Latin American Government Documents Archive (LAGDA) seeks to preserve and facilitate access to a wide range of ministerial and presidential documents from 18 Latin American and Caribbean countries. The Archive contains copies of the Web sites of approximately 300 government ministries and presidencies. Capture of sites began on multiple dates in 2005 and 2006, and will continue with regularly scheduled captures.
Content in the Archive includes not only the full-text versions of official documents, but also original video and audio recordings of key regional leaders. Archive contents include thousands of annual and "state of the nation" reports; plans and programs; and speeches by presidents and government ministers. Content can be accessed via full-text search (search help), or by browsing by country or by specialized sample collection, such as "Presidential Messages" or "Ministerial Documents."
LAGDA is a joint project of the University of Texas Libraries, The Nettie Lee Benson Latin American Collection, and the Latin American Network Information Center at The University of Texas at Austin. Web archiving services are provided by the Internet Archive's Archive-It service.
- http://lanic.utexas.edu/project/archives/lagda/
- Legal Deposit of the French Web: Harvesting Strategies for a National Domain
Lasfargues, France; Oury, Clément; Wendland, Bert
(Date Created: Sep 2008)
(France)
- This paper delivered at IWAW '08 (September 18–19, 2008, Aarhus, Denmark) describes the Bibliothèque nationale de France (BnF) web harvesting activities in the context of the French Copyright Law August 1 2006, which extended legal deposit to the Internet. It compares the .fr domain harvests conducted over the last four years and discusses collaboration with the Internet Archive.
- http://iwaw.net/08/IWAW2008-Lasfargues.pdf
- LiWA - Living Web Archives
LiWA - Living Web Archives
(Regularly Updated)
- This three year project is aimed at developing the next generation of web archives through creation of innovative methods and services for web content capture, preservation, analysis and enrichment. This European funded project is led by the LS3 Research Center, University of Hanover and brings together researchers, industrial users and archiving organisations.
- http://www.liwa-project.eu/
-

Long-Term Preservation of Digital Material - Building an Archive to Preserve Digital Cultural Heritage from the Internet
Aschenbrenner, Andreas
(Date Created: Dec 2001)
(Austria)
- Submitted as a Diploma thesis, this report presents a detailed case study of the Austrian On-Line Archive and a prototype for automatic retrieval of interactive online documents. A review of long-term digital archiving initiatives, issues and proposed strategies in digital preservation are also included.
- http://www.ifs.tuwien.ac.at/~aola/publications/thesis-ando/
- Also available in PDF or gnu-zipped PostScript ( http://www.ifs.tuwien.ac.at/~aola/publications.html ).
-
Management of Networked Electronic Publications: A Table of Status in Various Countries
Martin, Elizabeth
(Last Updated: Nov 2001)
- A comparison of national libraries in sixteen countries on matters of deposit legislation and arrangements, approach and policy, plans, negotiations with publishers, access arrangements and implementation for networked electronic publications.
- http://epe.lac-bac.gc.ca/100/206/301/lac-bac/consultation_online_publi
cations-ef/8/7/r7-100-e.html
- Also available in PDF and RTF. Available in French at http://nlc-bnc.ca/obj/r7/f2/r7-100-f.pdf
- MetaArchive
MetaScholar Initiative
(Regularly Updated)
(United States of America)
- This project commenced in 2004 and was aimed at the preservation of digital content with the subject focus of southern United States culture and history. The original project had six partner institutions in collaboration with the National Digital Information Infrastructure and Preservation Program (NDIIPP) and Library of Congress. The goals of the project included the creation of a conspectus of digital content held by partner sites, a harvested body of the most critical content to be preserved, development of a model cooperative agreement for ongoing collaboration and the implementation of a distributed preservation network infrastructure based on the LOCKSS system. The type of content to be preserved included digital masters of scanned images, research databases and websites. Since 2007 the MetaArchive Cooperative membership has been extended. It is now "responsible for preserving Member organizations’ content in a decentralized, distributed preservation network comprised of subject-based archives (e.g., Southern Digital Culture, Electronic Theses and Dissertations, etc.), as well as maintaining and extending its methodology and approach to distributed digital preservation."
- http://www.metaarchive.org/
-
MINERVA : Mapping the INternet Electronic Resources Virtual Archive
Library of Congress
(Date Created: 2001)
(United States of America)
- The MINERVA Web Preservation Project, developed by the US Library of Congress, was established to initiate a broad program to collect and preserve primary resources that are 'born digital' and not available in any other format. The site has links to various subject based archives such as the September 11 archive. The final report was issued in 2002. Minerva was the early development project for the Library of Congress Web Archives.
- http://www.loc.gov/minerva/
- mod_oai
Old Dominion University
(Regularly Updated)
(United States of America)
- Mod-oai is a joint project of the Old Dominion University Digital Library Research Group, the Los Alamos National Laboratory and is funded by the Mellon Foundation. The aim of the project is to make the content from Apache Web servers accessible via the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). Web servers and Web crawlers would be able to selectively harvest from data repositories, for instance data gathering could be limited to files of a certain mimetype. The website provides an overview of the project including links to relevant organisations and describes members of the project team.
- http://www.modoai.org/
-
NEDLIB : Networked European Deposit Library
National Library of the Netherlands
(Last Updated: Feb 2001)
- This site is the homepage of NEDLIB, a collaborative project consortium, which was headed by the National Library of the Netherlands (Koninklijke Bibliotheek) and included eight other European national libraries, a national archive and three main publishers. The project finished on 31 January 2001.The main goal was to find ways to preserve access to both online and offline (physical format) digital publications. It aimed to construct the basic infrastructure upon which a networked European deposit library could be built. Project Technical Working Papers are available by following the link 'Working Papers' on the home page.
- http://nedlib.kb.nl/
-

netarchive.dk
Denmark's Electronic Research Library
(Regularly Updated)
(Denmark)
- The netarchive.dk project is a joint-initiative of the Royal Library, The State & University Library and the Centre for Internet Research, University of Aarhus in Denmark. This year long study, which will run from August 2001 to July 2002, will examine the capture and archiving of Danish internet activity relating to the country's November 2001 municipal elections. Project reports will be issued quarterly, and are expected to address some of the issues associated with the collection and preservation of dynamic and interactive resources.
- http://netarchive.dk/index-en.php
-
netarchive.dk : The Pilot Project
Netarchive.dk
(Denmark)
- This was the web page of the Netarchive.dk pilot project which was the first phase (2001-2002) investigating strategies for collecting and archiving Danish Internet materials. This initiative represented a cooperative effort between Danish Royal Library, The State and University Library, and the Centre for Internet Research. Project’s key recommendation was to harvest material by three methods including automatic ‘snapshot’ gathering, ‘selective’ detailed archiving, as well as ‘event-based’ collecting of web sites. The Pilot Project website also has links to the project reports. The second phase focused on provision of legal deposit for digital material. The new netarchive.dk website provides current information.
- http://web.archive.org/web/20060301064321/http://netarchive.dk/pilot-i
ndex-en.php
- Nordic Web Archive
Helsinging yliopiston kirjasto; Kansalliskirjasto
(Scandinavia)
- This article outlines the application for Nordunet2 project, which was submitted by five Nordic National Libraries and featured software developers at every participating National Library. Nordunet2 project was part of the Nordic Web Archive (NWA) initiative. The Nordic Web Archive aims at archiving the whole of the web space of Nordic countries, while the Nordunet2 part of it is focused on developing software for accessing web documents, that is it applies to defining and implementing the Access module and the interface between the Archive and the Access module. The Access module for the NWA will enable the users to search and retrieve documents in the Archive.
- http://www.lib.helsinki.fi/tietolinja/0100/nwa.pdf
- Our Digital Island: A Tasmanian Web Archive
State Library of Tasmania
(Regularly Updated)
(Australia)
- Our Digital Island provides access to Tasmanian Web sites that have been preserved for posterity by the State Library of Tasmania, as required by the legal deposit provisions of the Libraries Act 1984. It includes sections on: Selection policy, Metadata development, and a description of the project.
- http://odi.statelibrary.tas.gov.au/
- PADICAT (Digital Heritage of Catalonia)
Biblioteca de Catalunya
(Regularly Updated)
(Spain)
- The Biblioteca de Catalunya (www.bnc.cat), the National Library of Catalonia, as the institution responsible for compiling, processing and distributing the bibliographic heritage of Catalonia (Spain) initiated in 2005 a web crawling and access project: the Digital Heritage of Catalonia (PADICAT). It consists of collecting, processing and providing permanent access to the entire cultural, scientific and general output of Catalonia in digital format. The objective of PADICAT is to archive Catalan web sites.
PADICAT is based on the application of a number of computer programs that allow web pages published on the Internet to be collected, stored, organized, preserved and permanently accessed. In accordance with the general trend among national libraries, the archive model used by BC is a hybrid system consisting of the following:
- Mass compilation of open-access digital resources published on the Internet
- Systematic archiving of the web site output of Catalan organizations
- Fostering of lines of research through themed integration of the digital resources pertaining to specific events in Catalan public life
By 2009, BC should be in an optimum position, whereby this system -a pioneer in Spain and a benchmark in Europe- operates at full capacity, with quantitative indicators of 100,000 Web pages captured in different editions. This may include some 50 million files and 30 terabytes of data. Furthermore, cooperation agreements are scheduled to be signed with 300 institutions of all kinds and online open access to a considerable part of the collection will be available.
- http://www.padicat.cat/en/index.php
-

PANDORA - Preserving and Accessing Networked Documentary Resources in Australia
National Library of Australia
(Regularly Updated)
(Australia)
- PANDORA began as a project initiated by the National Library of Australia to investigate strategies and implement procedures for the capture, storage, preservation and access to digital data by creating an electronic archive of library materials. It is interested in capturing the full range of materials published online in Australia - including serials, newspapers and books, as well as unique online formats such as homepages (see also Digital Services Project). PANDORA is Australia's web archive managed by the Library with contributions from partners.
- http://pandora.nla.gov.au/
-
Paradigma : Preservation of Digital Material
Project of Paradigma
(Norway)
- Paradigma (Preservation, Arrangement & Retrieval of Assorted DIGital MAterials) was a three year project conducted by the National Library of Norway. The project began in August 2001and was scheduled to end on 31 December 2004. Paradigma's goal was to find the technology, methods and organization for the collection and preservation of electronic documents, and to give the National Library's users access to these documents in compliance with the Legal Deposit Act.
-

Political Communications Web Archive Project
Center for Research Libraries (U.S.)
(Last Updated: Jun 2004)
- This project was undertaken by the Center for Research Libraries (CRL) and is now completed. Its purpose was to ensure the long term availability of web documents by non-government political groups. The project attempted to establish a framework and methodologies on three aspects of archiving- long term resource management, curatorship and technology. This site includes links to the project proposal and final report.
- http://www.crl.edu/archiving-preservation/digital-archives/past-projec
ts/political-communications
-

Preserving Presidential Websites
Gupta, Amarnath
(Last Updated: 18 Jan 2001)
(United States of America)
- A San Diego Super Computer Centre Technical Report (SDSC TR-2001-3) on developing a persistent scalable infrastructure dependent store for the long-term accessibility of presidential websites, which captures the knowledge structure of the digital objects within the representation when archived.
- http://legacy.sdsc.edu/TR/TR-2001-03.pdf
-
Preserving Web History
Mitchell, Robert L
(Date Created: 08 May 2002)
(United States of America)
- Focusing on the USA, this article looks at the challenges faced by a number of companies in relation to website archiving. With companies having statutory and legal obligations to preserve records of past transactions and activities, the need to preserve information on websites has posed a number of difficulties for these organisations. The article also includes a case study of web archiving techniques employed at Corporate Express Inc in the USA.
- http://www.computerworld.com.au/index.php/id;1965761530
- Available in Computerworld, Vol 36, Issue 32, 8 May 2002, pp. 26-28, ISSN 00104841 and by subscription online at http://www.ebscohost.com/
- Report on the Whole Domain Harvest
Koerbin, Paul
(Date Created: 10 Oct 2005)
(Australia)
- This is the report of the National Library of Australia's first whole Australian domain harvest which was conducted during June and July 2005. The crawl was undertaken by the Internet Archive on behalf of the Library. Approximately 185 million unique documents were crawled from 811,000 hosts. The total size of the harvested content is 6.69 terabytes. The main objectives of the project were to observe and analyse the performance of the Heritrix crawl robot, test the ability to automatically crawl a large web domain and analyse the success of the breadth and depth of the crawl, obtain content that could be used as a test bed for approaches to delivering access to content and development of a clear understanding the Australian web domain.
- http://pandora.nla.gov.au/documents/domain_harvest_report_public.pdf
- UK Government Web Archive
The National Archives (UK)
(Regularly Updated)
(United Kingdom)
- This archive started as a one year project comprised of a selection of 50 UK government websites archived since August 2003 at varying intervals by the Internet Archive on behalf of The National Archives. Evaluation of the results of the project were used to inform policy and best practice for future archiving of government websites by The National Archives. Since 2005 archiving has been undertaken by contract to the European Archive. In November 2008 a program to archive UK central government websites 3 times per year was started. Additionally sites reflecting topical themes and issues of public interest have been collected into themed collections.
- http://www.nationalarchives.gov.uk/preservation/archivedwebsites.htm
- UK Web Archiving Consortium
United Kingdom Web Archiving Consortium (UKWAC)
(Date Created: Jun 2004)
(United Kingdom)
- The UK Web Archiving Consortium (UKWAC) comprises 6 UK institutions including the British Library, JISC (Joint Information Systems Committee), National Archives, Wellcome Trust and the National Libraries of Scotland and Wales. The project will run for 2 years during which approximately 6,000 websites will be collected and archived. PANDAS software, developed by the National Library of Australia, will be used to carry out the archiving process. The purpose of the project is not only to collect and archive web sites that are relevant to the Consortium’s interests, but to develop compatible selection policies as well as to investigate the complex technical challenges involved in collecting and archiving web material.
- http://www.webarchive.org.uk/
- Umich.edu Archives Project
University of Michigan. School of Information
(Date Created: 2005)
(United States of America)
- The purpose of this project is to demonstrate the feasibility of archiving an entire website and providing access to the resulting archives. The project features a technical demonstration using Heritrix, Perl, MySQL, & PHP. There are also links to reports, papers, and presentations created by students of the School of Information related to this project (2004-2005)
- http://si.umich.edu/mirror/
- VidArch at SILS
School of Information and Library Science, University of North Carolina
(Date Created: 2006)
(United States of America)
- This project was focused on developing a preservation framework for digital video context by initially applying it to two important digital video collections: the complete series of NASA broadcast educational videos and the complete set of juried ACM SIGCHI videos presented at annual conferences from 1983 to the present. Later focus was on the US Presidential Election of 2008 and Internet services such as YouTube. Bibliography of papers and demonstrations available from the site. This project was an NDIIPP demonstration project and supported by a grant from the National Science Foundation.
- http://www.ils.unc.edu/vidarch/
- Web Archivering / Web Archiving
Koninklijke Bibliotheek
(Netherlands)
- This web site describes the Koninklijke Bibliotheek's web archiving program. The KB began selective archiving in 2006. The project is described, and there are links to information about selection, technical and legal aspects and related documents.
- http://www.kb.nl/hrd/dd/dd_projecten/webarchivering/index.html
- Available in Dutch and English versions (http://www.kb.nl/hrd/dd/dd_projecten/webarchivering/index-en.html)
-

Web Archiving Strategy Project (WASP)
University of Melbourne
(Last Updated: 27 Jul 2004)
(Australia)
- Instigated in 2003, this project was designed to develop a strategy to archive the University of Melbourne's web pages. The strategy included identifying web pages for short and long term retention, risk management, defining the roles and responsibilities of the various stakeholders and uniformity of metadata creation. It led to the development of the Web Archiving @ the University of Melbourne web site.
- http://pandora.nla.gov.au/pan/49714/20050927-0000/www.unimelb.edu.au/w
eb-archiving/index.html
- Website archived by PANDORA
- Web at Risk : a Distributed Approach To Preserving Our Nation's Political Cultural Heritage
California Digital Library
(Date Created: 2005)
(United States of America)
- Following a grant from the NDIIPP (National Digital Information Infrastructure and Preservation Program), the California Digital Library and its partners are developing a distributed approach to preserving political and government web pages. The 4 and a half year project is developing open source web archiving tools to capture, curate, and preserve collections of web-based government and political information. The project has four key areas - Content identification and selection, Content acquisition, Content retention and transfer and Partnership building.
- http://www.cdlib.org/inside/projects/preservation/webatrisk/
- Web Curator Tool Project
(Regularly Updated)
(New Zealand; United Kingdom)
- This is the Sourceforge web page for the Web Curator Tool Project, a Web harvesting management system. The project is a collaboration between the National Library of New Zealand and the British Library and was initiated by the International Internet Preservation Consortium (IIPC). Software was developed by Sytec Resources.
- http://webcurator.sourceforge.net/
- Web Harvesting White Paper
(Date Created: 14 Feb 2007)
(United States of America)
- The white paper reports on the context of the results of the EPA Web Publication Harvesting Pilot Project, lesson learned about automated web publication discovery and harvesting technologies and planned further steps.
- http://www.fdlp.gov/home/repository/doc_details/543-web-harvesting-whi
te-paper
-

Web Preservation Project : Final Report : a Report to the Library of Congress
Arms, William Y.
(Date Created: 03 Sep 2001)
(United States of America)
- The objective of the Web Preservation project was to initiate a program to collect and preserve open access materials from the World Wide Web on behalf of the Library of Congress. The report discusses partnerships with other organisations as well as discussing the process by which the Library of Congress can select, organise and preserve web resources. Issues raised as a result of the Minerva prototype are also canvassed, including copyright, access, selection and the difficulties in cataloguing and indexing web sites.
- http://web.archive.org/web/20041106155137/http://www.loc.gov/minerva/w
ebpresf.pdf
- Archived by the Internet Archive.
-

Web-Based Government Information : Evaluating Solutions for Capture, Curation and Preservation
Christenson, Heather; Colvin, Jennifer; Cruse, Patricia; Eckman, Charles; Greenstein, Daniel; Hutton, Cate; Kunze, John
(Date Created: Nov 2003)
(United States of America)
- Set within the context of the USA, this project report addresses the roles of libraries and other memory institutions in preserving web-based government information. The report outlines the methodology employed by the project team, including an extensive literature review and analysis, a demographic review of the dot-gov domain and surveys of existin
|