Staff Papers
A practical approach to ensuring the persistence of digital collections at the National Library of Australia
A paper by Tony Boston and Ninh Nguyen, National Library of Australia, Parkes Place, Canberra, ACT, 2600 {tboston, nnguyen}@nla.gov.au, May 2002.
Abstract. The National Library of Australia has embarked on a major digitisation program for parts of its collection to build an Internet-accessible digital library. The National Library also maintains a selective archive of Australian web sites for long-term preservation. The paper describes a system of persistent identification for digital collections which has been developed. The system includes naming schemes and techniques for resolution of resource locations. A rule-based persistent identifier resolver is described in detail. This allows citation of digital collections and is a core component of a developing digital services architecture. These issues are illustrated with particular reference to a web site delivering the papers of Sir Edmund Barton, Australia's first Prime Minister, and the PANDORA archive of Australian web sites.
Introduction
The National Library of Australia (NLA) has embarked on a major digitisation program for parts of its collection to build an Internet-accessible digital library. Material being digitised includes manuscripts, pictures, sheet music and maps. [1] The National Library also maintains a selective archive of Australian web sites for long-term preservation, the PANDORA archive. [2]
The world wide web has enabled a revolution in access to information resources and is opening the collections of libraries, archives, museums, and galleries to viewing online. One of the key requirements in making any digital collection accessible via the world wide web is the need to ensure persistence of names for and references to the digital objects that make up that collection. It has been argued that names, identifiers and technologies for their resolution are one of the basic building blocks for the digital library. [3]
The persistence problem
Institutions such as libraries, archives, museums and galleries make a commitment to the long term preservation and availability of their collection materials. The traditional library functions of collection, storage, management, discovery, access, delivery and long-term preservation are as equally applicable to digital collections as they are to physical format material. To support these functions, references or citations to digital collections in an open networked environment such as the web need to remain accessible for the long term.
As is well known, standard Uniform Resource Locations (URLs) on the web have a precarious existence and are subject to change. URLs are tied to particular filesystem locations on web-accessible Internet servers, or to particular applications that deliver content. They will resolve to an intended target resource as long as the filesystem or application continues to be available otherwise they resolve to a dead end error page.
There are two basic requirements for persistent citation of digital objects. The first is a logical and consistent scheme for globally unique naming of objects and the second is a method to resolve names to the current location of the resource. These issues were understood early in the development of the web and were outlined in the 1994 proposal for Uniform Resource Names. [4]
In 2001, the National Library of Australia undertook a study of persistent identification systems. [5] The study investigated the role of persistent identifiers in ensuring consistent availability and accessibility of resources, existing schemes for persistent identifiers, technologies for their resolution and recommended a course of action to implement a persistent identification scheme at the Library. Existing systems examined included PURLs (Persistent URLs), the Handle system, Digital Object Identifiers (DOIs) and the Archival Resource Key (ARK). All existing systems suffer from the lack of a globally available method of resolving resource names to resource locations. All exiting schemes rely on http and URLs for resolution.
The study recommended that in the absence of a globally supported persistent identification and resolution system, the Library develop its own naming scheme for digital collections and a URL-based resolver. This paper describes the naming scheme and resolver in more detail.
Naming scheme
The Library has adopted a naming scheme for each type of digital collection material, that is manuscripts, pictures, sheet music, maps, books, serials, oral history interviews, folklore recordings and the PANDORA archive. The identifiers applied to each type of material are, where possible, intelligent in that is they have in-built meaning, and are often based on an identifier used for the equivalent physical collection item. Identifiers are made globally unique by addition of a collectionId prefix that identifies the institution (NLA) and collection type.
Assigned persistent identifiers are made up of a number of attributes which are dash - separated, including a collectionId, a series of identifier attributes and a roleCode. The naming scheme for the Manuscript Collection is:
<collectionId>-<collectionNo>-<seriesNo>-<itemNo>-<sequenceNo>-<roleCode>
<collectionId>: Unique identifier for a collection, ie nla.ms for manuscript material
<collectionNo>: Unique collection or accession number, eg ms51 for the Papers of Sir Edmund Barton, Australias first Prime Minister
<seriesNo>: Collection subdivision or series number, eg 1,2,3
<itemNo>: Item number - up to 5 digits including optional letter, eg 1042a
<sequenceNo>: Image sequence number, eg s1, s2, s3
<roleCode>: A code drawn from a standard list indicating the role of the object, eg m = master image, t = thumbnail image, v = view copy
Example persistent identifier:
nla.ms-ms51-13-1296-s2-t for thumbnail image of the obverse of gold medallion presented to Edmund Barton by Pope Leo XIII in 1902.
The naming scheme for web sites within the PANDORA archive is:
<collectionId>-<titleNo>-<archiveDate>-<pubURI>
<collectionId>: Unique identifier for a collection, ie nla.arc for PANDORA material
<titleNo>: Unique 5 digit number for web site in PANDORA archive, eg 13467
<archiveDate>: The date the web site was archived in format YYYYMMDD
<pubURI>: The uniform resource identifier (URI) of the file on the publishers site
Example persistent identifier:
nla.arc-13467-20000911-http://www.acfonline.org.au/aboutacf/about.htm for a page about the Australian Conservation Foundation (ACF) from the 11 September 2000 harvest of the ACF web site in the PANDORA archive.
Other naming schemes have been developed for each type of material such as pictures, sheet music, maps, books, serials, oral history interviews and folklore recordings.
Resolver
A resolver service has been developed that redirects a request for a digital object to the current storage location of the object. The resolver is implemented as a Java servlet, within the Tomcat servlet container operating with an Apache web server. URLs passed to the resolver are of the form:
http://nla.gov.au/<pi>
where <pi> is the persistent identifier for the digital resource or object which conforms to the naming scheme described above.
Example persistent identifier URLs for the site which delivers the Barton papers, and the Australian Conservation Foundation web site within the PANDORA archive are:
http://nla.gov.au/nla.ms-ms51 Papers of Sir Edmund Barton
http://nla.gov.au/nla.arc-13467 Australian Conservation Foundation
The form of these URLs has been designed to be as brief as possible to allow them to be easily referenced. A virtual web server, nla.gov.au, has been set up separated from the Librarys main server at www.nla.gov.au. This allows the persistent identifier URLs to be kept as simple as possible and enables separate logging of accesses to the Librarys digital collections and PANDORA archive.
The resolver parses the persistent identifier and, based on a set of rules, determines a destination URL to redirect the user to. The destination URL may be a filesystem location or an address of an application that is delivering the digital object. These locations can and will change over time, but the persistent identifier URL will remain the same.
A rules-based approach to resolution is possible if parsing of a persistent identifier can be used predictably to determine the destination URL. This is possible within a managed environment in a single organisation such as a library. Taking a rules-based approach to resolution gives a number of advantages over a list-based approach which is used by systems such as PURL, DOI and Handles. At its most basic level, a list-based approach requires that each source and destination address is recorded in a database that is consulted by the resolver service. This list needs to be maintained over time, and may not be scalable as the number of resources referenced increases. Several million resources are accessible through the NLA resolver including the PANDORA archive. PANDORA is a selective archive of resources from the Australian Internet domain and at the time of writing contained over 2,500 titles and 5 million web pages. The rules-based approach is simpler and enables global changes to be made to resolution quickly and easily.
The mapping of persistent identifier URLs to destination URLs is defined in a configuration file written in XML format. An XML DTD for the configuration file is listed in the Appendix. The configuration file is monitored by the Java servlet that does the resolution. An administrator has access to the configuration file. Any changes to the file which, for example, may be needed if a resource is moved on the web site, are easily made and are immediately reflected in the resolver service. Resolution rules defined in the configuration file may vary for each type of collection material and different delivery systems may be used to provide access to different digital collections. Digital collections referenced by the resolver include the PANDORA archive, manuscripts, pictures, sheet music and maps. The configuration file consists of an identification section, and a mapping section. The identification section for the Manuscript Collection is shown below.
<identification>
<description>Manuscript Collection of the National Library of Australia</description>
<delimiter>-</delimiter>
<resolver>nla.gov.au</resolver>
<collectionId>nla.ms</collectionId>
<destination changeURL="yes">http://www.nla.gov.au/ms/mscoll.html</destination>
</identification>
The identification section defines the resolver server name, the collectionId for the Manuscript Collection at NLA, the delimiter used within the persistent identifiers and a default destination URL for resolution.
The next section of the file defines the rules used for mapping persistent identifiers, including rules for the format of identifiers. The resolver firstly makes sure that the persistent identifier conforms to the naming scheme for that collection material and then redirects the user to the destination URL. A set of attributes are defined in the mapping section of the configuration file which correspond to components of the persistent identifier. An example showing three attributes for the Manuscript Collection is:
<mapping>
<attributes>3</attributes>
<attribute name="collection" sequence="1" obligation="mandatory">
<label>Collection</label>
<description>Identifier for manuscript collection</description>
<contents minsize="3" maxsize="8" form="alphanumeric" format="ms\d{1,6}"/>
</attribute>
<attribute name="series" sequence="2" obligation="optional">
<label>Series</label>
<description>Number for manuscript series</description>
<contents minsize="1" maxsize="3" form="numeric" format="\d{1,3}"/>
</attribute>
<attribute name="item" sequence="3" obligation="optional">
<label>Item</label>
<description>Item or subseries number within manuscript collection</description>
<contents minsize="1" maxsize="6" form="alphanumeric" format="\d{1,5}[a-z]?"/>
</contents>
</attribute>
</mapping>
In this example, following the collectionId nla.ms, there are three attributes within the persistent identifier, namely the collection, and optional series and item numbers.
The final part of the configuration file simply maps each type of persistent identifier URL to a destination URL using attributes parsed from the persistent identifier, ie collection, series and item, substituted within the destination URL where required via the $$ references:
<mapping>
<match field="item">
<case value="null">
<description>Papers of Sir Edmund Barton Finding aid Maps to static web pages</description>
<destination changeURL="no">http://www.nla.gov.au/ms/findaids[/$$collection$$][/series-$$series$$.html]</destination>
</case>
<case value="*">
<description>Papers of Sir Edmund Barton Digital Items Maps to web delivery application</description>
<destination
changeURL="no">http://www.nla.gov.au/apps/msview?collection=[$$collection$$]
&series=[$$series$$]&subseries=[$$item$$]</destination>
</case>
</match>
</mapping>
<nomapping changeURL="no">http://www.nla.gov.au/nlaredirect/error.html</nomapping>
The <nomapping> element directs the user to an error page if the resolver was unable to parse or map the persistent identifier. Persistent identifier URLs that can be resolved via this part of the configuration file are:
http://nla.gov.au/nla.ms-ms51 Papers of Sir Edmund Barton Home page
http://nla.gov.au/nla.ms-ms51-1 Series 1 of Barton papers
http://nla.gov.au/nla.ms-ms51-1-2 Item 2 within series 1 of Barton papers
Persistent identifier URLs can resolve to a digital object itself, for example an image file of the obverse of a gold medallion, or a contextual display which provides access to the digital object along with metadata about the object and navigational devices such as finding aids, page turners, zooming/panning tools etc. For example, item 1296 from series 13 of the Barton papers can be referenced as:
http://nla.gov.au/nla.ms-ms51-13-1296-s2
Gold medallion in contextual display
http://nla.gov.au/nla.ms-ms51-13-1296-s2-v
Gold medallion view copy image
The NLA resolver was initially implemented in August 2001. Version 1 of the resolver implemented simple redirection of URLs. This involves a two step process in which the initial browser request is redirected to a new location using a HTTP 302 moved temporarily web server response as shown in figure 1.
Figure 1: Simple redirection in version 1 on the NLA resolver

Version 2 of the resolver was released in May 2002. In version 2, the ability to suppress redirection of the persistent identifier URL was added via support for the changeURL attribute to the <destination> and <nomapping> elements. This was implemented using a directive to the mod_rewrite URL rewriting engine within the apache web server. The impact of this change on users is that the persistent identifier URL displayed in the browser location window does not change during resolution when changeURL=no is set in the resolver configuration file. This enables the persistent identifier URL to be bookmarked or copied from the browser address field for citation purposes.
When redirection is suppressed, resolver performance is also improved as mapping to the destination URL is handled within the web server without any round trip back to the client browser as shown in figure 2. In this situation, if links from the destination URL to other pages are to work, a line has to be added within the <head> section of the destination document to indicate its base URL. For example, for the Barton papers home page: <base href="http://www.nla.gov.au/ms/findaids/ms51/index.html> is added after the <head> element in the HTML page at http:// www.nla.gov.au/ms/findaids/ms51/
Figure 2: Server based resolution implemented in version 2 of the NLA resolver.

The resolver in context
The NLA resolver is a core component of an evolving digital services architecture. This architecture provides support for the traditional library functions of collection, storage, management, discovery, access, delivery and long-term preservation in a digital context.
Since 1998, the National Library has been developing components of this architecture through its Digital Services Project. The Library has invested in infrastructure for capture and storage of digital collections, developed database applications for digital collections management and has begun to develop systems to provide online delivery and access.
The resolver provides persistent links between components of the digital library. For example links between resource discovery services like the Librarys Catalogue or collaborative services such as PictureAustralia at http://www.pictureaustralia.org, and web delivery systems for digital collections. The resolver allows simple and persistent citation of objects within digital collections which should continue to be able to be used as long as http and URLs are supported on the web.
Conclusion
The ability to provide persistent identification of objects within digital collections is essential for their effective management. The persistent identification system and the resolver service developed at the National Library of Australia underpins management of and access to its digital collections.
References
- National Library of Australia. Digitisation of traditional format library materials. http://www.nla.gov.au/digital/program.html
- National Library of Australia. PANDORA archive Preserving and Accessing Networked Documentary Resources of Australia. http://pandora.nla.gov.au/
- Arms, W.: Key Concepts in the Architecture of the Digital Library. In: D-Lib Magazine (July 1995). http://www.dlib.org/dlib/July95/07arms.html
- Sollins, K., Masinter, L. RFC 1737 Functional requirements for Uniform Resource Names. (1994). http://www.ietf.org/rfc/rfc1737.txt
- Dack, D. Persistent Identification Systems (May 2001). http://www.nla.gov.au/initiatives/persistence/PIcontents.html
Appendix: XML DTD for NLA resolver configuration file
<?xml version="1.0" encoding="UTF-8"?>
<!ELEMENT redirect
(record+, nomapping)>
<!ELEMENT record (identification, mapping+,
nomapping)>
<!ELEMENT identification (description, delimiter,
resolver, collectionId, destination)>
<!ELEMENT description
(#PCDATA)>
<!ELEMENT delimiter (#PCDATA)>
<!ELEMENT
resolver (#PCDATA)>
<!ELEMENT collectionId (#PCDATA)>
<!ELEMENT attributes (#PCDATA)>
<!ELEMENT attribute (label?,
description?, contents)>
<!ATTLIST attribute
name CDATA #REQUIRED
sequence CDATA #REQUIRED
obligation
(mandatory | optional | conditional) "optional" >
<!ELEMENT label (#PCDATA)>
<!ELEMENT contents
(value)*>
<!ATTLIST contents
minsize CDATA #IMPLIED
maxsize CDATA #IMPLIED
form (code |
alphanumeric | alphabetic | numeric | date) "alphanumeric"
format CDATA
#IMPLIED
padding (yes | no) "no"
default CDATA #IMPLIED >
<!ELEMENT value (#PCDATA)>
<!ELEMENT mapping (attributes,
attribute+, match+)>
<!ELEMENT match (case+)>
<!ATTLIST
match
field CDATA #REQUIRED >
<!ELEMENT case (description, (destination | match+))>
<!ATTLIST case
value CDATA #IMPLIED >
<!ELEMENT destination (#PCDATA)>
<!ATTLIST
destination
changeURL (yes | no) "yes" >
<!ELEMENT nomapping (#PCDATA)>
<!ATTLIST nomapping
changeURL (yes | no) "yes" >
