A persistent identifier is a name for a resource which will remain the same regardless of where the resource is located. Therefore links to the resource will continue to work even if it is moved. In 2001, after assessing the persistent identifier schemes currently in use or under development internationally, the National Library of Australia concluded that there was no scheme sufficiently mature to offer a suitable solution in the short-term for its digital collections. See the consultant's report prepared for the National Library for details of the assessment.
The National Library has implemented its own managed naming scheme for digital resources and coupled this with an internal resolver service that seamlessly directs a request for an item to the current storage location of that item. This scheme supports the digital surrogates that the Library produces from items in its collections, as well as the online publications archived in the PANDORA Archive.
Digital surrogates of collection items, web sites captured for long-term preservation in the PANDORA archive and audio material from the Oral History collection are stored in the National Library's Digital Object Storage System (DOSS). All digital objects are assigned a persistent identifier which is used to locate the object in the storage system, to track its process and preservation history and to associate descriptive, administrative, technical and structural metadata with the digital object.
A National Library of Australia persistent identifier consists of two or more dash ("-") separated fields of the form:
<CollectionId>-<Field1>-<Field2>-<Field3>-...
where <CollectionId> is:
| CollectionId | Collection |
|---|---|
| nla.arc | PANDORA archive |
| nla.aus | Australian books and serials |
| nla.con | Conservation |
| nla.gen | Overseas books and serials |
| nla.int | Internal |
| nla.map | Maps |
| nla.ms | Manuscripts |
| nla.mus | Music |
| nla.oh | Oral history |
| nla.pic | Pictures |
The scheme is summarised in the following table along with an example of a persistent identifier URI for each type of collection material:
| Collection | Scheme | Profile | Example |
|---|---|---|---|
| Internal (nla.int) | CollId-CollNo-UnitNo-SubUnitNo-SeqNo-RoleCode-DisplayCode-Version | nla.int-(nl|ls|ex)[a-z]{0,4}\d{1,6}[a-z]?-(\d{1,4})?-[a-z]{1,4}\d{0,4}-(s\d{1,3})?-(t|v|m|c|a|p|sd|gd\d{1,4}n?|ed\d{1,4}w?n?)-(do|cd)?-(v\d{1,2})? | http://nla.gov.au/nla.int-ls98039-cp-s8-v |
| Manuscripts (nla.ms) | CollId-CollNo-UnitNo-SubUnitNo-SeqNo-TileNo-RoleCode-DisplayCode-Version | nla.ms-ms\d{1,6}[a-z]?-(\d{1,3}[a-z]?(\.\d{1,3}[a-z]?){0,5})?-(\d{1,5}[a-z]?)?-(s\d{1,3}[a-z]?)?-([a-r][1-9])?-(t|v|e|m|c|a|s|st|ut|fa|sm|p|sd|gd\d{1,4}n?|ed\d{1,4}w?n?)-(do|cd)?-(v\d{1,2})? | http://nla.gov.au/nla.ms-ms51-6-1022-s1-e-cd |
| Maps (nla.map) | CollId-UnitNo-SubUnitNo-SeqNo-TileNo-SubTileNo-RoleCode-DisplayCode-Version | nla.map-[a-z]{1,4}\d{1,5}[a-z]?-(\d{1,3}[a-z]?)?-(s\d{1,3}[a-z]?)?-([a-r][1-9])?-([a-r][1-9])?-(t|v|e|m|c|a|p|sd|gd\d{1,4}n?|ed\d{1,4}w?n?)-(do|cd)?-(v\d{1,2})? | http://nla.gov.au/nla.map-rm2099-e-cd |
| Monographs (nla.aus & nla.gen) | CollId-UnitNo-SubUnit1-SubUnit2-SubUnit3-SeqNo-RoleCode-DisplayCode-Version | nla.(gen|aus)-(isbn\d{9}(\d{1}|x)[a-z]?|(an|vn)\d{1,12}[a-z]?|(f|nk)\d{1,5}[a-z]?)-{0,4}?-(s\d{1,3}[a-z]?)?-(t|v|e|m|c|a|s|st|ut|fa|sm|p|sd|gd\d{1,4}n?|ed\d{1,4}w?n?)-(do|cd)?-(v\d{1,2})? | http://nla.gov.au/nla.aus-an3281107-0-s3 |
| Music (nla.mus) | CollId-UnitNo-SeqNo-RoleCode-DisplayCode-Version | nla.mus-(an|vn)\d{1,12}[a-z]?-(s\d{1,3}[a-z]?)?-(t|v|e|m|c|a|s|st|ut|fa|sm|p|sd|gd\d{1,4}n?|ed\d{1,4}w?n?)-(do|cd)?-(v\d{1,2})? | http://nla.gov.au/nla.mus-an7579855-s1-e-cd |
| PANDORA (nla.arc) | CollId-TitlePI-ArchiveDate-PubURI | nla.arc-(([a-z]|[0-9])\d{4})-(\d{8})?-(http://)?... | http://nla.gov.au/nla.arc-13071-20000516-http://www.cinemedia.net/SFCV-RMIT-Annex/rnaughton/index.htm http://pandora.nla.gov.au/pan/41142/20040326/www.ablnat.com.au/index.html |
| Pictures (nla.pic) | CollId-UnitNo-SubUnitNo-SeqNo-RoleCode-DisplayCode-Version | nla.pic-(((an|vn)\d{1,12}[a-z]?)-(\d{1,6}[a-z]?)?-(s\d{1,3}[a-z]?)?|acc[-a-z0-9]+)-(t|v|m|c|a|p|sd|gd\d{1,4}n?|ed\d{1,4}w?n?)-(do|cd)?-(v\d{1,2})? | http://nla.gov.au/nla.pic-an7678346-1-v-cd |
| Oral History (nla.oh) | CollId-TRCCollNo-TRCSeriesNo-TRCItemNo-SeqNo/SubUnitPartNo-PieceNo-FragNo-RoleCode-DisplayCode-Version | nla.oh-\d{4}-[a-z0-9]{4}-[a-z0-9]{4}f?-(\d{1,5})?-(s?\d{1,3}[a-z]?)?-(pe\d{1,3}[a-z]?)?-(fr\d{1,3}[a-z]?)?-(o|m|c|dm|w|a|d|ad|sa\d|ra\d?|sb\d|rb\d?|s|tr|fa|rm)-(do|cd)?-(v\d{1,2})? | http://nla.gov.au/nla.oh-4841-0000-0001-s1-d |
| Serials (nla.aus & nla.gen) | CollId-SerialId-IssueVol-IssueNo-IssuePart-IssueDate-SubUnitNo-SeqNo-RoleCode-DisplayCode-Version | nla.(gen|aus)-(issn\d{8}[a-z]?|(an|vn)\d{1,12}[a-z]?|(f|nk)\d{1,5}[a-z]?)-(v[a-z0-9]{1,6}|t\d{1,6})?-(n[a-z0-9]{1,6})?-(p[a-z0-9]{1,6})?-((c|i|m|s)\d{4,8}[a-z]?)?-(\d{1,5}[a-z]?)?-(s\d{1,3}[a-z]?)?-(t|v|e|m|c|a|s|st|ut|fa|sm|p|sd|gd\d{1,4}n?|ed\d{1,4}w?n?)-(do|cd)?-(v\d{1,2})? | nla.aus-issn00279633-v207-n5003-pa-m19880701-s1-t |
| Conservation (nla.con) | CollId-SubUnitNo-SeqNo-RoleCode-DisplayCode-Version | nla.con-d{1,5}[a-z]?-(d{1,5}[a-z]){0-3}-(s\d{1,3}[a-z]{0,2})? | nla.con-694 |
For each collection, rules have been implemented for the construction of persistent indentifiers as documented in the above scheme. An example persistent identifier from the manuscripts collection might be for the view copy of the first page of item 1 from series 1 of the collection holding the Papers of Sir Edmund Barton. This digital object has a persistent identifier of nla.ms-ms51-1-1-s1-v. The elements of this PI are as follows:
| nla.ms | PI prefix for manuscripts |
|---|---|
| ms51 | Unique number for the manuscript collection 51, the Papers of Sir Edmund Barton |
| 1 | First level of the hierarchy within the collection - for manuscripts this is generally the series within the collection |
| 1 | Second level of the hierarchy within the collection - for manuscripts generally the item or folder within the series |
| s1 | The first part of the item or folder |
| v | view copy of the part |
The Library's PI scheme is stable for the PANDORA archive, and for digitised pictorial, map, manuscript and music material but may be subject to further refinement for other material as experience is gained with its digitisation and management.
Profiles are expressed with the following definitions:
Valid characters for fields within persistent identifier are lowercase a-z and 0-9 only.
The role code is used to define the purpose of a particular digital object, for example the object may be a digital master or preservation copy (-m role) in 300 ppi TIFF version 6 format, or a thumbnail (-t role), view (-v role) or examination (-e role) copy in 72 ppi JPEG format for viewing over the web. Role codes allow a separation of the purpose of the digital file from its specific format, and allow the possibility that identifiers can be preserved over the long-term as digital file formats change. For example, the National Library's thumbnails are currently in 72 ppi JPEG format, in future they may be migrated to PNG or JPEG2000 format. If this occurs, existing persistent identifiers can continue to be used with no disruption to applications delivering digital content.
Filenames for digital objects are based on the persistent identifier and always include the role code to indicate the purpose of the file:
| Code | Description |
|---|---|
| o | original |
| m | master or preservation copy |
| c | co-master. This version may be cropped, enhanced for web legibility, positive where master is negative, or stitched where masters are tiled. |
| dm | derivative master. This version is used for creation of dynamically generated derivatives. |
| t | thumbnail - web derivative |
| v | view - web derivative |
| e | examination - web derivative |
| a | archive (image files) or audition (audio files) |
| s | summary |
| tr | transcript |
| st | structured transcript |
| ut | unstructured transcript |
| fa | finding aid |
| rm | related metadata |
| sm | structural map |
| p | |
| f | filtered copy |
| d | distribution copy |
| ad | analogue distribution copy (audio files) |
| pm | production master |
| sd | special delivery |
| sa | streaming delivery format 'a' (audio files) |
| ra | reference file for streaming delivery format 'a' (audio files) |
| sb | streaming delivery format 'b' (audio files) |
| rb | reference file for streaming delivery format 'b' (audio files) |
There are two additional role codes, gd and ed which are used for dynamically generated image derivatives. Dynamic derivatives are created from a 72 dpi 1600 pixel "derivative master" TIFF image (role code -dm). The derivative master image is created automatically when a file is uploaded to the storage system and is stored in the derivatives file system. Dynamic derivatives are created using the Image Alchemy software (http://www.handmadesw.com/) and are cached so that after initial creation they are fast to load.
Persistent identifiers for dynamic derivatives conform to the Library's PI scheme with additional information about the dimensions and properties of the image encoded using the role code. The following table shows the role codes used for dynamically generated derivatives.
| Role code and image size | Switch (optional) | Example PI | ||
|---|---|---|---|---|
| Code | Description | Code | Description | |
| gd500 | image with greatest dimension (gd) of 500 pixels, either landscape or portrait | - | - | nla.gen-an6520463-1-1-1-s324a-gd500 |
| gd500 | as above | n | image will have no footer | nla.gen-an6520463-1-1-1-s324a-gd500n |
| ed500 | image with equal dimensions (ed) of 500 pixels, "square" image | - | Where original image is rectangular, background fill colour is black | nla.gen-an6520463-1-1-1-s324a-ed500 |
| ed500 | as above | w | Where original image is rectangular, background fill colour is white | nla.gen-an6520463-1-1-1-s324a-ed500w |
| ed500 | as above | wn | image will have no footer | nla.gen-an6520463-1-1-1-s324a-ed500wn |
The display code indicates if the digital object itself is to be displayed (do) or the digital object within a contextual display (cd). This is an optional field.
| Code | Description |
|---|---|
| do | digital object (digital object itself is returned to the user) |
| cd | contextual display (digital object is returned to user within a page including metadata and other contextual information) |
When a role code is specified and no display code is given, the default behaviour of the resolver is to return the digital object itself. If no role code is supplied, then the default behaviour of the resolver is to return the digital object in its contextual display.
| Example | Description |
|---|---|
| http://nla.gov.au/nla.mus-an7579855-s1-v | digital object returned |
| http://nla.gov.au/nla.mus-an7579855 | digital object returned in contextual display |
The version field is made up of a "v" for version followed by a sequential number from 01 to 99 to indicate different versions or generations of a digital object. Versions are only applied to master copies of images. Versions of derivatives are not retained by the Library. The current version of any object does not have a version number, eg:
| PI | Comments |
|---|---|
| nla.pic-an2678983-m-v1 | Version 1 |
| nla.pic-an2678983-m-v2 | Version 2 |
| nla.pic-an2678983-m-v3 | Version 3 |
| nla.pic-an2678983-m | Current version (implied version 4) |
Map collections are represented by the map unit number which is made up of an alphabetic code up to four characters for the map collection followed by a sequential number. The code set for map collections is:
| Code | Description |
|---|---|
| db | Daisy Bates collection |
| f | Ferguson collection |
| lfsp | Loose Ferguson Sale Plans |
| nk | Nan Kivell collection |
| t | Tooley collection |
| ra | Rare Atlas |
| rm | Rare Maps collection |
This code set will be extended as new map collections are digitised. The code is followed by a number (up to 5 digits) and optional letter (a-z).
For large format items, for example maps, it is not currently possible to capture the entire image using a digital camera at the required resolution. In these cases the map is "tiled" with separate images captured in a grid. These tile images are stored as separate digital masters and a stitched co-master is created to be used as the source for creation of derivatives. The numbering scheme for image tiles is represented in the two character TileNo field for maps as follows:
Tile number positions representing a four image tile grid show below:
| a1, eg nla.map-nk2413-a1-v |
a2, eg nla.map-nk2413-a2-v |
| b1, eg nla.map-nk2413-b1-v |
b2, eg nla.map-nk2413-b2-v |
This example shows the persistent identifiers to be applied to the four view copy tile images making up map nk2413 from the Nan Kivell collection.
This scheme for tile images can also be applied to manuscript material and, for example, is used for describing individual TAPA cloths collected from South Sea Islands by Captain Cook which are mounted in a grid layout on bound pages of a book.
A resolver service has been developed that redirects a request for a digital object to the current storage location of the object. The resolver is implemented as a Java servlet, within the Tomcat servlet container operating with an Apache web server and is described in more detail in this staff paper.
The resolver service is designed to interpret web addresses of the form:
http://nla.gov.au/<pi>
where <pi> is a persistent identifier for material from the Library's digital collections.
The PIs assigned by the Library to its digital collections provide a reliable and stable way of naming items that can be used with confidence for citation purposes. The PI for citation of a particular item is shown on a web page in the form "To cite this page use: http://nla.gov.au/<PI>"
The resolver parses the persistent identifier and, based on a set of rules held in an XML configuration file, determines a destination URI to redirect the user to. The destination URI may be a filesystem location or an address of an appropriate application based on the type of object being delivered, e.g. if the item is a manuscript, the request is directed to the Library's generic delivery system for digital collections, if the object is an electronic publication in the PANDORA archive it is directed to the delivery system for this archive. These locations can and will change over time, but the persistent identifier URI will remain the same. The resolver does this using the PI collection identifier, e.g. nla.ms, to determine the type of material being described and the appropriate application to deliver it.
This system has several advantages:
Three separate areas have been defined for storage of digital collections on the DOSS. These areas are used for storage of archival masters of digital objects (/master), derivatives for access by Library users (/derivative) and a working area where files are stored and checked for short periods before being submitted to the archive (/working). Digital master files are stored on tape while web derivatives are stored on disk.
Under each of these areas, directories exist for each collection reflecting the logical division of material held by the Library such as pictures, maps, manuscripts, and archived web sites. These top level directories are defined below:
| Digital store | Collection identifier |
|---|---|
| /master /derivative /working |
/nla.arc /nla.aus /nla.con /nla.gen /nla.int /nla.map /nla.ms /nla.mus /nla.oh /nla.pic |
Directories below the second level for the /master and /derivative filesystems vary for each type of material and are based on the (often hierarchical) persistent identifier scheme for that type of material. Filenames of digital objects correspond to the persistent identifier for that object along with the three letter code representing the format of the digital file, ie <pi>.<format> (eg nla.ms-ms000051-001-00002-s001-t.jpg). Filenames always include a role code to ensure that there is no potential for surrogates created for different purposes having the same filename. Numeric fields within the persistent identifier portion of the filename are left padded with zeros to enable file sorting. Extra directory levels are created to reduce the number of files per directory to a manageable level. Filesystem performance is maximised if the number of files per directory is less than 2000.
As a general rule it is preferable for the directory structure to reflect the PI scheme as this will allow applications that deliver digital objects to predict the filesystem location of a digital object based on the PI. The approach taken with the Library's digital collections is to break up fields of the PI into groups of three characters and use these as directory names. This provides for a maximum of 1,000 objects per directory and a predictable filesystem location for digital objects.
The persistent identifier scheme and resolver service developed by the National Library is a basic building block of its digital services architecture and it provides a foundation the long-term preservation and management of the Library's digital collections. For more information about the persistent identifier scheme or resolver service, please send us an email.
| About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us |