Last updated 30 September 2002
This document provides detailed specifications for a Digital Collections Manager which meets the Librarys needs for recording and processing management and technical information about image digital surrogates. The Library has embarked on a major digitisation program to provide improved access to and long-term preservation of collection materials. The initial focus of this program is capture of material such as pictures, maps, manuscripts, rare books, 19th century serials and sheet music to produce high resolution digital masters for preservation and low resolution display copies for access via the Internet. The Digital Collections Manager will assist the Librarys collection managers to manage the digitisation process, as well as ongoing format migration and preservation processes through recording of appropriate metadata. It will replace a number of existing in-house systems and provide for long-term management of digital material by the Library.
In the future it is planned to enhance the Digital Collections Manager to support:
The estimated disk storage requirements for the digital materials to be managed by the Digital Collections Manager are 2 terabytes in 2001 rising to 20 terabytes in 2006. This corresponds to approximately 200,000 digital assets in 2001 rising to over 2,000,000 by 2006. The Digital Collections Manager must be capable of managing this volume of data and its associated metadata with no effect on functionality or performance.
The Library has purchased a four processor Sun 450 system with 2 TB of RAID disk and a tape library for backup and archiving of data. The Digital Collections Manager will be developed using Oracle and the Librarys Application Development Framework.
The Digital Collections Manager must be able to meet the following performance requirements:
Response times in this section apply to the speed of response from the server whose performance is being measured to client software in the same room or on the same network segment. The response will be in the form of a web page viewed using Microsoft Explorer Version 5 and including no markup beyond that required to delimit the database fields displayed. The response time will be measured from the moment the user submits the transaction until the first screen of the response is completely displayed. The hardware platform for measurement of performance is that specified in section 1.2. Performance will be measured using the data requiring to be loaded on day 1 and a PC based within the Library, connected to the DCM via the Local Area Network. Response time will be measured from the moment the user submits the transaction until the first screen of the response is completely displayed.
As part of a separate project the Library is building a Generic Delivery System for Digital Objects. The Generic Delivery System will use metadata stored in the Digital Collections Manager to construct the user interface and navigation workflows required to access each object stored in the Digital Object Storage System. The performance requirements specified above should not be impacted by the activity of the Generic Delivery System. This may require the data to be duplicated in a second instance that is targeted by the Generic Delivery System.
The user interface must be web browser based, so that special software will not need to be installed and upgraded. The Library uses Microsoft Explorer Version 6 as its default browser with all staff able to access web based services via a screen size of 800 x 600 pixels. The user interface must be optimised for use with this browser and screen size. However, only features supported by the browser that are aligned with W3C standards should be used for core functionality. In addition to the W3C markup and style sheet standards, the Library requires support at least to level 1 of the W3C Content Accessibility Guidelines for all of its web browser based user interfaces.
The Digital Collections Manager will deliver a mission-critical workflow system to the Library. The guiding principles for development of the user interface must be simplicity, speed and ease of use. The user interface must allow the user to do as much as possible with as few keystrokes/ mouse clicks as possible. Standard alternatives to mouse clicks must be provided for occupational health and safety reasons. The user interface must present the most likely action as the default and do as much as possible to prevent common mistakes and maintain the integrity of the Digital Collections Manager. It must remember the context in which the user is working and provide appropriate templates and default values.
In the Specifications the term data is used to refer to objects in digital form, eg master copies of pictures in TIFF format or objects, while the term metadata is used for structured information that describes the data (data about data). Data may also be used generically, to describe both data and metadata, for example, when specifying data migration requirements. Metadata may also be used for structured information that describes a work that has not yet been digitised.
A work is anything that is collected by the Library. It may be a physical object, book, manuscript, map, picture, sheet music, serial or sound recording. The term collection is used to describe all works collected by the Library and also to refer to a specific sub-collection, eg the Pictorial Collection, that is managed by a separate group of collection managers.
The Library may choose to manage a work as a unit or a subunit. A unit may be an item which consists of just one primary object, eg a book or painting; or a set, which consists of multiple subunits (eg the papers of a famous person or a photograph album or a group of glass negatives with shared provenance). A subunit may itself be an item or a set; or it may be a part. A part is a component of a complex item such as a book or three-dimensional object, eg a cover, page or view. Using this terminology, a manuscript collection such as the Barton Papers is a set-level unit. The diary series is a set-level subunit. A diary in this series is an item-level subunit. A page in this diary is a part. A component is a part that is also treated as an item in its own right, for example, a plate in a book.
The table below maps units and subunits against ten likely work hierarchies. This model will support a six level hierarchy, eg a detail (level 6) on a page (level 5) in a piece of manuscript music (Level 4) forming a component part of a bound volume (level 3) in a series (level 2) of a manuscript collection (level 1). Ideally, however, there should be no limit to the number of levels supported. Decisions to minimise the number of levels should be business, not system-driven.
| Likely Work Hierarchies | ||||||||||||||
| item | item | item | item | item | set | set | set | set | set | set | set | set | ||
| unit | 1 | part | part | comp | comp | item | item | item | item | set | set | set | set | |
| subunit | 2 | part | part | part | part | part | comp | comp | item | item | set | set | ||
| subunit | 3 | part | part | part | part | part | part | comp | comp | |||||
| subunit | 4 | part | part | part | part | |||||||||
| subunit | 5 | part | ||||||||||||
| subunit | 6 | |||||||||||||
A copy is a manifestation or instantiation of a work. A work can have multiple copies. The Library may hold one or more copies of a work, eg the original paper version of the page described above, a number of microform replicas and a set of digital replicas made from either the original or from the microform copy. In this specification a copy may also be an access aid such as a transcript or finding aid. The Library may also hold one or more original copies of a work, each of which may be a target for digitisation for access or preservation reasons.
When a work is selected for acquisition by the Library, a catalogue entry is raised for the unit and copy-specific details are recorded. If the work is a set, subunits are listed and electronic list(s) or finding aid(s) generated that are linked to the catalogue entry through a URL. These processes may occur as part of raising the catalogue entry or at a different time, for example, when physical copies are being batched for digitisation. Subunit items and components may be given their own catalogue entry if the work is considered sufficiently significant to be listed in catalogue result sets.
For units and subunits with catalogue entries, the resource description in the NLA Catalogue is the master copy. For subunits without their own catalogue entry but listed in finding aids, the resource description in the finding aid is the master copy. For all other subunits, the resource description in the Digital Collections Manager is the master copy.
A job is an event that causes a process or set of processes to be performed on a copy or group of copies. A process is an event that uses a physical or digital copy as input and may result in transformation of the existing copy (eg, through repair, restoration or media transfer) or the creation of a new one (eg through digitisation, microfilming, photocopying or format migration). A project is an event that provides the rationale for a job. A project may involve multiple jobs and processes over time.
The Librarys in-house digitisation teams will generally digitise copies on a one-by-one basis. The Digital Collections Manager will first know about the digital copies when a job is initiated to upload them. In this case the digitisation workflow will be tightly coupled with the loading workflow. A group of copies may also be batched and submitted for digitisation as a single job. The batch will be accompanied by a job submission report itemising the input copies. The digitiser may upload the copies via an online process or return the files together with a process report in the standard import format defined below. The latter workflow will generally be used by contractors. In all cases, procedures will be supported to enable various degrees of checking, with appropriate correction action taken if quality control standards are not met.
Metadata will be imported to and exported from the Digital Collections Manager via an import file and export file that conform respectively to a standard import format and standard export format. This format should be defined in the form of an XML DTD or schema.
The Library will assign a persistent identifier to each work and copy managed by the Digital Collections Manager. The Library will maintain a separate Persistent Identifier resolver service that will algorithmically determine the appropriate means of web access to the referenced work or copy.
The persistent identifier for a work will be composed of the following elements:
collectionId-workId
- where:
The work persistent identifier will correspond to the string included after the http://nla.gov.au/ part of the URL in the catalogue entry and will also be used as the work ID in the DIgital Collections Manager. Records will not be able to be added to the Digital Collections Manager from the NLA Catalogue or a finding aid unless the work ID can be derived from the catalogue entry or the entry in the finding aid.
Generally the persistent identifier for a child work will be based on the persistent identifier for the parent work with an added sequence number: e.g.:
Exceptions to this rule include members of sets that have been catalogued separately or that have been physically numbered in a flatter sequence that the Library prefers to reflect in the persistent identifier: e.g.:
Policies for assigning persistent identifiers to a work are outlined in Persistent Identifier Scheme Adopted by the National Library of Australia.
The persistent identifier for a copy will be composed of the following elements:
collectionId - copyId - role - version
where:
The copyId component of the persistent identifier will generally be the same as the workId, e.g.:
Where there are multiple original copies of a work, each copy will have its own unique persistent identifier. In this case, the persistent identifier of the "best copy" will usually be the same as the persistent identifier of the work.
Digital copies to be managed by the Digital Collections Manager will be created using a range of technologies including flatbed scanner, digital camera, roll film scanner (for microfilm scanning) and overhead scanner. The digital master images created will be stored in 24 bit RGB TIFF version 6 format. Completed master images will be moved from a temporary work area to a network accessible master file system. This file system will be an Digital Collections Manager hot folder and images stored in the file system will be automatically imported into the Digital Collections Manager.
1. How files are named or stored must be transparent to the Digital Collections Manager.
2. The Digital Collections Manager must be able to automatically create derivatives of imported master files.
3. It must be possible to vary the number of derivatives and their maximum dimension by Collection; that is, to maintain separate derivative profiles for each Collection.
The following draft standards and standards development activities have been instrumental in the development of these specifications:
Reference Model for an Open Archival Information System (OAIS).
Making of America II Document Type Definition (MOA II).
Metadata Encoding and Transmission Standard (METS).
Technical Metadata for Digital Still Images.
OCLC/ RLG Preservation Metadata Working Group.
National Library of Australia. Preservation Metadata for Digital Collections.
The OAIS provides a reference framework for an Open Information Archival System. This specification does not use the OAIS terminology because it is a functional requirements specification based on use case methodology, not a system design document. A deliberate effort has been made to use terms that are familiar to collection managers in both the use cases and the user interface templates. However, as part of the system design process, the system should be audited against the OAIS framework as a quality assurance method.
The Library has already developed a Delivery System using metadata based on the MOAII DTD for generating contextual displays (see http://www.nla.gov.au/ms/findaids/ms51/. However, a decision has now been taken for performance and scalability reasons to derive the information needed to support delivery of the Librarys digital objects dynamically from the Digital Collections Manager. However, the data model has been aligned with standards such as METS so that data may be output in METS or an allied format if required at a later stage, for example, to participate in a federated delivery system.
Most preservation metadata initiatives are based on the development of data dictionaries or languages to support the recording of preservation metadata. In the logical data model in this specification, the metadata is described as properties of a range of different objects that need to be brought together for the purposes of import, display or export. In addition, certain preservation metadata elements required by the above draft standards are not defined in the data model, as they can be generated externally. For instance, because the Library manipulates scanned images before uploading them to the Digital Collections Manager, only the display orientation is stored. A default orientation of normal can be set if an XML schema or DTD requires this element.
In April 2002, the Library implemented v1.0 of the Digital Collections Manager (DCM). DCM v1.0 supports the following functions:
DCM V1.0 was designed as an early deliverable to support the upload process. This enabled the spreadsheets that were being used to store technical metadata to be thrown away. However, V1.0 did not include a means of keeping work details updated in line with the Catalogue or of generating updates to the Pictures Catalogue. A second early deliverable, DCM V1.1 was implemented at the end of June to deliver the required catalogue interfaces. This enabled the Pictorial Manager system previously used to manage the digitisation of pictures to be deprecated. A small group of fixes to DCM V1.1 was implemented soon afterward as V1.1.1.
Work has now begun on DCM V1.2, which has as its aim to complete the implementation of all functions required for the management of still images.
This section provides information on related systems and the sources of metadata to be loaded into the Digital Collections Manager on implementation and managed by the Digital Collections Manager in accordance with the Specifications. Interfaces between the Digital Collections Manager and existing systems used by the Library are also described and requirements identified for sub-tasks to be included in the project plan to cover all system interface and data migration requirements. Future releases of the Digital Collections Manager will have additional data migration requirements.
PANDAS is the system used to collect and manage web sites selected for long term archiving in the PANDORA archive. No interfaces will be required between PANDAS and the Digital Collections Manager and there is no future requirement for the Digital Collections Manager to replace PANDAS. The workflows to collect and manage web sites are sufficiently different from the workflows to collect and manage digital surrogates for two separate applications to be needed. However, there is a requirement to ensure that the data models for the two systems are aligned. This will enable future enhancements, such as the enhancement to support the recording of intellectual property information and the enhancement to support management of the preservation of the Librarys digital collections, to operate across both data collections.
Current status: The task to align the data models of the two services has not yet been scheduled.
The Library already has an Integrated Library Management System (ILMS), which is used for management of the Librarys collections. This system does not meet the Librarys needs for digital object management outlined in Section 1.1. In particular, it does not support the digitisation process. The new ILMS to be implemented during 2002/2003 will also not support the digitisation process. The Digital Collections Manager is being developed to provide this functionality.
A critical success factor for the Digital Collections Manager will be the extent to which collection management workflows currently supported by the ILMS and the workflows needing to be supported to manage the digitisation process are integrated from the point of view of the collection manager.
To achieve this, the ILMS will be the master source for catalogue entries. To keep the two databases aligned, a utility will be developed that exports new and updated catalogue entries from the ILMS for automatic loading to the Digital Collections Manager. The Digital Collections Manager must check for and load a new import file at short time intervals or, ideally, synchronously through definition of the file as a named pipe.
Currently the Library uses the national bibliographic utility, Kinetica as the master source for its catalogue records and downloads the records to the ILMS. The same process will be used for extracting in-scope records for the Digital Collections Manager. As part of implementing the new ILMS a task has been identified to review the ILMS-Digital Collections Manager interface requirements.
A single standard record identifier (the Immutable Number) will be used by the Digital Collections Manager for matching updated and deleted entries. Because it will not be possible to alert the Digital Collections Manager of deleted entries in the ILMS through this process, the Digital Collections Manager will support a use case for deleting entries. A small number of records in the ILMS do not have an Immutable Number. These will be matched through a separately defined standard record identifier.
From time to time, copies will be identified for digitisation from one of the Librarys other collections, eg a map, manuscript, out-of-print book or a 19th century periodical. The Importing a Catalogue Entry use case must enable the ad hoc importing of catalogue entries in this scenario. This will work by enabling the user to initiate an immutable number search of the ILMS using Z39.50 protocols . It will be necessary to develop and maintain a utility that conducts this search and imports the record to the Digital Collections Manager.
As part of the implementation process, it will be necessary to identify and download the in-scope ILMS metadata. The following table identifies the likely number of records that will be involved and the estimated growth rate per year for the sheet music and pictures collections.
| Release 1 | Subsequent releases | Annual growth rate | |
| music | 76,786 | - | 3,500 |
| picture | 32,812 | - | 30,000 |
| ad hoc | - | 500 | |
| TOTAL | 109,599 | 34,000 |
This means that, after implementation, new and updated records from the ILMS will be added to the Digital Collections Manager at a rate of approximately 150 records per working day depending on digitisation priorities.
Current status: The ILMS-Digital Collections Manager interface is fully implemented. The task to review the ILMS-Digital Collections Manager interface requirements as part of implementation of the new ILMS has been scheduled.
Pictorial manager is the system currently used by the Library to manage the digitisation of its Picture Collection. The Digital Collections Manager will entirely replace the Pictorial Manager System.
Metadata to be migrated from Pictorial Manager to the Digital Collections Manager are as follows:
It will be necessary to provide this metadata in the standard import format for loading to the Digital Collections Manager during implementation.
The master image files for the bulk of items from the Pictorial Collection are held in Kodak PhotoCD (.pcd) format on CD-ROM. All items digitised since July 2001 will have TIFF masters. The Library plans to gradually migrate the PhotoCD masters to TIFF format after implementation of the Digital Collections Manager. Derivatives (thumbnails and medium resolution) for material with PhotoCD masters will need to be migrated to the Digital Collections Manager. It is assumed that derivatives for material with TIFF masters will be regenerated from the master as part of the loading process. A process will need to be defined to ensure a concordance between current file naming conventions for the PhotoCD masters and their derivatives and the new naming conventions based on the persistent identifier.
Current status: The Pictorial Manager has now been deprecated. The task to migrate the PhotoCD masters to TIFF has not yet been scheduled.
Pictorial lists contain item level metadata such as picture captions. Until recently, the Library has used a range of separate tools (eg, Microsoft Word, Access and Excel) for creating pictorial lists. In March this year, a decision was taken to raise a catalogue entry in the ILMS for all pictorial items in a set, using MARC tags to establish the parent-child relationship. This policy has now been implemented for all new material. Catalogue entries for items in existing pictorial lists will be migrated to the ILMS via Kinetica. There is thus no requirement for the Digital Collections Manager to support the creation of subunit records for pictorial material or the loading of pictorial lists; and there is no requirement to migrate any existing pictorial lists to the Digital Collections Manager.
Current status: The new policy is currently under review because of its impact on cataloguing workloads. There may be a requirement to maintain pictorial lists for some pictorial collections.
Manuscript finding aids contain detailed descriptions of the content of significant manuscript collections. The description will generally not go down to item level; but will give a broad idea of what is contained in specific series or folders. Some specific items may be listed. Finding aids are currently produced in Word and then marked up in HTML for publication on the Librarys web site. A URL linking to the finding aid is inserted in the catalogue entry.
The Library plans to migrate the existing HTML manuscript finding aids to the Encoded Archival Description (EAD) format and to develop a simple web-based form to enable manuscript collection managers to create and maintain finding aids directly in EAD, bypassing Word. A tool for doing this has not yet been identified, and it is possible that the Digital Collections Manager could provide this capability. The Digital Collections Manager will in any case need to support the creation of subunit entries for objects that are digitised on demand and some way of updating the finding aid to make the digital copy accessible through the Librarys resource discovery services. In addition, the Digital Collections Manager will need to support a process to generate subunits from an existing finding aid, for example, when a collection is to be digitised. To do this it will be necessary to develop conversion utilities from EAD to the Digital Collections Manager and vice versa.
Because a manuscript set can be very large, digitisation of a set will generally be treated as a project and resources provided to prepare the required metadata. This may be done through four quite separate processes:
The lister may enhance the finding aid outside the Digital Collections Manager using a separate tool.
The lister may import the finding aid to the Digital Collections Manager and create additional subunit entries there.
The lister may use the Digital Collections Manager to create subunit entries for a Collection that does not yet have a finding aid.
The digitiser may generate additional descriptive metadata as part of the digitisation process. At the very least, a minimal level record will be created for instances of each subunit and copy object in the hierarchy, using algorithms to predict the persistent identifiers. These records will be loaded to the Digital Collections Manager as a by-product of processing the process report.
· The lister may add additional descriptive details at a later stage to improve the usability of the finding aid and associated structural maps for subunits, either through the application, or by enhancing the finding aid and re-importing it.
The Digital Collections Manager must be flexible enough to support all of these workflows.
Current status: A separate system has now been implemented for creating and maintaining manuscript finding aids outside of the Digital Collections Manager and it is planned to extend this system to manage finding aids for other collection materials. The requirement to support all of the above workflows is still relevant, but the Finding aid will now be the master source for the subunits it describes. There is thus no requirement to export finding aids from the Digital Collections Manager.
The Library has two digitisation teams equipped to produce TIFF images using a range of devices and manipulation software. The Library may also, from time to time, submit batches of material to a contractor for digitisation. The Digital Collections Manager will not interface directly with the devices and software used by the digitisers. Technical metadata generated by the digitisation process will be extracted directly from TIFF masters when they are uploaded to the Digital Collections Manager. However, metadata not derivable from the file will need to be recorded by the digitiser. Use cases have been defined that will enable a range of different digitisation workflows to be supported. These include support for uploading of images and recording of metadata as they are scanned through to a full batch submission and processing capability.
Pending implementation of the Digital Collections Manager, metadata is being created and stored in the form of spreadsheets. The following metadata will need to be migrated to the Digital Collections Manager as part of the development process:
Quantities will be as follows:
| Project |
Sets |
Items |
Parts | ||
| Units | Subunits | Units | Subunits | ||
| Barton | 1 | 13 | 1843 | 4,000 | |
| Wills diary | - | - | 1 | - | 120 |
| Sheet music | TBA | ||||
| Rare maps | - | - | 25 | - | 25 |
| Pictures | TBA | ||||
| Maps | TBA | ||||
The files produced by these projects will already be on the System in the form of TIFF masters. It is assumed that the thumbnail and medium resolution versions will be generated by the Digital Collections Manager according to the derivatives profile for the collection.
Large maps are dependent on a delivery system that uses files in Mr Sid format. This category of material has additional derivative-generating and storage requirements that will need to be taken into account when the data and metadata is migrated.
Current status: All of this metadata has now been migrated and the spreadsheets deprecated.
This is a collection of 70,000 Tiff master copies, 7,000 PDF files and 1,,000 OCR text files created to support an online delivery system for 70 ninetenth century Australian periodicals. It is not intended to migrate these copies to the Digital Collections Manager as part of the first release, but There will be a need to migrate them once the system is in production. The delivery system is based on multi-page PDF files. It is recommended that this technical solution be reviewed before the files are migrated. A solution based on structural maps and JPEG derivatives would have been adopted if we had this project to do again now.
Current status: The task to bring these files under DOSS control has not yet been scheduled.
The NLA Catalogue is a public web browser based service that interfaces with the ILMS to provide the Librarys end users with access to catalogue entries. If the catalogue entry is for an item-level unit or subunit that has been digitised, the end user can exploit a persistent identifier for the digital copy to view files and associated metadata through different workflows depending on the record type
If the catalogue entry is for a set, the end user can browse a linked list or finding aid. From the finding aid, the end user can exploit persistent identifiers for digital copies of subunits to view files and associated metadata through different workflows depending on the record type.
No direct interface is required between the NLA Catalogue and the Digital Collections Manager. A Generic Delivery System for Digital Objects is being developed through a separate project that will use metadata stored in the Digital Collections Manager to support this process.
Numerous other resource discovery services access the Librarys catalogue entries through Z39.50 search and retrieval protocols or the sharing of metadata. These services will exploit the same Generic Delivery System as the NLA Catalogue and there is no requirement for them to interface with the Digital Collections Manager.
Current status: The Generic Delivery System has been implemented for sheet music and maps. Tasks to develop the Generic Delivery System to support pictures and manuscripts are also scheduled. The relationship between the NLA Catalogue, the Digital Collections Manager and the Generic Delivery System has been documented further in the paper Architectures for Web Collection Delivery.
The NLA Catalogue provides a single point of entry to all of the Librarys collections at the catalogue entry level. There are also separate manuscripts, maps, music, oral history and pictures catalogues. These are subsets of the NLA Catalogue that provide enhanced access to the collections through more extended indexes. The Pictures Catalogue (formerly Images1) is based on a separate metadata repository derived from Pictorial Manager.
The Digital Collections Manager must be capable of interfacing with the Pictures Catalogue by placing an export file in a designated directory on the Librarys LAN at regular intervals containing all picture records that have been added, updated or deleted since the last file was generated. Ideally, the export file for the Pictures Catalogue should be updated at the same time that the Digital Collections Manager is updated, but the Library will accept a service level of once a day.
In order to support the generation of the Pictures Catalogue from the Digital Collections Manager, a full work description will be passed through from the ILMS to the Pictures Catalogue.
There may be a future requirement to support additional special materials catalogues for manuscripts, maps, music and oral history through the same process. There may also be a future requirement to support a resource discovery service that provides access to all of the Librarys digitised collections at primary object level. Both of these needs must be capable of being met by the same process as for the Pictures Catalogue.
These specifications have not taken into consideration the data migration or resource discovery needs of collections other than the pictures collection. As delivery systems are developed for the other special materials collections, there may be a requirement to extend the number of work description elements and to migrate all the unit-level records for the collection from the ILMS.
Current status: The interface between the Digital Collections Manager and the Pictures Catalogue has been implemented but will not be needed in the longer term. The Librarys new ILMS will support the delivery of catalogue interfaces for the special materials subsets with the same level of functionality as the current Pictures Catalogue. A decision has been taken to derive all of the catalogue subsets from the ILMS. Therefore, there is no longer a requirement to generate additional special materials catalogues from the Digital Collections Manager.
Where a work needs to be replicated in order to fulfill an order, the Library will create a master copy to the standards set by its Digitisation Projects and upload and manage the master copy and its derivatives through the Digital Collections Manager. It is not intended that the Digital Collections Manager will replace existing manual (or future automated) processes for managing orders. However, it will be possible to trace the progress of orders that require a master to be created, via an Order Number in the process record. A use case has also been defined in this specification for downloading copies of master files to a designated directory for copying to an appropriate carrier or delivery online.
Current Status: This level of support for orders has been implemented.
The Library maintains a range of manual rights management systems in paper and card files. The Digital Collections Manager will enable coded rights management information and expiry dates to be stored at any level in the hierarchy. This information will be sufficient to suppress the display of metadata and/or derivatives in delivery systems or to make display conditional on domain address. However, there will be no support in the first release for linking this rights management information to rights holders or varying it based on user category, as specified in the original Digital Services Project Information Paper. PANDAS has some additional support for recording the details of rights holders. Full support for rights management is planned for a future release. At that time it will be particularly important to align the data models of PANDAS and the Digital Collections Manager to ensure an appropriate rights management framework and language can be set in place to meet the Librarys needs across all applications.
Current Status: High level specifications have been developed for the Rights Management System and it has been scheduled for release after DCM V1.2.
The Library publishes a range of online exhibitions and other multimedia publications via the NLA Web site. These are (or will be) catalogued and have their own archiving and preservation management requirements, which will be addressed either through PANDAS or through the Librarys future website archiving strategy. Where digital copies are created of a work for exhibition or publication purposes, the Library will create a master to the standards set by its Digitisation Projects and upload and manage the master copy and its derivatives through the Digital Collections Manager.
It is not intended for the Digital Collections Manager to operate as a pre-press system for the Librarys multimedia managers or that copies of multimedia publications will be uploaded and managed through the Digital Collections Manager. Multimedia managers will use the persistent identifier of the appropriate derivative or structural map to create their own contextual and interpretative displays. Alternatively, they will create derivatives from a downloaded version of the TIFF master to their own standards, as any other publisher.
Online exhibitions currently exist that may contain images of works in the Librarys collections that have not been digitised and assigned persistent identifiers according to the Librarys current digitisation policies and guidelines. These works will be brought under the Digital Collection Managers control as priorities dictate. The Raper Collection is a significant example that may need early attention.
Current Status: Support for the management of digital objects derived from non-NLA collections will be included in DCM V1.2.
This is an in-house system used to manage sound recordings. The Library plans to migrate the functionality supported by this system to the Digital Collections Manager as part of a later phase.
Current Status: This task has not yet been scheduled.
Quadriga Sound System
This is a third party system used to migrate audio material from analogue to digital format or to create and manipulate digital audio files. Master audio files will be stored in BWF format and include a full set of sound preservation metadata. The Quadriga system can import and export metadata to standard XML files. In the first instance, descriptive metadata for works migrated from analogue to digital format via Quadriga will be derived from the Oral History Stocktake Database. The first release of the Digital Collections Manager will not deal with sound. However, there will be a requirement in a later phase to migrate BWF and sound files generated by the Quadriga Sound System to the Digital Collections Manager. Once the Digital Collections Manager has been enhanced to support sound, sound process managers will interact with the Digital Collections Manager to upload BWF and sound files through the range of ad-hoc and batch workflows already defined for works in other formats.
Current Status: This task has not yet been scheduled.
The Library has a Radius Authentication Server in place for authenticating access by staff to the LAN and other networked services, including http authentication for Intranet services. The Digital Collections Manager will support a Log In use case that enables staff to use the same userid and password for accessing the Digital Collections Manager that is used for accessing other services. Radius supply a Java client for querying the authentication server. The Digital Collections Manager will still need to support a user object for each user that records user roles and profiles but passwords need not be stored. User roles and profiles for the Digital Collections Manager (other than passwords) will be managed by members of the Administrator role.
Current Status: This has been implemented.
The Library has a Directory System that records staff names, sections, roles, telephone numbers, email addresses etc. These details are currently stored in an Oracle database but the Library plans to migrate them to an X.500 directory server so that third-party software which requires information about library staff can query the directory using the LDAP protocol. The Digital Collections Manager will not be required to support X.500 or LDAP protocols at this stage but it should be a strategic direction for the Application Framework.
Current Status: No change.
This section defines the objects needing to be supported by the Digital Collections Manager and their properties. An entity-relationship model is used to detail the requirements.
Obl = Obligation (M = mandatory, O = optional, C = conditional)
Occ = Occurrence (S = single, R = repeatable)
Requirements for repeatable fields are included but could be met by storing values in a single field separated by semi-colons.
The following sections define each entity in the Entity-Relationship diagram and their required attributes (metadata fields).
Dublin Core mappings are provided for the Work and Copy entities in case a decision is taken to support the descriptive properties of these entities in the Application Framework through an implementation of Dublin Core.
All entities must have the following record status attributes or equivalent:
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| dateTimeCreated | Date and time the record was created | M | S | Numeric | 19 | Date/Time | Oracle date |
| recordCreator | Agent responsible for creating the record | M | S | char | 38 | Identifier | User: userid |
| dateTimeUpdated | Date and time the record was last updated | M | S | Numeric | 19 | Date/Time | Oracle date |
| recordUpdater | Agent responsible for last updating the record | M | S | char | 38 | Identifier | User: userid |
Anything that is a target for collection by the Library. It may be a book, manuscript, map, picture, sheet music, serial or sound recording.
1. A work may be the subunit of just one other work.
2. A work may have as subunit one or more other works.
3. A work must be either a set or item or part.
4. An item must be the subunit of just one set.
5. A subunit must be of just one subunit type.
6. A part must be the subunit of just one item.
7. A work must have at least one copy of copyRole original.
8. A work may have two or more copy(s).
| DC | Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
|
identifier |
workPID | The persistent identifier assigned to the work | M | S | char | 38 | Text | This can be used as the unique key for this work |
| altworkPID | An alternative persistent identifier assigned to the work at some stage and needing to remain persistent | O | R | char | 38 | Text | ||
| immutable | The unique identifier of the catalogue entry, for records sourced from the ILMS. | C | S | char | 38 | Text | Mandatory if recordCreator=ILMS | |
| workNo | A unique number associated with a work for collection management purposes | O | S | char | 38 | Text | ||
| subunitNo | A distinguishing number for the subunit intrinsic to the arrangement of the parent work , eg page number. | O | S | char | 10 | Text | I, ii, iii; 1, 2, 3, A, B, C, etc. | |
| standardID | A standard identifier such as ISSN, ISBN or ISMN assigned to the work. | O | R | char | 38 | Text | ||
|
type |
collection | A code identifying the collection to which the work belongs. | M | S | char | 7 | Code | Enumeration: nla.bk (book); nla.ms (manuscript); nla.map (map); nla.mus (music); nla.pic (picture); nla.ser (serial); nla.oh (sound) |
| form | A code identifying the form of the work if different from that implied by collection. | O | S | char | 7 | Code | Enumeration: nla.bk (book); nla.ms (manuscript); nla.map (map); nla.mus (music); nla.pic (picture); nla.ser (serial); nla.oh (sound) | |
| bibLevel | A code identifying the bibliographic level of the work | M | S | char | 4 | Code | Enumeration: set, item, part | |
| subunitType | A code identifying the type of subunit. | M | S | char | 4 | Code | Codes will vary based on collection and bibliographic level. (See separate subunitType object) | |
| digitalStatus | A code identifying whether the work or any of its descendents has been digitized | M | S | char | 1 | Code | Digitised; Non-digitised; Partially digitised. | |
| Local | digitalStatusDate | Date the digital status was assigned | M | S | num | 10 | Date | Oracle date |
|
title
|
title | The name given to the work by the creator or publisher. | O | S | char | 254 | Text | |
| otherTitle | Another name by which the work is known. | O | R | char | 254 | Text | ||
|
creator
|
creator | The entity primarily responsible for creating the work. | O | S | char | 254 | Text | |
| creator Statement |
A statement of responsibility associated with the work. | O | S | char | 254 | Text | ||
| background | Biographical or administrative history note | O | S | char | 4000 | Text | ||
|
subject |
subject | A subject heading that describes the topic of the work | O | R | char | 254 | Text | |
| addressee | For correspondence the party to which the correspondence is addressed | O | S | char | 254 | Text | ||
| occupation | A term that describes the occupation of the subject. | O | R | char | 254 | Text | ||
| classification | A character string that classifies the work | O | R | char | 254 | Text | ||
|
description |
edition | A means of identifying works issued at different times with substantially the same content | O | S | char | 254 | Text | |
| notes | A general note about the work. | O | S | char | 4000 | Text | ||
| summary | A summary of the scope of the work | O | S | char | 4000 | Text | ||
| contents | A note listing the contents of the work | O | S | char | 4000 | Text | ||
| publisher
|
publisher | The publisher, place of publication and/or date of publication of the work. | O | S | char | 254 | Text | |
| contributor
|
contributor | Another entity who has made a significant contribution to the work. | O | R | char | 254 | Text | |
|
date |
startDate | The first date associated with the work, expressed in date form for searching purposes. | O | S | Numeric | 4 | Date | layout: YYYY |
| endDate | The last date associated with the work, expressed in date form for searching purposes. | O | S | Numeric | 4 | Date | layout: YYYY | |
| format | extent | Physical description of the work | O | S | char | 254 | Text | |
|
relation |
parentPID | The persistent identifier assigned to the parent work | C | S | char | 38 | Text | Mandatory if work is a subunit |
| firstPart | For a multi-part item, the persistent identifier of the child work to be used as the thumbnail for the parent work and treated as the first part in displays | O | S | char | 38 | Text | If not used defaults to the first child in the sequence. | |
| series | A series of which this work is a part. | O | R | char | 254 | Text | ||
| childRange | A range of workPIDs assigned to children of the work | O | S | char | 100 | Text | Takes the form n/n/n -n/n/n The full workPID can be constructed by replacing "/" with "-" and adding the workPID of the top-level antecedent to the string | |
| indexes | Identifies indexes listing the contents of the work | O | S | char | 254 | Text | ||
|
coverage |
scaleEtc | Cartographic mathematical data | O | R | char | 254 | Text | |
| coverage | The spatial coverage of the work. | O | R | char | 254 | Text | ||
|
rights |
access | Restrictions on access note | O | S | char | 4000 | Text | |
| rights | Terms governing use and reproduction of the work | O | S | char | 4000 | Text | ||
| language | language | Information about the language of a work | O | S | char | 254 | Text | |
| local | index | A flag indicating to the Delivery System that an index should be generated for children of this work | O | S | char | 1 | Code | Enumeration: (name) |
A means of assigning a subunit type based on collection and bibliographic level.
1. A subunit type may apply to one or more work(s).
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| form | The form to which the profile applies | M | S | char | 7 | Code | Uses the work: form code set |
| subunitType | The subunit type, eg page, cover, series, index | M | S | char | 38 | Text | |
| bibLevel | A code for the bibliographic level | M | S | char | 1 | Code | Uses the biblevel code set. |
| isSupported | A tag indicating whether this type is relevant to the specified combination of collection and bibliographic level. | M | S | char | 1 | Boolean | yes; no |
A specific instantiation of a work.
1. A copy must be the instantiation of just one work.
2. A copy may be the input for one or more processe(s).
3. A copy may be the output of one and only one process.
4. A copy may be a file.
| DC | Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
|
identifier |
copyPID | A persistent identifier (PID) assigned to the copy | M | S | char | 254 | Text | |
| altCopyPID | An alternative persistent identifier assigned to the copy at some stage and needing to remain persistent | O | R | char | 254 | Text | ||
| currentVersion | A code indicating whether this is the current version or not | M | S | char | 1 | Boolean | Enumeration: Y,N | |
| version | The version number of the copy | O | S | Numeric | 10 | Pos. integer | Enumeration: 1,2,3 | |
| accessionNo | A unique number assigned by the Library to a copy when it was accessioned. | O | S | char | 38 | Text | ||
| callNo | A number uniquely identifying the physical location of a copy | O | S | char | 254 | Text | Digital copies may have a physical location if stored offline. | |
| otherNo | Other numbers associated with the copy | O | S | char | 38 | Text | ||
|
type |
copyType | A code distinguishing physical from digital copies | M | S | char | 1 | Code | Enumeration: p(physical), d (digital) (note: digital includes physical format digital publications) |
| copyRole | A code distinguishing the role played by a copy in use of the work | M | S | char | 2 | Code | Enumeration: o (Original); t (Thumbnail); v (View); e (Examination), m (Master); c (Co- master); a (Archive); s (Summary); t (Transcript); fa (Finding aid); mf (Microform); sd (Special delivery); p (Print) | |
| bestCopy | A code indicating this copy is preferred for delivery purposes | O | S | char | 1 | Code | Enumeration: (Y) | |
| date | dateCreated | For replicas, the date the copy was created | C | S | Numeric | 10 | Date | Process:dateStarted |
| source | sourceCopy | For replicas, the copyPID of the source copy | C | S | char | 38 | Text | Process:inputCopy |
|
description |
notes | A general note about the copy | O | S | char | 4000 | Text | |
| condition | A description of the condition of the copy | O | S | char | 4000 | Text | ||
| relation | exhibition | Information about exhibitions in which the copy has appeared | O | S | char | 4000 | Text | |
|
rights |
access | A code indicating any access conditions | O | S | char | 2 | Code | Unrestricted; Copyright; Indigenous; Adult content |
| expiry date | The year in which an access condition expires | C | S | Numeric | 4 | Date | ISO (YYYY) | |
| format | carrier | A code indicating the carrier when a work consists of one or more files | M | S | char | 10 | Code | Enumeration: online; CD-R; CD-ROM |
| DC | Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| identifier | filename | The assets file name | M | S | char | 254 | Text | Eg 1234001v.jpg |
|
format |
MIMEType | The MIME type associated with the file | M | S | char | 38 | Text | Eg image/tiff |
| fileFormat | The format of the file | M | S | char | 38 | Text | Should include format name and version number | |
| fileSize | The physical size of the file | M | S | Numeric | 38 | Positive integer | ||
| compression | The compression scheme used to store the file | M | S | char | 50 | Text | Default = none | |
| checksum | A value generated prior to or during load that is used to monitor file integrity | O | S | char | 38 | Text | ||
| checksumType | the type of checksum used, eg, MD5 | C | S | char | 38 | Text | mandatory if checksum is included | |
| encoding | The system used to encode the file content | M | S | char | 10 | Text | default=unknown; Enumeration: IBMPC; MAC; utf-8 |
| DC | Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
|
format |
x-resolution | The x-spatial resolution of the image, eg 300 ppi | M | S | Numeric | 10 | Positive integer | |
| y-resolution | The y-spatial resolution of the image, eg 300 ppi | M | S | Numeric | 10 | Positive integer | ||
| bitDepth | The pixel depth or tonal resolution of the image | M | S | Numeric | 10 | Positive integer | ||
| colourSpace | designates the colour space of the decompressed image | M | S | char | 20 | Text | Photometric Interpretation in the NISO draft | |
| orientation | A code designating the orientation of the image | M | S | char | 30 | Code | portrait; landscape | |
| imageWidth | The width of the image in pixels | M | S | Numeric | 10 | Positive integer | horizontal or x dimension | |
| imageLength | The length of the image in pixels | M | S | Numeric | 10 | Positive integer | vertical or y dimension | |
| colourProfile | A code indicating whether the Colour Profile resides within the image or at an external location | O | S | char | 1 | Code | within image; external | |
| CPLocation | The URL of the Colour Profile if stored externally | C | S | char | 254 | URI | Mandatory if ColourProfile=external | |
| zoomLevel | The maximum zoom level | C | S | Numeric | 2 | Positive integer | Mandatory if a Mr Sid file |
Text file properties will be defined in a future specification.
Sound file properties will be defined in a future specification.
Video file properties will be defined in a future specification.
A business initiative providing the authority for conducting a set of digitisation or other processes over time.
1. A project may be the rationale for one or more job(s).
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| projectId | Unique identifier of the project | M | S | char | 10 | Identifier | |
| projectName | The name given by the Library to the project | M | S | char | 254 | Text | |
| parentProject | The projectId of a parent project | O | S | char | 10 | Text | Used to group projects for reporting purposes |
| projectStatus | A code defining the status of the project | M | S | char | 15 | code | Enumeration: Registered, Started, Completed, On hold, Restarted, Aborted |
| DateStatusAssigned | Date status assigned | M | S | char | 8 | Date | ISO YYYYMMDD |
| dateStarted | Date the project was started | M | S | Numeric | 8 | Date | ISO YYYYMMDD |
| scope | A description of the scope of the project | 0 | S | char | 4000 | Text | |
| hostComputer | Computer and/or operating system in use at the time of the project | O | S | char | 4000 | Text | |
| notes | General notes | O | S | char | 4000 | Text | |
| URI | Link to a file containing a full description of the project | O | S | char | 254 | URI |
An event that causes a process or set of processes to be performed.
1. A job must generate one or more process(es).
2. A job must have its rationale in just one project.
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| jobNo | A running number assigned to the Job by the system | M | S | char | 10 | Identifier | agent-x |
| agent | A code for the agent initiating the job | M | S | char | 2 | Code | Digitisation Term; Imaging Team; Contractor1; etc |
| projectId | Identifier of the project providing the rationale for the job | M | S | char | 10 | FK | |
| jobType | A code identifying the type of job | M | S | char | 2 | Code | Enumeration: (Unassigned; Upload) |
| orderNo | An additional number for tracking commercial orders | O | S | char | 10 | Text | |
| customer | customer details for a commercial order | O | S | char | 254 | Text | |
| priority | a code identifying the priority of the job if not routine. | O | S | char | 1 | Code | Value: High |
| jobStatus | A code designating the current status of the job | M | S | char | 15 | Code | Enumeration: Pending; Requested; Submitted; Started; Completed; Acquitted; Aborted |
| DateStatusAssigned | Date status assigned | M | S | char | 8 | Date | ISO YYYYMMDD |
| Notes | Notes | O | S | char | 4000 | Text |
An event that uses a physical or digital copy as input and may result in transformation of the existing copy (eg, through repair, restoration or media transfer) or the creation of new copies (eg through digitisation, microfilming, photocopying or format migration).
1. A process must have as input one and only one copy.
2. A process may have as output one or more copies.
3. A process may have one or more sub-process(es).
4. A process must be initiated by just one job.
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| processed | Unique identifier for the process | M | S | char | 10 | Identifier | |
| jobNo | Identifier of the job which initiated this process | O | S | char | 38 | Identifier | Business-assigned |
| processType | A code itentifying the process type | O | S | char | 2 | code | Enumeration: (Unassigned; Replicate; Ingest; Derive) |
| processRole | A code distinguishing primary from secondary processes | M | S | char | 1 | code | Enumeration: (Primary; Secondary) |
| inputCopy | Identifier of the input copy | M | S | char | 254 | Identifier | |
| outputCopy | Identifier of the output copy | O | S | char | 254 | Identifier | |
| replaceCurrentVersion | A means of recording whether to replace the current version | O | S | char | 1 | Boolean | Enumeration: y; n |
| manipulation | A code indicating the form of manipulation applied to a Master to generate a Co-master | O | R | char | 1 | Code | Only applicable to Co-masters. Enumeration: (Cropped; Enhanced; Downsized; Rotated) |
| processStatus | A code defining the status of the process | O | S | char | 2 | code | Enumeration: (Pending; Requested; Submitted; Started; Completed; Failed; Acquitted; Aborted) |
| dateStatusAssigned | Date status assigned | M | S | char | 8 | Date | ISO YYYYMMDD |
| notes | notes | O | S | char | 4000 | Text |
A sub-process requiring use of a tool such as a scanner or camera or software.
1. A tool role must be used in just one process.
2. A tool role must use just one tool.
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| ProcessId | The process in which this tool role was used. | M | S | Foreign Key | |||
| ToolId | The tool used in this tool role. | M | S | Foreign Key |
A means of storing information about a tool.
1. A tool may be used in one or more tool role(s).
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| toolCategory | A code designating whether the tool is a device or software | M | S | char | 1 | Code | d (device); s (software) |
| toolType | A code designating the type of tool. | M | S | char | 38 | Code | Enumeration: transmission scanner; reflection print scanner; digital still camera; still from video; image capture software; OCR software; other |
| toolName | The manufacturer, name or number and version. | M | S | char | 254 | Text | |
| toolResolution | Maximum tool optical capture resolution. | O | S | char | 254 | Text | Eg 800x1600 ppi. |
| notes | Notes. | O | S | char | 4000 | Text |
A means of specifying what derivatives should be generated based on collection.
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| form | The form to which the profile applies | M | S | char | 7 | Code | Uses the work:form code set |
| copyRole | A code for the role of the derivative | M | S | char | 2 | Code | t (thumbnail); v (view); e (examination) |
| isGenerated | A tag indicating whether a derivative of this role should be generated for copies of works in the nominated collection | M | S | char | 1 | Boolean | yes; no; maybe |
| maxDimension | The maximum dimension for the derivative (eg, 150 pixels) | M | S | char | 4 | Numeric | (in pixels) |
| collectionPattern | The collection pattern to which the profile applies | M | S | Character | 254 | Text | |
| writeFooter | A tag indicating whether the persistent identifier should be stamped on the footer of the jpeg | M | S | Character | 1 | Boolean | yes; no |
| convertOptions | Parameters used in the program to create derivatives | O | S | Character | 254 | Text |
A means of specifying valid patterns for persistent identifiers.
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| pattern | A pattern for pi | M | S | Character | 254 | Text | In the form of a Perl regular expression |
| directoryLevel | A number indicating the directory level of the pattern. It is used to tell where the object will be stored in DOSS | M | S | Numeric | 2 | Positive integer |
A means of recording authority levels and default values for users of the system. Collection managers that do not have a User Profile in the Digital Collections Manager will still be able to login and perform collection manager use cases but they will not be able to perform roles requiring a higher authority level or to set default values until a user record has been created for them.
| Name | Definition | Obl | Occ | Datatype | Length | Form | Comments |
| userid | The users LAN userid | M | S | char | 8 | Text | |
| userRole | A code setting the users authority level | O | S | char | 2 | Code | lister; senior lister; finding aid manager; process manager; curator; administrator |
| collection | The users default collection. | O | S | char | 7 | Code | Uses the work:collection code set |
| Defaults | To be defined. | ||||||
| agent | The user's default agent. | O | S | Character | 2 | Code | Uses the job:agent code set |
| project | The user's default project. | O | S | IDREF | project instance | ||
| software | The user's default software | O | S | IDREF | tool instance | ||
| device | The user's default device. | O | S | IDREF | tool:instance | ||
| autoAcquit | A flag indicating whether the user is authorised to bypass acquittal | O | S | Char | 2 | Code | Values: Y |
A. Collection Manager Use Cases
B. Digitisation use Cases
C. Update Use Cases
D. Administration Use Cases
E. MARC-DCM Conversion Rules
| About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us |