National Library of Australia
Digital Services Project Information Paper

Extract from Chapter 1

16 December 1988

On 16 December 1998, the Library issued an Information Paper on its Digital Services Project to provide potential suppliers of products and services with guidance to the new requirement and to obtain industry comment. This document is an extract from Chapter 1 of that paper. The digital services strategies outlined in this extract remain a definitive statement of the Libray's system requirements for its digital services.

Table of contents

Background

A primary objective of the National Library of Australia is to ensure Australians have access to a comprehensive collection of Australian and other documentary resources. The Library strives to achieve this by:

  1. Collecting, organising and preserving Australian library materials and selected non-Australian publications for current and future access.
  2. Providing national and international access to the Library’s collections and other documentary resources through a range of targeted services to remote and onsite readers.
  3. Delivering improvements to the nation’s cooperative library and information network through collaborative projects and provision of resource sharing services.

For several decades, information technology has been an important tool for meeting the Library’s objectives. However, growth in use of the Internet as a publication medium and as a mechanism for service delivery has presented significant new challenges and opportunities not yet being adequately addressed by the Library’s current systems and services. These include:

  1. The increasing amount of material published only in digital form.
  2. Emergence of digitisation (creation of a digital copy of a physical object) as a cost-effective means of enhancing access to traditional library materials.
  3. Opportunities provided by the World Wide Web for electronic service delivery.
  4. Industry developments improving the ability of systems to support the Library’s objectives, including:
  5. Take-up by library system vendors of standard application protocols such as ISO/FDIS 23950 (Z39.50) for information retrieval and ISO 10160 /10161 for interlibrary loan.
  6. Widespread adoption of the Dublin Core metadata set as a simple and easy-to-use format for describing electronic resources on the Web.
  7. Growing recognition of the importance of the Standard Generalised Markup Language (SGML) as an archival format for machine-readable text.
  8. Development of a subset of SGML - the Extended Markup Language (XML) - as a standard format recognised by Web browsers.
  9. A move from prototypes and research-based systems to widespread commercial use of the Internet.

Strategies and Requirements

In response to the environmental changes identified above, and in order to meet its service objectives more effectively, the Library has developed a set of strategies specifically directed to the delivery of digital services. These are outlined below under the following headings:

  1. The Library’s collection.
  2. The Library’s catalogue
  3. The Library’s information services.
  4. National network services.

Requirements for new systems or services to support the strategies outlined are indicated at the end of each section. These requirements are also summarised at the end of the paper.

The Library’s Collection

Electronic Publications

The National Library Act requires the Library to “maintain and develop a national collection of library material, including a comprehensive collection of library material relating to Australia and the Australian people”. This mandate requires the Library also to collect electronic publications. Electronic publications are considered to include physical format digital publications such as CD-ROMs and computer disks as well as online publications stored using digital technology, such as e-journals and electronic conferences.

The Library will collect, organise and preserve Australian electronic publications within a framework of cooperation that recognises that other collecting institutions have responsibility for collecting and preserving certain segments of Australia’s documentary heritage.

For the last two years, the Library has been developing a strategy for the preservation of Australian online publications. The first step, at the beginning of 1996, was to develop selection guidelines. In the second half of 1996, the Library set up the PANDORA Project. PANDORA stands for Preserving and Accessing Networked Documentary Resources of Australia. During 1997 the Library began work to develop a proof-of-concept archive to provide long term preservation and access to significant Australian online publications. The archive is now operational. The Library is also developing a national model for the long-term preservation of Australian online publications and has begun to talk to the Australian State libraries about a collaborative approach to achieve broader coverage of the national output.

The Library has also undertaken significant policy development and some research in relation to the preservation of and access to physical format digital publications in its collection. The collection contains physically unstable media as well as an increasingly diverse set of software dependencies that complicate longer-term access. The Library has set up mechanisms to identify what comes into the collection, and interim procedures to record relevant information to support ongoing management of the titles. The Library's strategy for dealing with the problems of unstable media has been to copy titles from magnetic floppy disks to CD-R (CD-Recordable) with appropriate descriptive concordances. This does not address the software dependency problems, for which the Library has initiated a modest research program looking at software archiving, migration, and emulation.

System requirements:

  • The PANDORA proof-of-concept archive needs to be migrated to a permanent facility sized to deliver services at required levels of performance and scalable to accommodate future growth in business volumes and complexity.
  • Systems are needed to support the management of physical format digital publications, preferably integrated with the digital archiving facility.

Digitisation of Collection Items

For physical format material in the collection, the Library will use digital technologies where it is cost effective and legal to do so, to enhance both preservation and access.

Microfilm and photographic transparencies are still used as the Library’s main strategy for preserving the content of physical collection items in case of loss of the original. This is because the shelf life of digital file formats and storage media is still uncertain. However, the Library is actively exploring digital reformatting as a future preservation option when the technology permits this. Over the last three-four years the Library has also become increasingly involved in the creation of digital objects or “surrogates” to enhance access to the collections.

Since 1995, the Library has been operating a service, Images 1, that provides access to digital copies of items in the Library’s Pictorial Collection. The Images 1 database contains records with linked thumbnail and medium resolution images for approximately 17 000 collection items, with new items being added at a rate of over 5 000 a year. It can be searched directly through its own Web interface or accessed via a hypertext link in the NLA Catalogue record. Users may order a reproduction-quality copy of the image through a manual process. There are no online ordering facilities yet, pending system support for digital signatures and online payment facilities.

The Library routinely produces finding aids or guides to the papers and manuscripts of individuals, families and organisations deposited with the Library. Over 300 of these are now accessible in electronic form on the Library’s Web server and also through links from the NLA Catalogue record. Similar finding aids are produced for the Library’s oral history recordings (see Section 1.4.1.3 below). The Library is currently investigating the feasibility of migrating its existing electronic finding aids from HTML to the Encoded Archival Description (EAD) format. This is an SGML Document Type Description (DTD) developed especially for describing archival and manuscript collections.

The Library is working on a number of other digital projects, including:

  1. The Australian Cooperative Digitisation Project (the outcome will be a new digital service entitled Australian Periodicals, 1840-1845),
  2. A pilot project to assess the feasibility of digitising the rare map collection
  3. A proposal to develop a multimedia publication based on Cook’s Journal.

The Library provides a digitisation-on-demand service whereby the Library’s Document Supply Service scans print articles for delivery to customers over the Internet. Currently, the scanned file is discarded after the document is delivered. The Library is considering the benefits of storing these files for re-use.

Journal articles and conference papers indexed by the Library for the Australian Public Affairs Information Service (APAIS) and the Australian Medical Index (AMI) are routinely scanned, with the intention of making the scanned copies available to users via the Web, pending resolution of copyright issues. There are now about 40,000 files averaging six pages per file and the number is growing by 12,000 per year.

The Library recently handed over responsibility for the databases hosted by its former Ozline service to RMIT Publishing. Users now access these databases – including APAIS and AMI - using the Informit Online Service. The databases contain data describing the articles and papers only. When the scanned images are made available through hypertext links inserted in the descriptions, RMIT Publishing will be responsible for managing use of the database service and the Library for storing, delivering and reporting use of the linked digital objects. The need for a technical infrastructure that will support separate systems for resource discovery and digital object delivery also applies to the Library’s other digital services.

System requirements:

  • The current Images 1 management system need to be replaced to support digitisation as a routine process in ways that optimise the workflows of collection managers and minimise system support issues as new digitisation projects become operational.
  • Systems are needed that can exploit the EAD format to enhance access to the Library’s manuscript collections.
  • Systems are needed that will support management of the navigational, rights management and charging aspects of digital object delivery separately from the resource discovery service used to discover the existence of collection items.

Audio Recordings

The Library has an extensive collection of audio recordings consisting of around 32,000 hours of unique material. Some of the recordings are commercial publications but the Library itself has created or commissioned many of the items, which include interviews with eminent Australians, social history interviews, songs, dance music, recitations and stories. The Library has begun to use digital technology “in the field”, but most of the existing material is recorded in analogue form.

A project is under way to copy these audio tapes to digital masters for preservation reasons, with the analogue copies being kept as backup copies until the Library is sure that risks with digital reformatting have been resolved. About 3,000 hours of audio has been converted and is now stored on 3,500 CDs. Conversion is expected to proceed at a rate of about 2,500 hours per year. Sound preservation staff generate the digital version from the analogue original using sonic Solutions® digital audio workstations linked via a separate high speed network (Medianet®).

The Library lists its audio recordings in the Library’s catalogue but has also had to develop a database to manage technical information about the recordings. This is just one example of a number of small in-house databases created to address the specific curatorial needs of the Library’s special collections (see Section 1.4.2 below).

Transcripts in electronic form are available for many of the interviews and finding aids for the social history and folklore collections. The Library has begun to index the content of its audio recordings with the aim of enhancing access to the collection through an electronically linked combination of summary, transcript and sound.

There may be a future requirement to collect and store digital video files, although at present this is not a key priority for the Library.

System requirement:

  • A robust storage and management system is needed to integrate with existing audio management systems to support the program of migration of the Library’s analogue audio recordings to digital form and to support the Library’s ongoing program for the creation and collection of audio recordings.
  • Systems are needed to support enhanced access to audio recordings through an electronically linked combination of summary, transcript and sound.

The Library’s Catalogue

Like other libraries, the National Library manages its collection through a separate Integrated Library Management System that has its own public access catalogue containing approximately 2.2 million records.

This system was enhanced during 1997 to deliver Web access to the Library’s catalogue through an HTTP/ Z39.50 gateway. An HTTP/ Z39.50 gateway is software enabling the contents of databases that conform to the Z39.50 information retrieval protocol to be searched via a web browser using the HyperText Transfer Protocol (HTTP).

The current Integrated Library Management System provides a good level of support for managing the printed collections but does not yet deliver full support for the Library’s collections of manuscripts, rare books, maps, oral history, printed music and pictorial materials. These collections have special requirements for recording the provenance and condition of items, their preservation history, intellectual property rights and conditions of access that are currently met through a range of separate manual and automated systems.

Another limitation of the current system is that the catalogue supports only a single level of bibliographic description. This has meant that users have needed to come to the Library to discover material described in the catalogue at collection level only (for example, a set of photographs or the papers of an eminent Australian). The Web interface to the Library’s catalogue has partially addressed this problem, by enabling hypertext links to be inserted in the catalogue record to more detailed finding aids and listings. The Web interface is also the means by which library users gain access to the online electronic publications in the PANDORA archive and to digital copies of collection items.

The Library will maintain and develop its public access catalogue as a comprehensive listing of materials in its collection – both print and electronic - with hypertext links from the catalogue description to:

  1. A digital copy, transcript or summary where one has been created to enhance access.
  2. The digital object itself, for items in the collection published in electronic form.
  3. More detailed information about items catalogued at collection-level only.

The Library will provide access to more detailed information about the content of its manuscript collections through electronic finding aids in the Encoded Archival Description (EAD) format. For its pictorial collections, the Library will ensure that the system replacing the current Images 1 management system provides support for up to five levels of description for collections and sub-collections and that this information can be accessed from the catalogue through hypertext links.

The current integrated library management system supports the definition of separate logical views of the catalogue. These have been set up for journals and newspapers, manuscripts, maps, oral histories and folklore, pictorial material and printed music. The Library will continue to provide separate logical views for the special collections and plans to merge its Images 1 service with the pictorial subset when the technology allows it. (Currently Images 1 is operating as the digital object delivery system for the Catalogue.)

The Library will also develop separate digital services, guides, online exhibitions and other multimedia publications tailored to promote specific aspects of the collection. As a general principle, the Library will make digital content developed for these products accessible through the catalogue and vice versa.

System requirements:

  • Manual and automated systems developed in-house to manage materials in the Library’s collections that require special curatorial control need to be replaced with a single system that operates as a logical component of the Integrated Library Management System.
  • Systems are needed to support a fully hierarchical digital library data model for items described at collection-level only in the catalogue.
  • Systems are needed to support the delivery of services tailored to promote specific aspects of the collection through digital content shared with the public access catalogue.

The Library’s Information Services

The World Wide Web will be the Library’s primary strategy for providing both onsite and remote users with access to its collections and services. In March 1995, the Library launched its World Wide Web server. The Library’s web site now stores and provides access to around 5,000 HTML files relating to the Library’s policies, collections and services. The Library also uses the server to publish interpretative guides to the collections, directories, exhibitions, education services and multimedia publications.

With the catalogue now accessible through the Web, the Library plans to migrate reading room users from the current character-based terminals providing access to the Catalogue to an integrated workstation. The intention is to develop a single Web interface for both onsite and remote users. For onsite users only, the Library will deliver access to networked CD-ROMs through integrated workstations in the Library’s reading rooms, together with a range of subscription-based services including Kinetica.

The Library operates on the principle that there should be no charge for access to its collections, taking into account the framework for protecting the intellectual property rights of creators and publishers provided by the Commonwealth Copyright Act and the costs of reproduction and delivery. In accordance with this principle, where it is cost effective to do so, the Library will provide onsite users with free access to commercial electronic publications and network services.

The Library will strive to ensure protection of the intellectual property rights of the creators and publishers of content to which it mediates access, in line with fair use for research purposes.

In cases where value-added services require user registration or where access conditions or charges apply, the Library will develop its capacity to minimise the administrative aspects of service delivery for users.

System requirements:

  1. More robust management/administration tools are needed to support the growing number of files stored in Unix file directories on the Library’s Information Server.
  2. Systems are needed to support access to networked CD-ROMs through integrated workstations in the Library’s reading rooms.
  3. An integrated approach is needed across all services to user registration, authentication, online ordering, rights management, electronic payment and billing.

National Network Services

The cooperative nationwide library and information network encompasses a range of network services facilitating national access to Australian and other documentary resources. The Library is committed to the development of services that integrate access to electronic publications with access to print materials, and that extend access to the content of publications.

Strategies for achieving this include:

  1. Maintaining and developing the National Bibliographic Database (see Section 1.4.4.1) as a national union catalogue for both print and electronic publications in Australian library collections.
  2. Promoting the use of metadata to enhance access to information on the Internet that is of interest to researchers but is not catalogued in the National Bibliographic Database.
  3. Collaborating with state libraries and other collecting institutions to build services that integrate access to collections of special materials such as manuscripts, oral history recordings and pictorial material, that are unlikely ever to be recorded in the National Bibliographic Database.

To facilitate national access to physical format materials in Australian library collections, the Library will maintain and develop national library directory services and engage in projects that improve support for document delivery in a distributed environment.

National Bibliographic Database

Since 1981 the National Library has operated a National Bibliographic Database that lists Australian and overseas books, magazines and other materials in the collections of over 850 Australian libraries. This information makes possible the concept of a distributed national collection. Users of the National Bibliographic Database can search across the collections of Australian libraries using a single query and obtain access to items they locate through interlibrary loan. Libraries also use the National Bibliographic Database to share cataloguing information. Every time libraries contribute their cataloguing or locations information to the National Bibliographic Database they help strengthen the national library and information network which is a significant benefit to all Australians.

During the first half of 1998, the Library participated in a project (the ZedWeb Project) in collaboration with the Distributed Systems Technology Centre and the University of Queensland Library to develop a service linking Australian catalogues using Z39.50. The ZedWeb Project confirmed that the virtual union catalogue model is not yet able to compete with a service based on a central model such as the National Bibliographic Database.

Kinetica will deliver major improvements to the national library network when it is implemented by positioning the National Bibliographic Database to operate more fully in an open systems environment. The Library has already implemented a web interface for participants in its National Chinese Japanese Korean Service (CJKS) which provides full cataloguing support facilities for materials written in the Chinese (traditional and simplified), Japanese and Korean scripts, together with transliteration capabilities. In the first quarter of 1999 the Kinetica search service (Kinetica Web) will provide access to the National Bibliographic Database through a Web interface. The resource discovery infrastructure for the distributed national collection will then be in place

The Library will develop the capacity of the National Bibliographic Database to serve as a gateway to the distributed national collection by encouraging contributors to include links in catalogue descriptions to electronic publications and to digital copies of items they have created to enhance access to their own collections.

The National Bibliographic Database is a subscription service that has been targeted traditionally to libraries rather than direct to individual end users. The Library is exploring the most cost-effective mechanisms for delivering access to end users. A trial will be conducted in the Library’s own reading rooms during 1999 to assess the implications for development of the Kinetica search service for end user access.

System requirement:

None within the scope of this project.

Metadata Activities

The online electronic publications collected and brought under bibliographic control through the National Bibliographic Database will be a small subset of the information publicly available in the Australian Internet domain. The Library considers it has a national role in facilitating improved access to this information but it is still in the process of defining this role. It is collaborating at the international level and examining different models for providing access to Internet information that are being developed overseas. In addition, the following projects are in place to help the Library to clarify its strategies in this area:

  1. The Library is actively involved in the promotion of standards and tools for indexing the Web through initiatives such as the Metaweb Project (a collaborative project with the Distributed Systems Technology Centre, the Australian Defence Force Academy and Charles Sturt University).
  2. The Library is exploring mechanisms for enabling publishers to register electronic publications for archiving. This may take the form of an online template. Publishers may use this template to create Dublin Core metadata for their own copy of the publication as a by-product of the registration process. Publishers may also be able to apply for Uniform Resource Names (URNs) for their publications as part of an Australian URN service.
  3. The Library is considering how it might deliver access to the content of Australian electronic publications selected for archiving by the Library. The service provided will need to be scalable to include items from the distributed national collection of Australian electronic publications in the custody of other archiving partners.
  4. The Library is a participant in a range of projects to develop subject gateways to Australian electronic content such as the MetaChem (chemistry resources) and Agrigate (agricultural resources) projects. The Library's primary role in relation to these gateways is to facilitate the development of such services as part of the national library and information network and to make them visible and accessible through appropriate service entry points.
  5. The Library is considering the need to create a separate service based on the National Bibliographic Database to provide access to Australian publications only or to significant subsets such as Australian government publications or Australian electronic publications. This would facilitate integrated access to material in both print and electronic formats. For instance, service entry points could be developed for:
  6. Australian information (a service providing access to Australian publications in the National Bibliographic Database as well as digital services indexing Australian electronic resources)
  7. Australian Government information (a service providing access to Australian Government publications in the National Bibliographic Database as well as the Australian Government Internet Gateway).
  8. The Library’s traditional indexing services - the Australian Public Affairs Information Service (APAIS) and the Australian Medical Index (AMI) - are evolving to deliver access to both electronic and print materials, with scanned document images as the means of delivering access to material not available directly in electronic form.

System requirements:

  • Systems are needed to support delivery of a service indexing the content of the distributed national collection of Australian electronic publications, preferably integrated with services supporting registration of online electronic publications for archiving and forming the basis of an Australian URN Service.
  • Systems are needed to support the delivery of digital services that integrate access to electronic and print resources through both hosted database and gateway mechanisms.

Directory Services

As part of its mandate to facilitate resource sharing at a national level, the Library provides access to and ensures the ongoing currency of a range of directory resources. Traditionally this has been done through dissemination of printed products. However, in 1997, the Library made a database version of the Interlibrary Resource Sharing Directory accessible through its former Ozline service and via the Web. Then in early 1998, the Library implemented an Australian Libraries Gateway. A database that maintains information in one place about contact details, collection strengths, lending policies and special materials collections of Australian libraries and that can be updated directly by the organisations it describes, is central to the Gateway’s operation. When fully developed, the Australian Libraries Gateway will be a key component of the national library and information network, replacing the current Interlibrary Resource Sharing Directory.

The Library is conscious that directories will increasingly be based on distributed services rather than on centralised databases. For this reason, international directory service standards such as X.500 are likely to become more important in the cooperative development of library directories. The Library will aim to develop its technical expertise in these protocols, and it will consider how to apply them to the future development of the Australian Libraries Gateway and the national interlibrary loan network.

In the short term, the Library plans to develop the Australian Libraries database as a Z39.50-enabled central repository. This will require collaboration with other agencies at a national and international level to develop a data schema and transfer syntax for directories that can form the basis of both X.500 and Z39.50 implementation profiles within the library domain.

System requirement:

  • The Australian libraries database needs to be accessible as a Z39.50 target with support for a data schema and record syntax that is sufficiently granular and capable of extension to form the basis of future X.500 and Z39.50 implementation profiles within the library domain.

Collaboration with other Collecting Agencies

The library is currently exploring models for the development of specialist union lists which complement the holdings information in the National Bibliographic Database for selected categories of material to include materials in the collections of museums and archives as well as libraries. One such example that has already been implemented is the Register of Australian Archives and Manuscripts (RAAM). Contributors update the database through a variety of online and batch methods.

Discussions are now proceeding on the most appropriate strategy for implementing a service that would allow users to search across the pictorial collections of Australian cultural agencies. It is envisaged that digital services of this nature will most likely depend on a hybrid architecture that exploits a combination of a centrally hosted database and HTTP/Z39.50 gateway to ensure integrated access to all relevant resources.

The Library is interested in the opportunities provided by the Web for provision of access to thesauri and the possibility of cooperative thesaurus development. Plans for this are still in the conceptual stage.

System Requirement:

  • Systems are needed to support the delivery of digital services that integrate access to selected categories of material in the collections of museums, archives and libraries through both hosted database and gateway mechanisms.
  • The Library’s databases need to be enabled for access through standard protocols from a third party gateway service (where this is not the case already).
  • Systems are needed to support the delivery of digital services that provide access to thesauri using standard protocols and that support cooperative thesaurus development.

Summary of Requirements