Feasibility study for a digital Australiana resource discovery serviceThis document outlines the feasibility of a new resource discovery service for the National Library of Australia. PREPARED BY:
|
|
This goal was recently reinterpreted for electronic sources of information by the Electronic Information Resources Strategies and Action Plan 2001-2002, which describes the first stage of the process and endorses actions to be taken to achieve it.
The National Library's role in relation to electronic resources includes:
- collecting, preserving and making available Australian and other information resources in a wide range of formats, in accordance with the provisions of the National Library Act 1960;
- provision of services to support the Australian library system in areas such as resource discovery (bibliographic and metadata services) and directory services;
- contribution to international developments in library and information services in areas such as the development and promulgation of standards.
The term "electronic information resources" is defined as:
- digital copies of existing formats such as print, manuscripts, maps and pictorial items;
- born digital formats such as electronic journals and web sites;
- metadata describing the above electronic resources and resources in other formats.
5. What is meant by a resource discovery service ?
The National Library has defined resource discovery as 'bibliographic and metadata' services, but these are to be provided to the Australian library system, not to all Australians using the Web to access resources. The term 'resource discovery' has thrived as part of the Web generation of services [5], in answer to concerns about what search engines provide. The term is underpinned by an expectation of the creation and dissemination of metadata. Structured services built on metadata have made digital resources freely available to everyone with some guarantee of authenticity and reliability. Similarly to search engines, structured services are provided with an expectation that no mediation is required to use them.
Bibliographic services on the other hand are currently restricted to researchers, educators or learners who are either affiliated with an educational institution or are using the services within a library building - some form of payment has been made by or on behalf of the researcher or learner. In other words, they are restricted to those who are stakeholders in the Australian library system and access is possible only via authorisation or via an intermediary.
The National Library is attempting to open up direct client access to Kinetica for individuals such as independent scholars. It is also seeking public funding to underwrite access for as broad a client group as possible, but is unlikely to receive it given the current economic climate. The cost of hosting such services suggests that access to them cannot be easily reengineered, in a business sense, to support freely available resource discovery. The content in metadata services, however, could be made more obvious to all who value it via a resource discovery service and be repurposed to contribute to and maintain bibliographic services.
6. Is resource discovery already available?
Some Australian services already exist which support resource discovery to a certain extent, but they exhibit gaps in functionality which prevent the achievement of 'seamless access'. The gaps are created by collection scope, focus on a specific client group, and commercial imperatives. There are also gaps between the services, as discovery currently requires knowledge of the existence of the relevant entry points for each service. A full range of existing services is discussed in Appendix A.
7. New resource discovery initiatives
Newly proposed initiatives are even more far-reaching in their determination to provide federated access to electronic information sources, but not all combine it with information about print materials. A range of new initiatives is provided in Appendix B. Use of these services is generally restricted to specific client groups, leaving a niche to be filled for unaffiliated clients which could be met by a new resource discovery service.
8. Proposal for a National Library resource discovery service
The National Library will provide a resource discovery service for all Australians to access freely available Australian digital resources from a single Web entry point. The integrated service will meet general client requirements for fact-finding, learning, teaching and research. It will not be confined to a particular discipline and will be accessible with minimal functional constraints, for example, an authorisation overlay won't be introduced.
The resource discovery service will be used to support interoperability between a range of Australian content sources. For clients of Kinetica, the digital resources provided in the resource discovery service will be made available to the National Bibliographic Database to support record-sharing. The resource discovery service also will be a search target for other initiatives such as AARLIN, COLIS and SOCCI. [6]
Licensed or other forms of commercial content will primarily remain accessible via Kinetica. Some restricted content may be included in the resource discovery service at the metadata level only, for example sensitive material in PANDORA, but access to this material will be limited.
8.1 Strategic benefits to the National Library
The National Library would benefit from a national resource discovery service by being able to:
- achieve the stated goal of 'direct seamless access for all Australians to (print and) electronic sources of information' while strengthening the national information infrastructure; o identify resources suitable for archiving in PANDORA;
- identify resources for, and promulgate access to, the nation's intellectual record as represented in Kinetica and the Australian National Bibliography;
- demonstrate the value of digital preservation and persistent identification of electronic resources;
- move forward with new forms of digital service delivery; and
- meet serendipitous demands, i.e. those unexpected requests for business and functional connections which will emerge when a new service is operational.
8.2 Impacts of such a service on the National Library
- There will be impacts on the National Library's existing services, including:
- issues for the Kinetica business model;
- the costs of providing the new service;
- a need to rethink and add new linkages between various National Library services and the national metadata repository. Albeit a significant task in its own right, it would improve navigation between library services at the content level; and
- increased client expectations of enhanced access to all materials exposed via digital surrogates, such as manuscripts disclosed via finding aids.
9. Advantages and disadvantages of such a service to content contributors
Participating in a larger integrated service will provide benefits to content contributors such as:
- increased awareness of individual content services, similar to the way in which the PictureAustralia service brings attention to individual institutions' image collections;
- a new source of subject-specific materials for inclusion in subject-specific services; and
- an easy method of supporting cross-searching of Australian digital content.
Conversely, clients of content contributors may eschew the services provided by their institutions or use them less.
10. A resource discovery service client
A client of a national resource discovery service may be defined as any person with an interest in discovering and using Australian content for research or other information-seeking purpose. The person may or may not have access to an information intermediary (who may also be a client in the role of librarian, tutor, mentor, etc.).
A client may take the role of undergraduate or postgraduate student, academic researcher, educator, information intermediary, practising professional such as scientist, life-long learner (for example, at the University of the Third Age ), or a distance learner either remote from a learning environment such as a university campus, or offshore. The resource discovery service will allow the National Library to reach the growing number of independent scholars and fact-finding individuals. The expectations of clients are summarised in Appendix C.
11. How a national metadata repository and search service could meet client expectations [8]
A national metadata repository would be used to:
- increase exposure of selected resources, and thereby encourage the need to secure their longevity. This would in turn permit reliable persistent citation of electronic resources;
- disclose the rich information content of 'memory' organisations (such as libraries), teaching organisations (such as universities) and new services (such as e-print archives) to increase its use in teaching and learning;
- import/export metadata in bulk to other national services
hosted by the National Library or other service providers. As an example, the
user scenarios in the IMS Digital Repository project which would require
support from the National Library's services are [9]:
- Agent gathers a collection of related metadata, derives new metadata for the collection and stores it in a repository;
- Infoseeker discovers a set of resources some of which are learning objects in a single search and may access/use any of the resources regardless of their original context;
- Learner is set an activity in the learning path which triggers infoseeking activity; - consolidate and augment the scholar's portal developments with content from other authoritative sources which are relevant to the tertiary education sectors; and
- decomplexify the confusing brandscape [10] by federating the entry point for many rich information services into one. This would simplify linking to services such as the scholar's portal and also enhance awareness of the range of collections and services already available.
A complementary search service could meet client expectations by being able to:
- facilitate fast, more efficient access to Australian digital resources by providing an integrated, if not seamless, convenient one-shop stop approach;
- reinforce the value of a consistent search experience; and
- relegate search engines to an appropriate level of use.
12. High level use case scenarios
- Client uses search/browse facilities of the resource discovery service to select and link to digital content;
- Client uses a resource, which may include reading, bookmarking, printing, saving to a file, downloading, or sending the resource elsewhere via email;
- Client registers to use an alerting service.
13. Recommendations
The following recommendations are made as a result of the feasibility study:
- that the National Library proceeds with a prototype for a resource discovery service which provides access to freely available Australian digital resources via the World Wide Web for all Australians (a prototype will reduce the risk in creating a large scale service, and keep the participation barriers for content contributors low); and
- that the resource discovery service invokes a relationship with the Kinetica service which will facilitate integrated search access to print materials.
Appendix A: existing resource discovery services
Kinetica:
- The National Bibliographic Database which underpins the Kinetica service focuses on information about print resources. This is partly due to its longevity (11 million records have been catalogued over 20 years) and partly to a reluctance of NBD stakeholders to catalogue electronic resources onto the NBD (currently 65,000 resources or approximately 0.59%), although this is now changing. Access to Kinetica is protected by affiliation with an institution.
- Efforts to export records from the NBD for reuse in resource discovery services have proven not to be sufficiently scaleable to support single subject disciplines. [11]
IMAGES1, PANDORA, AJOL, OzLife, RAAM:
- The National Library has established separate services for access to electronic resources and information about them, including IMAGES1, PANDORA, and AJOL. Integrated access is provided at the entry point level via the National Library's home page on its Web server, but this could not be defined as 'seamless' because a different interface needs to be invoked to use each one.
PictureAustralia, Australian Digital Theses Programme, subject gateways:
- The National Library has assisted stakeholders in the community to create separate new services for access to electronic resources including subject gateways, PictureAustralia and the Australian Digital Theses Programme, without directly providing incentives for cataloguing onto the NBD. Nevertheless, the ADTP has met this commitment because there is a precedent to do so for print theses. The subject gateways are currently not supported by any federated access service, but are freely available. They would respond to similar incentives.
In single Australian institution hybrid libraries:
- Some NBD stakeholders also integrate access to print and electronic resources via their own OPACs, although not comprehensively. Licensing of content usually requires use of a separate interface to access each content source.[12] The OPAC services encompass links to electronic materials offsite and overlap in terms of resource access. The extent of overlap has not been extensively analysed.
In the United States:
- OCLC has recently announced the transformation of 'WorldCat from a bibliographic database and online union catalogue to a globally networked information resource of text, graphics, sound and motion' by centrally combining bibliographic and metadata records, some of which link to electronic resources, and some which provide access to print. [13]
- Centralisation by OCLC retains the commercial viability of the service for resource sharing. WorldCat is likely to remain available to licensed users only.
In the United Kingdom:
- The AGORA project created a demonstrator for the Distributed National Electronic Resource (DNER), to explore issues with scalability and interoperability. [14] AGORA is based on commercial VDX software from Fretwell-Downing. It supports a range of metadata schemas for learning objects and cultural assets.
- AGORA is considered to be appropriate for learning and teaching applications, and as such it was designed for use by the tertiary sector only. [15]
Appendix B: new resource discovery initiatives
In Australia:
- The Australian Academic Research Library Network (AARLIN) plans to allow 'university researchers to access Australian and international library information resources'. AARLIN has 19 partners in the university sector as well as the National Library.
- Information resources will be in print and electronic form, as evidenced by the cross-section of information targets proposed: Kinetica, subject gateways, OPACs with Z39.50, commercial databases, etc. Access will be restricted to affiliated researchers.
In the United Kingdom:
- In its newly proposed Distributed National Collection of Research Resources service [16], the United Kingdom is maintaining a clear separation between information sources for print materials and the Distributed National Electronic Resource service for sources of information about electronic materials.
- The partners in the DNC project are the British Library, the National Libraries of Scotland and Wales, and the JISC, and access is likely to be restricted to the tertiary sector. The DNER project partners include the DNC partners, and access is restricted to the tertiary sector. [17]
Between the United States/Australia/The United Kingdom:
- The IMS Digital Repositories Interoperability Working Group is focused on developing specifications that enable integration/interoperability of 'any knowledge assets used in the learning process' including courseware, assets of curatorial traditions such as libraries, museums, archives, and other sources such as news media repositories and government agencies. It assumes that information portals which may be used as a source of supply already integrate access to print and electronic assets.
- Australia's contribution to the project is to provide leadership to the standards development agenda, and deploy the software in its own COLIS project.
In Europe:
- The aim of the European Library (TEL), the Gate to Europe's knowledge, 'is to set up a co-operative framework which will lead to a system for access to the major national and deposit collections (mainly digital, but not precluding paper) in European national libraries' by 2003. [18]
- One output led by the British Library will be business plans and models 'to provide a realistic and sustainable service for implementation of digital library services and to maximise the benefits of co-operation. There will also be clarification of the partners' goals in exploitation of the service, which in turn affects the harmonisation of authentication and payment solutions.'
In the United States:
- In 1998, the Online Computer Library Centre commenced a new project known as CORC, the Cooperative Online Resource Catalog. [19] It seeks to become a reliable, international bibliographic database of records describing electronic information sources. The National Library of Australia participated in the project, but subsequent OCLC membership restrictions have made this unsustainable. The Distributed National Electronic Resource was also keen to participate in CORC by contributing its Resource Discovery Network records. [20] However, issues pertaining to the value of the RDN records, exemplified by the record matching and replacement routines, were not satisfactorily addressed by OCLC.
Appendix C: client expectations
Each type of client has different expectations of, and levels of skill with, using large-scale search services. These are briefly differentiated:
- Distance/Lifelong learner/independent scholar: increasing in number. May not be affiliated with an academic institution, or may be affiliated with several. Skill levels variable. Likely to be frustrated by access to expired content. [21]
- Fact-finders: a key client of a national metadata repository and search service, as other clients have primary access to other services. May not have recourse to an intermediary so is likely to appreciate a fast turnaround electronic reference desk. Time poor, and for this reason, more likely to require access to resources online. Unhappy with responses which suggest using printed materials. [22]
- Practising professional: this client is labelled in a broad way to reflect the individual who may or may not be affiliated with an organisation which supports a library. Probably has access to a network, but not a mediator. Time poor, and for this reason, more likely to require access to resources online.
- Mediator: may be looking for information for a client in situ, or may use a resource discovery service to find links to include in institution-specific guides.
- Undergraduate student: may not be highly IT literate, although this is likely to change. Often provided with lists of resources to study, so does not need to self-identify. Likes to use search engines because they provide simple search choices, they permit browsing, and they give access to resources immediately. Ignores print resources or is bothered by them if they appear; [23]
- Postgraduate: required to search print materials as well as indexes and databases and online sources. Has some IT literacy skills;
- Academic Researcher: Likes to browse print journals and/or monographs. Likely to have some experience searching highly structured indexes and databases in a particular discipline although only some disciplines now publish almost solely in electronic form (e.g., physics), others do not (e.g. religion). Finds email alerts re new resources valuable. Probably uses search engines, but doesn't admit it. Depending on discipline, will use serious discussion lists to present arguments, advise research results, and cite discussion list content in publications (for example, H-Net, an international discussion forum for Humanities and Social Sciences [24]). Uses informal networks to learn about resources. May self-publish online, which is content of interest to the National Library;
- Educator: would like fast and persistent access to relevant resources for teaching that are of high quality. Uses search engines to identify resources for undergraduates to develop their IT literacy skills, and show them how to differentiate authoritative resources. Finds email alerts re new resources valuable (a form of personalisation). More reluctant to use informal networks because their teaching content could be of a competitive nature. May rely on negotiated agreements between universities to access electronic serials which their own universities do not host.
CONTENT SELECTION
1. Introduction
The first part of the feasibility study showed that a national resource discovery service can meet the need to provide ubiquitous, timely access to Australian digital resources. Print-based services are unable to meet this need, except for people who have time to access a library or permission to access a library system. Print resources are sufficiently supported to meet the national requirement of documenting the nation's intellectual record. Digital resources are not yet sufficiently supported to contribute fully to this requirement. A first step in addressing this risk is to bring more attention to them by deploying a national resource discovery service.
2. What is content?
For the purposes of the feasibility study, electronic information resources are considered to encompass both assets of cultural heritage and scholarly output.
The term "electronic information resources" is defined as:
- digital copies of existing formats such as print, manuscripts, maps and pictorial items;
- born digital formats such as electronic journals and web sites;
- metadata describing the above electronic resources and resources in other formats.
A distinction needs to be made between content which is digital and content which is electronic, of which digital content is a component. Electronic items are defined to encompass hand-held or tangible media which work with computers such as CD-ROMs, oral history recordings etc. It is digital content only which is of primary interest to the national resource discovery service. Surrogates of print materials which do not link to the digitised versions of the originals are of secondary interest. One example is an online manuscript finding aid containing no content from the original work.
The national resource discovery service will be underpinned by a national metadata repository, where the metadata will be used to support discovery, and link directly to the digital content.
3. Inclusion criteria for content
- Content for a national resource discovery service may be selected on the basis of its value, its ease of use, its reusability, its quality, and its rarity. Selection will be supported by the following conditions:
- the ability of the client to access content easily, with no economic restrictions. In some cases, this may be achieved by waiving a licence. Pay-per-view services will not be included in the proof-of-concept, but could be linked in at a later time;
- the ability of the client to access content easily, with no functional restrictions. This may mean simply ensuring the content is online, or ensuring the availability of viewing software;
- the authenticity of the content provider and their ability to collaborate in a federated resource discovery service;
- Australian content which is considered to be valuable but might not otherwise receive ongoing exposure due to marketing or other constraints;
- applicability to a subject discipline, which will lead to a clear benefit of joining up research areas that do not neatly fit into established subject classifications;
- Currency may be of importance to some users. In order to ensure reasonable currency, some link-checking routines for URLs will be instituted;
- accompanying reasonable quality metadata. Some sources of content, such as the National Repository of Finding Aids and the PictureAustralia service, include electronic information resources not managed by the library community. Their associated metadata may not be created using bibliographic guidelines. The addition of non-bibliographic sources of data will therefore require quality control compromises to be made or assistance from the National Library in meeting descriptive standards to ensure adherence to a reasonable level [25] or both;
- Capturing sources of quality metadata excluded from other services such as the National Bibliographic Database or OzAuthors [26], for example, from the publishing sector; and
- future Legal Deposit conditions for digital materials, which will guarantee a reasonable expectation of advance notification. The National Library may need to consider purchasing access to attractive content for the service. If a critical mass of content cannot be negotiated, the NLA would not be able to pursue the resource discovery service.
4. Candidate content
The resource discovery service will primarily be concerned with collaborative sources of content, where contributing databases are compiled by several agencies and/or individuals contributing to a single brand.
The metadata sources will be:
a) Services hosted by the National Library:
Australian Journals Online (AJOL) **
Kinetica: digital Australiana (including PANDORA)
**
Online newspapers (as part of AJOL)**
Rare maps
b) Services hosted by the education sector:
Australian Digital Theses Program (ADT) **
Discipline-specific subject gateways: Agrigate,
AVEL **, AVELSD **
Education Network Australia
(EdNA) Online **
E-print archives (Australian
National University)
c) Services hosted by the cultural sector:
Australian Museums and Galleries OnLine (AMOL) **
d) Services hosted by the government sector:
Government online publications, via Kinetica **
e) Links to a range of services:
Google search engine**, the COLIS example of an IMS
digital repository, Schools Online Content Curriculum Initiative (SOCCI),
MusicAustralia (proposed) and PictureAustralia **
** denotes the sources to be tested in the proof-of-concept.
It is assumed that other sources of content including Bright Sparcs, Australian Science at Work, and World War I dossiers held by the National Archives would be better sourced from a single unified service such as the proposed National Archival Database. Existing aggregations such as PictureAustralia will be queried rather than gathered.
It is also worth noting that some of the preferred protocols such as the OpenURL may not be able to tested in the proof-of-concept. Further time will be required for them to be beta-tested.
5. Content currently out of scope
Full-text indexes
The demand for search access to full-text is likely to increase. Searching of full-text is considered to be a value-added component of commercial databases, where the high level of control over content negates the risk of false matches. Clients pay a high premium to use these services.
A repository of structured metadata could also be used to support indexes of full-text content of Australian electronic publications. However, this function will be addressed either by the National Library in other systems or in systems hosted by the content contributors themselves. The National Library could consider capturing full-text content of interest and providing an entry point to it (in the form of metadata) in the resource discovery service. [27]
Lack of search access to full-text should be partially offset by the ability to link directly to it.
Digital surrogates
The analysis of client expectations in the first part of the feasibility study showed that levels of frustration are likely to rise if a substantial portion of the resource discovery service does not give direct access to digital content. Nevertheless some metadata for print content from the following services would be worth including to increase their exposure:
Australian Historic Records Register
(AHRR),
Government Information Online Readers'
Guide (GIORGe),
Register of Australian Archives and
Manuscripts (RAAM),
and other sources as they are
identified.
ARCHITECTURAL OVERVIEW
1. Introduction
The range of query and data formats to be supported by the national resource discovery service will reflect the characteristics of the content selected. It is not considered feasible, given the diversity of resources to be included, to expect cultural and academic agencies to support a single standard for each function of description, search & retrieval, and interoperability. The service must also be able to accept new sources of content as they arise, and to contribute to other services as part of the national information infrastructure.
2. Placement in the national information infrastructure
Any service developed by the National Library must be cognisant of its place in the national information infrastructure. The National Library has already spent millions of public dollars on a range of online services including Kinetica with document delivery, PANDORA, the Pictures Catalogue and PictureAustralia. In addition, key products such as the Inter-Library Resource Sharing Directory, the Australian Libraries Gateway, the Oral History Directory and the Maps Directory are measures of our online outputs. Many of these services and products have been developed or enhanced to provide access to Australia's cultural heritage and scholarly research in the online environment, but more work needs to be done to effectively reposition and integrate them in the future. It is also essential to consider whether all services and products need to be retained in a new operational environment.
The National Library is working with many other information service providers to ensure that new or enhanced services complement other national and regional initiatives. These considerations have influenced the development of this new architectural model. The National Library has a strong track record in providing centralised services, but is aware that interoperability is a key component to future success in service provision. It also seeks to exploit existing services such as Kinetica to redeploy rich resource descriptions already created by Australia's information service providers.
There is a strong expectation that this type of service cannot be provided by others, not from a technical viewpoint but rather from a belief and sense of ownership in Australiana and Australia's research output. [28]The fact that Australia has equally viable systems such as Kinetica suggests that there is sufficient expertise and interest in reshaping our own services to meet new needs.
3. Other models reviewed
a) Svesok (Swesearch) and associated
Kulturarw3 Heritage
Project (http://www.kb.se/kw3/ENG/Description.htm)
b) Renardus,
(European)
c) the
Distributed
National Electronic Resource, (United Kingdom)
4. Options for the architectural model
Three options were considered for implementing the proof-of-concept for a national resource discovery service:
a) a fully distributed service based on Z39.50 and other
protocols;
b) a logical view of the National Bibliographic Database which
contains metadata for digital resources only, collocated with other
e-resources; and
c) a hybrid service, which combines distributed and
centralised elements.
Option 1
The first option to develop a fully distributed service is considered not to be sufficiently scaleable for the range of content sources proposed. Z39.50 operates most efficiently with similar sources of data, but even across a more homogenous set of services, this is known to be still hard to achieve. [29]
Choosing Z39.50 as the distributed search protocol also excludes potential sources of content beyond the bibliographic community which will never become Z39.50 compliant. The proof-of-concept will exclude this option.
Option 2
The second option to develop the service using a virtual slice of the National Bibliographic Database will not be considered in the proof-of-concept. While it is assumed to be technically feasible to interact with a logical slice of the NBD metadata in real time, the following business requirements take precedence:
- Kinetica serves a particular community;
- Kinetica requires an ongoing commitment from the National Library to support that community in unique ways, such as keeping the record of print and electronic holdings of Australia in a central facility;
- Kinetica places a high priority on a pay-for-use business model, and this business model should not be compromised without sufficient investigation;
- The National Library does not wish to compromise Kinetica performance by supporting a virtual resource discovery service in competition with its bibliographic utility function;
- Kinetica does not store records for all items which the resource discovery service will be expected to provide access to, such as article-level items.
These business issues will be revisited and informed by the parallel Australian National Bibliography service development. In the proof-of-concept, relevant valuable data hosted by Kinetica will be extracted for inclusion in the central metadata repository, and in return, metadata of interest will be uploaded into Kinetica for use by the library community.
Option 3: the preferred architectural model
The third option was recommended by the proof-of-concept Project Board as the preferred architectural model for the proof-of-concept. It will test a national resource discovery service based on a central metadata repository with several mechanisms for data collection, and integrated search access using some distributed search protocols. For example, access to large pre-existing data sources such as PictureAustralia will be provided through distributed mechanisms, such as Z39.50 or deep-level linking. The benefits of centralisation include:
- the potential to create a comprehensive inventory of Australia's digital output;
- acting as a convenient source of digital Australiana to include in the Australian National Bibliography via the National Bibliographic Database; o facilitation of record-sharing with other information providers;
- acting as a search target for scholars' portal services;
- providing a place for materials excluded by Kinetica's cataloguing policies to be recorded, such as article-level records; o supporting the ability to gather in the content of proposed new services, such as institution-based e-print repositories;
- deep linking to related services through the use of new protocols including the OpenURL.
A national gathering model which outlines this architectural option is appended.
5. Data collection and search access mechanisms will need to be
- flexible, to provide options for different content providers and allow various gathering schedules for different data sources;
- able to support incremental update, to ensure a minimal impact on source servers and national bandwidth; and
- capable of working over several different protocols.
The intended mechanisms are:
- Metadata gathering from key sources of content;
- Distributed Z39.50-based searching from compatible catalogues;
- SQL-based querying, including the application of the OpenURL protocol; and
- linking to search engines.
6. Metadata gathering standards
The national resource discovery service will support the following gathering protocols in the following order of preference:
a) the Open Archive Initiative Protocol for Metadata Harvesting,
to gather metadata from distributed repositories. The National Library will
conduct a trial with this standard by applying it to several in-house
repositories such as PANDORA and AJOL. The choice of the MHP permits the
addition of new sources over metadata over time, for example, from the QUT
e-print repository project and the COLIS project. The incremental nature of
this protocol allows a larger number of sources to be added with minimal
impact.
b) Dublin Core and extensions over http, to capture metadata
content from existing services such as the subject gateways which have
developed their own subject-specific DC-based profiles. It is assumed that some
MARC-based sources will be converted to the Dublin Core for inclusion.
Information service providers which have invested in the creation of MARC21 or
Dublin Core metadata may not become OAI-compliant for some time.
c) the
Australian Government Locator Service schema over http, to capture
publications' metadata content from the Web sites of government departments. As
the AGLS schema is based on the Dublin Core, gathering it should not require
much additional IT support.
It is assumed that all metadata will be provided and stored in XML.
7. Reasons for choosing these metadata gathering protocols
The Open Archives Initiative's Protocol for Metadata Harvesting has become the protocol of choice for a range of national services in development, including the United Kingdom's Distributed National Electronic Resource, internationally distributed e-print repositories, and proposed national cultural collections. [30] It has been selected because:
- it is an easy mechanism for exposing richly described digitised collections to harvesting, thereby increasing awareness of and access to special collections [31];
- it requires a modest technical investment compared to Z39.50;
- it is not restricted to bibliographic objects;
- it positions new services, such as institutional e-print repositories, for interoperation and federation;
- a high level of intended support and use has been expressed.
The Dublin Core and AGLS schemas will also be supported because:
- many desirable content sources are already described with this standard as a baseline;
- they are compatible with the OAI Protocol Metadata Harvesting; and
- they can be used to describe new sources of content which require mapping from a proprietary legacy schema.
8. Query and linking protocols
a) the DC-Libraries profile will be used as the common denominator
for the metadata in the central repository to facilitate more consistent
searching, contingent upon confirmation of its suitability.
b) the
National Metadata Repository will be a Z39.50 target for querying by other
national services, and could also use Z39.50 to query a modest number of larger
services. The number of services queried using Z39.50 will remain modest
because of well-known limitations in applying the protocol to a large number of
distributed services. [32]
c)
the OpenURL protocol will be used to direct queries to large-scale services
such as the proposed National Archival Database to be managed by the National
Archives of Australia and the National Database of Museum Objects managed by
Australian Museums and Galleries Online (AMOL). Search engines may also be
queried using this mechanism.
d) It is not yet known whether email
discussion lists will be accessible via any of these querying protocols.
Kulturarw3 rates these at a lower priority for inclusion. [33]
9. Interface functionality
a) keyword search
- against the
metadata in the central repository;
- in subject
categories, to test the validity of cross-searching;
b) browse function
- against the metadata in the central repository;
c) button services
- could activate the
OpenURL protocol, or an SQL search string;
d) subject category alerts
based on broad subject classifications;
- subject
categories are expected to be at the broad level only in the proof-of-concept.
While this may cause some issues, the overall concept should be reasonably well
tested at this broad level.
10. Preferred platform
For the proof-of-concept, the Metastar Enterprise platform has been recommended by the IT Division. In addition, some free software which supports the OAI protocol will be trialled.
This platform is unlikely to closely meet the full requirements of the new resource discovery service. However, the IT Division is currently assessing alternative large-scale platforms.
11. Issues for the NLA in the resource discovery service architecture
a) frequency of harvesting schedules;
b) currency of NLA
resources. This is described in the content section of the feasibility study;
and
c) where to provide suggestions for new materials to be
incorporated.
At least two services, PANDORA and AJOL, each provide a suggestions page for new resources. The pages contain different terminology, such as notification/contribution and electronic resource/online publication, and layout.
Formalising notifications into one page may require a resource to forward the suggestion to the appropriate service provider at timely intervals, unless software could be deployed to a particular e-mail address based on the type of resource being suggested.
EXPLANATORY NOTES FOR THE NATIONAL GATHERING MODEL
1. Filters
Some content sources contain a mix of material formats, others are content or medium-specific. In order to ensure a reasonable quality of content for the National Metadata Repository, the following filters will be applied:
a) The filter between the National Bibliographic Database and the
Australian National Bibliography will exclude non-Australian materials. The
Bibliography will contain digital and print-based Australiana, both scholarly
and cultural.
b) The records to be selected from the National
Bibliographic Database for the National Metadata Repository for Digital
Australiana will exclude print content, licensed content, and images (because
they will be linked in via PictureAustralia).
Metadata from the
National Metadata Repository of Digital Australiana will be migrated to the
National Bibliographic Database with the exclusion of article-level items. The
National Bibliographic Database will become the definitive source of all
relevant bibliographic metadata for older as well as current digital
Australiana.
c) The filter between the subject gateways, EdNA Online
and the National Metadata Repository for Digital Australiana will exclude
non-Australian materials. In addition, EdNA Online observes its own currency
policy (review at 12 months, sentencing beyond that).
2. Entry points
The records of the pictorial items in the National Library's collection are shown in this model to demonstrate the benefits of having multiple entry points to the same sources of content. The Pictures Catalogue has four entry points:
- The National Bibliographic Database,
- The Pictures Catalogue home page,
- The PictureAustralia service, and
- An Australian Digital Resource Identifier citation (which appears in the records of the three services) but provides a direct citable link to the item.
The extra entry points make the content available to a wider set of audiences. This is assumed not to take away the exclusivity of a particular service. If libraries want to maintain a marketing edge then it will be important to retain the original branding of each service, and this can be arranged at several points in the resource discovery service.
3. Repurposing of data
The metadata for images in the National Library's pictorial collection exists in two forms: MARC21 for the NBD representation and Dublin Core for the PictureAustralia representation. The MARC21 is repurposed in the Pictures Catalogue. Because of this, some services will duplicate some metadata - does this give other services a choice between linking or data-sharing, or does this just create a problem for the national services (the NBD, the NMR) in determining how to handle duplication in results sets?
For some sources of content, there is a choice for how they might be absorbed into other larger services. For example, PANDORA will become OAI-compliant, but MARC records for PANDORA are currently added to the National Bibliographic Database and can be re-extracted for the NMR. Online newspapers could be similarly comprehensively catalogued.
The OAI-compliance means that PANDORA, with some tweaking, could be used as a basis for the e-print repository.
4. Types of linking
Linking may be exploited at many levels in Web-enabled environment. Because the national infrastructure does not encourage close business relationships between individually branded services, the most common is a superficial link from home page to home page, or from metadata to a home page. The National Metadata Repository would like to support deeper linking of the following types:
a) an extension of any search to other repositories, using the
OpenURL protocol. For example:
if searching for a
person, try AMOL
if searching in a discipline, try
the appropriate subject gateway.
These may be effected for the end-user
by clicking on a button (an example of a button service) to activate a search
in real time. Other services may be introduced to the end-user by placing
buttons at strategic points in the interface.
b) similarly, a keyword
query may be referred to a search engine by using SQL or the OpenURL protocol.
c) links between the National Metadata Repository and the SOCCI and IMS
initiatives are currently shown as bi-directional. Until these services become
available, the preferred uni-directional link is not able to be determined.
d) distributed searching from other national services such as the
AARLIN scholars' portals, or from institutional library OPACs using Z39.50.
5. Functionality excluded from this model
Some functions are not shown in this model:
- copy cataloguing;
- document delivery;
- links with overseas information service providers, such as CORC at OCLC and creators of large primary datasets such as CSIRO;
- selection of content for PANDORA (input), and for the subject gateways (output); and
- the ability to find people in the information provision business. This type of directory service may best be managed by the scholars' portals.
In addition, some services are only newly proposed or are in early development stages:
- The National Archival Database;
- MusicAustralia;
- The national repository of finding aids;
- SOCCI; and
- COLIS, the IMS Digital Repository.
6. Abbreviations
AARLIN - Australian
Academic Research Libraries Information Network (Australian)
AMOL - Australian Museums and Galleries
Online
COLIS - Collaborative Online
Learning Information System
SETIS - Scholarly Electronic text
and Image Service (Australian)
SOCCI - Schools Online Curriculum Content
Initiative (Australian)
National Gathering Model
Diagram of the National Gathering Model
FOOTNOTES
[1] http://www.nla.gov.au/policy/electronic/resourcesplan.html[2] This is supported by a proposed Balanced Scorecard Initiative and the new policy of cataloguing all electronic materials in the NLA's Index and Databases service onto Kinetica for subsequent extraction.
[3] This term was first used by the NLA in 1999. The term is generic, and does not reflect any eventual branding which might be applied to a new service. The word 'system' is replaced by the word 'service' in this feasibility study, to encompass business as well as technical considerations.
[4] Scientific, Industrial, and Cultural Heritage: A Shared Approach, Lorcan Dempsey, July 1999.
[5] An Overview of the Dublin Core Metadata Initiative, http://dublincore.org/about/overview/, accessed 24th August 2001.
[6] Schools Online Curriculum Content Initiative (http://socci.edna.edu.au) accessed 4th September 2001; Collaborative Online Learning and Information Services (COLIS) Consortium, consists of Macquarie University partnering with the Universities of Newcastle, New England, Southern Queensland and Tasmania. The COLIS architecture may also be replicated by other universities, details available at http://www.colis.mq.edu.au/; the Australian Research Library Network (ARLIN), available at http://www.latrobe.edu.au/arlin/, accessed 24th August 2001.
[7] More information is available at http://www.u3acanberra.org.au/page2.htm, accessed 20th August 2001. [link broken Mar 2007]
[8] The ideas expressed here are a reflection of the vision for the DNER, a benchmark service, described at http://www.diglib.org/forums/fall00/dempsey_files/slide0001htm, accessed 20th August 2001.
[9] IMS Digital Repositories White Paper, v. 1.6, 2001, http://www.imsproject.org
[10] A term which describes the proliferation of branded online services which are marketed in competition with each other, as defined in http://www.ariadne.ac.uk/issue26/dner/, accessed 23rd August 2001.
[11] A recent test for AUSTLIT has resulted in the decision to search the NBD by keyword in real time for single records; RAAM records are extracted on a monthly basis according to material type, not discipline.
[12] The University of Queensland estimates that it provides access to 150 different databases via 150 different interfaces, September 2001.
[13] OCLC's Global Strategy http://www.oclc.org/strategy/, 2001, accessed 24th August 2001.
[14] The AGORA site is available at http://www.ukoln.ac.uk/services/elib/projects/agora/. Accessed 24 July 2006.
[15] Summative Evaluation of Phase3 of the eLib Initiative: Final report, May 2001, pp. 16-17.
[16] Co-ordinating the distributed national collection of research resources (http://www.bl.uk/about/cooperation/pdf/blhe-dnc.pdf), Report to the Research Support Libraries Group, accessed 24th August 2001.
[17] 'The JISC would like to provide wider access to resources in the DNER, however the economic and legal framework with which these resources are acquired or created affects our ability to do so.' http://www.jisc.ac.uk/dner/collections/strategy.htm#7, accessed 28th August 2001.
[18] http://www.europeanlibrary.org/, accessed 6th September 2001.
[19] http://www.oclc.org/research/publications/arr/1998/hickey/corc.htm, accessed 7th September 2001.
[20] http://www.dlib.org/dlib/december99/12dempsey.html, accessed 7th September 2001.
[21] A finding by the Information Services team in evaluating usage of AJOL.
[22] These traits are distilled from the Evaluation of the electronic mail reference service at the NLA, J. Ward, July 2001. In the companion Assessment of Reference Services at the NLA, 2001, there is a strong correlation between the demographic profile of highly educated clients and use of the NLA's reference services.
[23] A finding of the University of Queensland survey of usage of its Cybrary.
[24] H-Net is available at http://www2.h-net.msu.edu/, accessed 20th August 2001.
[25] A similar practice was invoked during the early provision of records to RAAM.
[26] The Ozauthors trial closed on 31st August, with 450 subscribers and 100 e-books, http://www.ozauthors.com.au/, accessed 14th September 2001.
[27] A description of the functionality which could be provided is at http://www.nla.gov.au/dsp/doms/domsrfq.html, section 2.5.
[28] An example of this was the experimentation with OCLC's CORC project.
[29] Metadata Harvesting and the Open Archives Initiative, Clifford A. Lynch, ARL Bimonthly Report 217, August 2001.
[30] Strategies for Building Digitized Collections, Abby Smith, Digital Library Federation, September 2001, http://www.clir.org/pubs/abstract/pub100abst.html
[31] A New Approach to Finding Research Materials on the Web, Council on Library and Information Resources, June 2000, http://www.diglib.org/architectures/vision.htm
[32] Metadata Harvesting and the Open Archives Initiative, Clifford A. Lynch, ARL Bimonthly Report 217, August 2001.
[33] Kulturarw3 Heritage Project, http://www.kb.se/kw3/ENG/Description.htm
| About This Site | Copyright | Privacy | Accessibility | Site Map | Site Search | Content A-Z | Contact Us |