Staff Papers

Smashing the Silos: Towards Convergence in Information Management and Resource Discovery

A paper presented by Warwick Cathro, Assistant Director-General, Information Technology, National Library of Australia at the Information Orienteering Conference, Canberra 5 April 2001


Introduction

In a recent paper1 Lorcan Dempsey describes libraries, museums and archives as "memory institutions" which constitute "an important part of the civic fabric, woven into people’s working and imaginative lives and into the public identity of communities, cities and nations".

Dempsey notes that libraries, museums and archives, which have always had shared research interests, are now moving into a "shared network space". They are using the World Wide Web to create network spaces through which they deliver databases, collection guides, exhibitions and digital surrogates of their collections.

It is easy for institutions to overlook the implications of the fact that their network space is shared. Institutions tend not to focus on the fact that "their" clients are actually the users of many services. These users do not wish to repeat their information search many times in different systems. To use Dempsey’s words, users wish to "refer to intellectual and cultural materials flexibly and transparently, without concern for institutional or national boundaries".

To meet the needs of our users, we must address the issue of convergence. At its basic level, convergence is about providing users with the power to search across categories of information resource that have traditionally been separated.

Convergence in its full sense addresses a wider set of issues. Our users need services which provide integrated access to:

  • both online and physical information resources;
  • collections, items within collections, and information fragments within those items;
  • both commercial and free information resources;
  • both significant and transient information resources; and
  • both local and remote information resources.

Providing discovery and delivery services which span these categories of information resource represents one of the great challenges for libraries and other memory institutions.

Lorcan Dempsey’s paper has been informed by recent experience of the electronic library (eLib) initiative in the United Kingdom. In attempting to devise projects to address user needs, the eLib initiative coined the phrase "hybrid information environment":

A hybrid information environment can be described as one where an appropriate range of heterogeneous information services is presented to the user in a consistent and integrated way via a single interface. It may include local and/or remote distributed services, both print and electronic. The environment will provide some or all of the following functions: discovery, location, request, delivery and use, regardless of the domain in which objects are held. Domains may include eg libraries, archives, museums, government.... The environment will depend on open systems and standard protocols2.

In Australia there have been significant recent achievements by our memory institutions in developing online web based services. Progress is being made in the digitisation of unique heritage materials. Web sites have been developed which offer rich interfaces to digitised collections, and sophisticated interpretation of collection resources. However, little has been achieved to meet user needs in the area of convergence.

It is a theme of this paper that technical solutions are at hand to address the problem of convergence in resource discovery. In fact, the obstacles to achieving convergence are not primarily technical - they are more managerial, organisational and attitudinal. What is needed is for people of key influence in the cultural sectors to grasp the importance of the convergence issue and to possess the will to solve it.

To begin with, though, this paper will draw attention to selected developments, initiatives and standards in order to indicate that promising technical avenues do exist.

The role of metadata

Sixteen years ago, Australia’s five most significant "memory institutions" (the National Archives, the National Gallery, the National Library, the National Museum and the War Memorial) formed a committee3 to examine how they might collaborate in providing access to their collections.

That committee - the Advisory Committee on National Collections - established a working group to examine the metadata which these institutions used to describe their collections, and to identify any commonality and overlap in that metadata.

The result of that effort was a publication entitled Major data elements. It listed 27 data elements which are of prime importance in describing collections. For each data element, it gave a definition, identified the type of material to which the element applied, suggested a preferred term and synonyms, and gave some examples4.

The publication of Major data elements did not generate any further collaborative action. Basically, it was an interesting idea that was ahead of its time. However, it is notable that 10 of its data elements correlate precisely with 10 of the 15 elements in the Dublin Core Metadata Element Set, developed over a decade later.

The Advisory Committee on National Collections was right when it identified commonality in metadata as an important area for study. It was right because integrated access to diverse materials is usually accomplished through services in which the metadata for those materials is searched simultaneously.

One potential barrier to integrated access is the variety of metadata standards used by different cultural sectors. The library community has seen a century of evolving standards practice, represented most obviously in the Anglo-American Cataloguing Rules (AACR) and the Machine Readable Cataloguing (MARC) specification. In the museum community, standards development has lacked the imperative provided by shared cataloguing in the library community. Nevertheless museum practice is becoming more formalised and standardized, represented through standards such as SPECTRUM5 (the UK museum documentation standard) and the CIDOC Conceptual Reference Model6, developed under the auspices of the International Council of Museums.

The CIDOC Conceptual Reference Model, developed in 1998, describes in a formal language the concepts and relations relevant to the documentation of cultural heritage. It is intended to serve as a basis for mediation of cultural heritage information and to provide the semantic 'glue' needed to transform localised information sources into a coherent global resource. It is being investigated by the Research Libraries Group as a means of providing meaningful, structured access despite a variety of descriptive practices7.

A current example of the use of metadata to advance convergence can be seen in the Picture Australia service, which is hosted by the National Library of Australia. Picture Australia is a cross-sectoral service: in March 2001 its collaborating institutions included a museum, a gallery, an archive and a government agency, along with six libraries.

Picture Australia is a resource discovery service which provides access to over 500,000 images hosted by the partner institutions. Resource discovery is integrated through the use of a central metadata repository which supports the search service. The metadata is harvested at regular intervals from the partner institutions. The search result sets make use of thumbnail images which are retrieved "on the fly" from the web sites of the partner institutions. Dublin Core is used as a common metadata standard.

It is appropriate at this point to make a few comments about Dublin Core. The Dublin Core Metadata Element Set was developed during 1995-96. Last year additional structure, in the form of element qualifiers, was added to the standard. The Element Set was always intended to support "simple resource discovery" for digital collections. It also provides a kind of "lingua franca", with the potential to integrate access to the digital services of a wide range of communities.

Dublin Core is a relatively crude standard which, it is generally conceded, is not under-pinned by a well designed data model. In a sense, though, its crudeness and simplicity have been its strengths. It has been found useful in supporting resource discovery in a variety of contexts. It has attracted the formal support of the Australian Government, being the basis of the Australian Government Locator Service (AGLS) standard. This has been matched by formal Government support in a number of European countries, including Denmark and the United Kingdom. Dublin Core has also been well tested by the museum community, and the results were published by CIMI (Computer Interchange of Museum Information) 8. The development of the standard has been supported by librarians, museum specialists, Government information specialists, academic communities, and the digital library research community.

Encoded Archival Description

The Encoded Archival Description (EAD) is another standard which has attracted support across traditional sectoral boundaries. It has exciting possibilities for facilitating convergence in resource discovery.

EAD is a structural markup standard. It defines a standard way of marking up a finding aid or collection description. It is a specific Document Type Definition within the SGML (Standard Generalised Markup Language) family of standards.

The simplicity and the power of the EAD standard springs from the fact that it "merely" defines structure in a textual document. Within the EAD framework, collections are described in an intellectual sense rather than a physical sense. That is, the description reflects the logical hierarchy of the collection, rather than physical components such as boxes or folders.

The Library of Congress, in partnership with the Society of American Archivists, is the maintenance agency for the EAD standard. As part of this role, the Library of Congress maintains an annotated list9 of current implementors of EAD. The list reveals many examples of the use of EAD to describe collections such as manuscript and pictorial collections in libraries, art museum collections (Berkeley Art Museum), medical archives (Johns Hopkins) and government archives (UK Public Record Office).

A particularly interesting example of the use of EAD may be found at the Natural History Museum in London10. The Museum’s collection includes over 60 million specimens, over 1 million volumes in its library, and over 500,000 items of original artwork. EAD will be used within the museum to solve problems such as this one:

Currently an enquirer who wants to know what the Museum has about a particular butterfly has to [search] several systems. [He or she] might discover that we hold the type specimen for that butterfly ... that we also hold a watercolour painting of the butterfly; that an example is on public display in the Museum’s exhibitions area; and that our library contains a copy of the published type description. All this research, however, would need to take place in separate systems.

The Museum is currently evaluating the suitability of both EAD (for collection-level descriptions) and Dublin Core (for item-level descriptions) as standards to support integrated discovery, while it continues to use a variety of systems to support the native descriptions of individual items.

This approach by the Natural History Museum provides a microcosm of convergence that we may be able to achieve on a national or even global scale. It is an exciting prospect.

To actually achieve convergence through the use of EAD, we need mechanisms for institutions to cooperate and to develop federated search platforms. A good example of such a platform is the one developed by the Research Libraries Group (RLG). RLG has archives and museums amongst its members, and can no longer be thought of as merely a library consortium. In 1998, RLG introduced its Archival Resources Service11, which enables users to locate finding aids that are distributed across the Web, and to study the entire finding aid content through a web-based search interface. Services such as this have the potential to create entry points to regional, national or global archival holdings across library, museum and archive boundaries.

Distributed search services

In a distributed global environment, institutions have little or no control over the technology choices made by other institutions with whom they may wish to collaborate, now or in the future. This is why open and platform-neutral standards are being increasingly adopted, and why technologies like Java are being used to connect data and applications across organizational boundaries12.

Another important type of integrating technology is a search broker – a set of client-based services that interface with external servers through standard search protocols. A broker allows a variety of distributed information resources to be searched with a level of semantic compatibility, and with a single consistent user interface. It may support a variety of standard search protocols (such as XML Query, LDAP, Z39.50).

An example of such a broker may be seen in "SA Central", a recently launched service of the South Australian Government13. This service supports searches of South Australian government information (based on AGLS metadata) but also allows the searcher to broaden the search to include other government information domains, as well as library catalogues supporting the Z39.50 protocol, and directories supporting the LDAP protocol.

Over the last two decades the library community has made a strong commitment to the Z39.50 protocol, and this is now the basis for distributed searching of library catalogues, union catalogues and indexing services. Possibly because of its complexity, Z39.50 has attracted little interest outside the library community, although the museum community, through the Consortium for the Computer Interchange of Museum Information, has developed the "CIMI Profile": a Z39.50 Application Profile for cultural heritage information14.

More needs to be done to explore the potential of search brokers to support convergence in resource discovery. For example, in Australia we already have in place two significant but sectorally limited sets of infrastructure supporting discovery of collections across multiple institutions. In the library community, we have the Kinetica service, coupled with the Australian Libraries Gateway and its affiliated directory services. In the museum community, we have Australian Museums On Line (AMOL).

It should not be too difficult to design a search broker that would launch searches of both Kinetica and AMOL, and return integrated result sets. This would be a great advance for researchers and other users who do not care if their information need is met by a library, a museum, or both. In fact, the difficulties in constructing such a service would lie more in the area of conflicting business models, since Kinetica charges for searches, while AMOL does not.

Stimulating effective collaboration

During the second half of the 1990s, there was an opportunity for the cultural arm of government to play an important role in fostering convergence in Australia. In its final years, the last Labor Government instituted a program called "Creative nation" which aimed to foster cultural expression through technologies such as CD-ROM delivery. That program was launched just as the potential of the World Wide Web was becoming apparent. The Coalition Government came to office in 1996 with a program entitled "Australia’s Cultural Network" (ACN) which received $10 million in funding over three financial years. Its objective was "to improve and develop online access to, and participation in, Australia's cultural endeavour, commerce, activities and events".

Furthermore, two councils of state and Commonwealth ministers - the Cultural Ministers Council and the Online Council - took an interest in programs to foster delivery of educational, cultural and government information through the Web. Under the auspices of these councils, a multi-state, multi-sectoral group, the New Technologies Working Party, was established. This Working Party was well placed to encourage the convergence of online information services across the museum, archive and library communities.

The central ACN web site has a useful facility to index and search across a large number of cultural web sites, and it also provides access to a number of event finders. Beyond the web site itself, the initiatives funded by the ACN program mainly comprised improvements to directory and discovery services, such as Australian Museums On Line (AMOL) and the Australian Libraries Gateway. These initiatives were explicitly mono-sectoral.

One of the segments of Australia’s Cultural Network, as originally conceived, was the Virtual Gallery Program, which was intended to support a series of online exhibitions articulating aspects of Australia’s culture. Each of these online exhibitions was to include digital items in a variety of formats: still images, moving images; audio files; and text. The items in the online exhibitions were to be digitised and described by a range of cultural institutions with collections relevant to the exhibition themes.

During 1996 the Virtual Gallery Program stimulated cultural institutions in different sectors to plan collaborative online exhibitions which would support public access to examplars of their distributed collections relating to a common theme. It was likely that the Program would have stimulated the institutions concerned to exchange information on digitisation techniques, common standards for indexing data, and approaches to copyright management. This planning effort came to nothing, because after the bids were submitted in 1997 the Virtual Gallery Program was abandoned.

In recent months the Government has initiated the development of multi-agency information portals, one of which will redevelop Australia’s Cultural Network. It remains to be seen whether this portal will be backed by policies designed to foster genuine convergence.

The New Technologies Working Party might have been able to support the development of a common infrastructure framework for access to cultural resources. If it had been provided with a significant budget, it may have been able to catalyse significant convergence projects. However, the budget was not there, and there was no support in the critical areas of administration for the use of the Working Party to promote convergence.

Barriers to collaboration exist, not only in the sphere of government administration, but also in the structure of the formal standards communities. Bodies such as the International Organization for Standardization (ISO) and Standards Australia are organized along industry lines in such a way that information management issues are dealt with in a range of different standards committees which have little effective contact with each other.

In Australia, there is a need to bring together representatives of the following communities to discuss intersecting issues of resource discovery and information management:

  • the library community;
  • the archives and record keeping community;
  • the museum and gallery community;
  • publishers;
  • the government information community;
  • education communities dealing with issues of metadata and online learning; and
  • the IT data management community.

Currently these communities are represented in Standards Australia though a range of different committees such as:

  • IT/19: Computer Applications - Information and Documentation;
  • IT/21: Records Management Systems; and
  • IT/1/32: Data Management and Interchange.

It is interesting to contrast this situation with the approach taken in CEN, the European Committee for Standardization. Within CEN, a body known as ISSS (Information Society Standardization System) deals with all information technology standards outside the telecommunications area. Standardization activity in CEN usually occurs through CEN Workshops. When a need for a standard is raised by a CEN member, a proposed workshop is described and the proposal is open for comment through the Web. The workshop will aim to bring all interested parties together. The outcome is a CEN Workshop Agreement (CWA) which can be developed rapidly and published by national standards bodies. A CWA does not have the status of a formal standard, but it does reflect the consensus of identified companies and organisations, and is often a first step towards a formal European standard.

The CEN model is attractive, because it brings together all interested parties across a range of sectors. It would be useful if representatives of Standards Australia committees, associated with the communities listed above, were to meet to discuss mechanisms to make the standards process in Australia more flexible.

Summary

Researchers, students and citizens who are seeking access to intellectual and cultural materials do not wish to be constrained by institutional boundaries in their discovery and use of information resources. If we have an interest in meeting their needs, we must address the issue of convergence.

Technical solutions are at hand to address the problem of convergence in resource discovery. These solutions may take advantage of standards such as the CIDOC Conceptual Reference Model, the Dublin Core Metadata Element Set and the Encoded Archival Description. They may utilise search brokers which interface with a variety of catalogues and databases through standard search protocols.

The obstacles to achieving convergence are not primarily technical - they are more managerial, organisational and attitudinal. What is needed is for people of key influence in the cultural sectors and in the standards communities to grasp the importance of the convergence issue and to possess the will to solve it.

Footnotes and references

1. Dempsey, Lorcan. Scientific, industrial and cultural heritage: a shared approach. Ariadne, Issue 22 (January 2000). <http://www.ariadne.ac.uk/issue22/dempsey/intro.html>

2. This phrase comes from MODELS (MOving to Distributed Environments for Library Services) Project, a UKOLN initiative funded by the eLib program. Russell, Rosemary; Gardner, Tracy and Miller, Paul. MIA Requirements Analysis Study: Hybrid Information Environments - Overview and requirements, UKOLN, Version 0.1 - Draft - 22 July 1999. <http://www.ukoln.ac.uk/dlis/models/requirements/overview/>

3. The Advisory Committee on National Collections was established in late 1982 to advise the portfolio Minister on matters affecting the "national collection". It issued a Report on the feasibility of an integrated computer-based catalogue information system. One of the recommendations of that report was that a study be undertaken to identify and define "data elements common to the National Collection institutions".

4. Major data elements: a study to identify data elements common to the ACNC institutions. Canberra: National Library of Australia for the Advisory Committee on National Collections, 1985.

5. SPECTRUM: the UK museum documentation standard. <http://www.open.gov.uk/mdocassn/spectrum.htm>

6. CIDOC Conceptual Reference Model. <http://www.geneva-city.ch/musinfo/cidoc/oomodel/>

7. Gill, Tony. Touring the RLG information landscape: the CIDOC Conceptual Reference Model (RLG focus 45, August 2000). <http://www.rlg.org/r-focus/i45tour.html>

8. Gill, Tony. The CIMI side of RLG (RLG focus 48, February 2001). <http://www.rlg.org/r-focus/i48cimi.html>

9. Encoded Archival Description sites on the Web. <http://www.loc.gov/ead/eadsites.html>

10. Thomson, Neil. Towards a whole-museum response: discovering the Natural History Museum’s collections. <http://www.cultivate-int.org/issue2/natural/>

11. Research Libraries Group: Archival resources Service. <http://www.rlg.org/arrhome.html>

  1. Berg, Cliff. The state of Java application middleware, Part 1. <http://www.javaworld.com/javaworld/jw-03-1999/jw-03-middleware.html>

13. South Australia Central web site. <http://www.sacentral.sa.gov.au/search/index.jsp>

14. CIMI Z39.50 profile. <http://www.cimi.org/old_site/downloads/CIMI_profile/profile.doc>

Back to the top